prometheus

- Prometheus on macos
- Prometheus data storage

Author: xy-241

content/Tools/Prometheus On Macos.md

@@ -0,0 +1,49 @@
+---
+Author:
+  - Xinyang YU
+Author Profile:
+  - https://linkedin.com/in/xinyang-yu
+tags:
+  - devops
+Creation Date: 2024-11-10, 14:50
+Last Date: 2024-11-12T13:38:05+08:00
+References: 
+draft: 
+description: 
+---
+## 1. Install Prometheus
+---
+- Install [[Prometheus]] with `brew install prometheus`, the default configuration file is located at: `/usr/local/etc/prometheus.yml`
+- Start Prometheus: `prometheus --config.file=/usr/local/etc/prometheus.yml`. This command runs Prometheus in the foreground, listening on port `9090`. Visit [http://localhost:9090](http://localhost:9090) in your browser to access the Prometheus Web UI
+- Alternatively, start Prometheus as a background service:
+	- Start: `brew services start prometheus`
+	- Stop: `brew services stop prometheus`
+
+>[!help] What is `brew services`?
+> Refer to [[Launchd]] for more details.
+## 2. Install Node Exporter
+---
+- Install Node Exporter with `brew install node_exporter`
+- Start Node Exporter as a background service: `brew services start node_exporter`. Node Exporter exposes metrics on port `9100`: `http://localhost:9100/metrics`
+
+### 2.1. Configure Prometheus to Scrape Node Exporter
+- Edit your Prometheus configuration file and add the following scrape configuration under `scrape_configs`:
+
+```yaml
+scrape_configs:
+  - job_name: 'node_exporter'
+    static_configs:
+      - targets: ['localhost:9100']
+```
+
+- Restart Prometheus with `brew services restart prometheus`
+
+## 3. Grafana Visualisation 
+---
+- Install Grafana with `brew install grafana`
+- Start Grafana as a background service: `brew services start grafana`
+	- Grafana will be accessible at: [http://localhost:3000](http://localhost:3000)
+	- Default login credentials: username - `admin`, password - `admin`
+- For more information refers to [here](https://grafana.com/docs/grafana/latest/setup-grafana/installation/mac/)
+- Add Prometheus as a data source, using `http://localhost:9090` as the URL
+- Import the ["Node Exporter Full" dashboard](https://grafana.com/grafana/dashboards/1860-node-exporter-full/) from Grafana Labs:

content/Tools/Prometheus.md

@@ -6,14 +6,14 @@ Author Profile:
 tags:
   - devops
 Creation Date: 2024-11-06, 14:39
-Last Date: 2024-11-07T16:51:48+08:00
+Last Date: 2024-11-12T13:36:05+08:00
 References: 
 draft: 
 description: 
 ---
 ## Abstract
 ---
-- The **standard monitoring tool** used in **container and microservices infrastructure**
+- The **standard monitoring tool** used in **container** and **microservices infrastructure**
 
 >[!question] Why we need Prometheus?
 > **Hundreds of interconnected processes** serve users in these environments. If one process fails, it can trigger a **cascade of failures in other processes**, making it **difficult to pinpoint the root cause** from the end user's perspective. Prometheus provides visibility into the application and infrastructure of these hundreds of processes, **enabling proactive identification of issues instead of reactive debugging**. With Prometheus, we can continuously monitor processes, receive alerts about crashes, and even configure alerts for predefined thresholds.
@@ -93,7 +93,12 @@ scrape_configs:
 - [[Prometheus]] evalutes the rules and push alerts to Prometheus alert manager which will fire the alerts to different notification channel like email 
 
 ### Prometheus Data Storage
-- [[Prometheus]] stores **time series data** on a local disk-based time series database, but it can also optionally integrate with remote storage systems. The data is stored in a **custom time series format**, so we can't write it directly into a [[Database Paradigms#Relational|relational database]]
+- [[Prometheus]] stores **time series data** in a time-series database (TSDB) on the local [[File System|filesystem]], but it can also optionally integrate with remote storage systems. The data is stored in a **custom time series format**, so we can't write it directly into a [[Database Paradigms#Relational|relational database]]
+
+>[!important]
+> By default, this data is stored in the **`data/`** directory within Prometheus's working directory. You can customize this location by modifying the **`--storage.tsdb.path`** flag when starting Prometheus.
+> 
+> To control how long Prometheus retains data, use the **`--storage.tsdb.retention.time`** flag. This flag accepts a duration value (e.g., **`15d`** for 15 days).
 
 ### Prometheus PromQL
 - The query format used to obtain data from the [[#Prometheus Data Storage]], used by data visulisation tool like Grafana to create nice dashboards