Skip to main content

Data and monitoring · Intermediate · 25 min

How to Monitor a Linux Server with Prometheus and Node Exporter

By Padding · Updated 16 Nov 2026
Ubuntu 26.04.1 LTS · Prometheus 3.16.0 · Node Exporter 1.12.1

You will install Node Exporter 1.12 and Prometheus 3.16 from the official release files on Ubuntu 26.04, run both as locked-down systemd services that listen on localhost only, and query your server's CPU, memory and uptime with PromQL.

On this page

You cannot fix what you cannot see. Monitoring means collecting numbers about your server, such as CPU use, free memory and disk space, and keeping them over time, so that you can answer "was it slow at 3 a.m.?" and "how fast is the disk filling up?".

Prometheus is a database for those numbers, called metrics, plus a scraper. Every 15 seconds it fetches a web page of current values from each target you list, and stores the results with a timestamp. Node Exporter is the small program that publishes a Linux machine's metrics on such a page: CPU, memory, disks, network and more. "Node" here means a machine, not Node.js. Together they give you history and a query language, PromQL, to ask questions about it. The tutorials on Grafana dashboards and Alertmanager build on this one.

You install both from the projects' release files. This is deliberate: Ubuntu's own packages are older (the archive carries Prometheus 2.53 and Node Exporter 1.10 for Ubuntu 26.04). The cost, covered in the step on upgrades, is that you now apply updates yourself. Both services listen on 127.0.0.1 only, and the later steps explain how to widen that safely.

Step 1 · Install the tools and set the versions

Install the tools this tutorial needs. Then record the versions and your CPU architecture in shell variables, so every command below uses the same release. Keep working in this terminal, or set the variables again if you open a new one:

Bash
sudo apt update
sudo apt install -y curl jq tar
PROM_VERSION=3.16.0
NODE_VERSION=1.12.1
ARCH="$(dpkg --print-architecture)"
echo "$ARCH"

dpkg --print-architecture prints amd64 on most servers and arm64 on ARM servers. Prometheus and Node Exporter publish release files with exactly those names. Check the current versions on https://github.com/prometheus/prometheus/releases and https://github.com/prometheus/node_exporter/releases before you start, and use newer ones if they exist.

Step 2 · Install Node Exporter

Download the release file and its checksum list into a work folder, then check the file against the list. The check catches a damaged or truncated download. It cannot prove who published the file, because the list comes from the same place, so download only from the official GitHub release address:

Bash
mkdir -p ~/monitoring-install
cd ~/monitoring-install
BASE="https://github.com/prometheus/node_exporter/releases/download/v${NODE_VERSION}"
curl -fsSLO "${BASE}/node_exporter-${NODE_VERSION}.linux-${ARCH}.tar.gz"
curl -fsSL -o node_exporter-sha256sums.txt "${BASE}/sha256sums.txt"
sha256sum --ignore-missing -c node_exporter-sha256sums.txt

The last command prints node_exporter-1.12.1.linux-<your architecture>.tar.gz: OK. If it prints FAILED, delete the file and download again.

Unpack it and install the one program you need. install copies it with the owner and mode in one step:

Bash
tar -xzf "node_exporter-${NODE_VERSION}.linux-${ARCH}.tar.gz"
sudo install -o root -g root -m 0755 "node_exporter-${NODE_VERSION}.linux-${ARCH}/node_exporter" /usr/local/bin/node_exporter
node_exporter --version | head -n 1

The output starts with node_exporter, version 1.12.1.

Create a system user that only runs this program. It has no password, no home folder and no shell, so a flaw in the exporter cannot turn into a login:

Bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin node_exporter

Step 3 · Run Node Exporter as a service

Write the systemd unit. --web.listen-address=127.0.0.1:9100 is the setting that keeps the metrics page private: without it, Node Exporter listens on every address, and anyone who can reach port 9100 could read details about your server:

Bash
sudo tee /etc/systemd/system/node_exporter.service > /dev/null <<'UNIT'
[Unit]
Description=Prometheus Node Exporter
After=network-online.target
Wants=network-online.target

[Service]
User=node_exporter
Group=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=127.0.0.1:9100
Restart=on-failure
NoNewPrivileges=true
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target
UNIT

The [Service] options run the program as its own user, restart it if it crashes, and stop it from gaining privileges, reading home folders or sharing /tmp. Start it now and at every boot:

Bash
sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter
sudo systemctl status node_exporter --no-pager

The status shows active (running).

Check it worked

Fetch the metrics page and pick out two values. node_load1 is the one-minute load average, and node_memory_MemAvailable_bytes is the memory that programs could still use:

Bash
curl -s http://127.0.0.1:9100/metrics | grep -E '^(node_load1|node_memory_MemAvailable_bytes) '

You get two lines, such as node_load1 0.08 and node_memory_MemAvailable_bytes 3.1e+09. They are plain text: a metric name and a number. Prometheus reads this format every 15 seconds.

Confirm it listens on the loopback address only:

Bash
ss -tln | grep 9100

The local address column is 127.0.0.1:9100, not 0.0.0.0:9100 or *:9100.

Troubleshooting

  • Failed to start: run sudo journalctl -u node_exporter -n 20 --no-pager. A typo in ExecStart or a missing user are the usual causes.
  • Address already in use: something else uses port 9100. Check with ss -tlnp | grep 9100.
  • Permission denied when starting: the binary must be executable by the service user. ls -l /usr/local/bin/node_exporter shows -rwxr-xr-x.

Step 4 · Install Prometheus

The Prometheus archive holds the server prometheus and a checking tool, promtool. Download, check and unpack it the same way:

Bash
cd ~/monitoring-install
BASE="https://github.com/prometheus/prometheus/releases/download/v${PROM_VERSION}"
curl -fsSLO "${BASE}/prometheus-${PROM_VERSION}.linux-${ARCH}.tar.gz"
curl -fsSL -o prometheus-sha256sums.txt "${BASE}/sha256sums.txt"
sha256sum --ignore-missing -c prometheus-sha256sums.txt
tar -xzf "prometheus-${PROM_VERSION}.linux-${ARCH}.tar.gz"
sudo install -o root -g root -m 0755 "prometheus-${PROM_VERSION}.linux-${ARCH}/prometheus" /usr/local/bin/prometheus
sudo install -o root -g root -m 0755 "prometheus-${PROM_VERSION}.linux-${ARCH}/promtool" /usr/local/bin/promtool
prometheus --version | head -n 1

The checksum line ends in OK, and the version line starts with prometheus, version 3.16.0.

Create its user and its configuration folder. The data folder is created for you by the service, in the next step:

Bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin prometheus
sudo install -d -o root -g root -m 0755 /etc/prometheus

Step 5 · Configure what Prometheus scrapes

The configuration lists what to scrape. Two targets: Prometheus itself, which publishes its own metrics, and Node Exporter:

Bash
sudo tee /etc/prometheus/prometheus.yml > /dev/null <<'YAML'
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ["127.0.0.1:9090"]

  - job_name: node
    static_configs:
      - targets: ["127.0.0.1:9100"]
YAML
promtool check config /etc/prometheus/prometheus.yml

promtool reads the file and prints SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax. Run it after every change, before you reload Prometheus. A job_name is a label that ends up on every metric from those targets, so jobs are how you tell servers and services apart in queries. To watch a second server later, add its address to targets ("10.0.0.5:9100").

Step 6 · Run Prometheus as a service

The unit is similar to the last one. StateDirectory=prometheus makes systemd create /var/lib/prometheus, owned by the service user, for the database. ExecReload lets you apply configuration changes without a restart, because Prometheus reloads its configuration when it receives the SIGHUP signal:

Bash
sudo tee /etc/systemd/system/prometheus.service > /dev/null <<'UNIT'
[Unit]
Description=Prometheus
After=network-online.target
Wants=network-online.target

[Service]
User=prometheus
Group=prometheus
StateDirectory=prometheus
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --storage.tsdb.retention.time=30d \
  --web.listen-address=127.0.0.1:9090
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
NoNewPrivileges=true
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target
UNIT

--storage.tsdb.retention.time=30d keeps 30 days of data; the default is 15 days. The data takes disk space in proportion to how many series you scrape and how long you keep them, so plan for it as in find what is using disk space. Start it:

Bash
sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
sudo systemctl status prometheus --no-pager

Check it worked

Prometheus has a health address that returns a short sentence once it is ready:

Bash
curl -s http://127.0.0.1:9090/-/ready

The output is Prometheus Server is Ready.. Wait about 30 seconds so that it has scraped twice, then ask which targets it can reach. up is a metric Prometheus creates for every target: 1 means the last scrape worked and 0 means it failed:

Bash
curl -s http://127.0.0.1:9090/api/v1/query --data-urlencode 'query=up' |
  jq -r '.data.result[] | "\(.metric.job) \(.value[1])"'

You get node 1 and prometheus 1, in either order.

The web interface is at http://127.0.0.1:9090 on the server. See the next steps for reaching it from your laptop.

Step 7 · Ask questions with PromQL

PromQL is the query language. A query that names a metric returns its newest value. rate() turns an ever-growing counter into a per-second speed. Try three queries against the HTTP API, which is what Grafana will use later. Each prints one number:

Bash
q() { curl -s http://127.0.0.1:9090/api/v1/query --data-urlencode "query=$1" | jq -r '.data.result[0].value[1]'; }
echo "Memory available (%):"
q '100 * node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes'
echo "CPU busy (%) over the last minute:"
q '100 * (1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1m])))'
echo "Uptime (days):"
q '(time() - node_boot_time_seconds) / 86400'
  • The first divides two gauges, values that go up and down, and multiplies by 100.
  • The second reads node_cpu_seconds_total, a counter of seconds each CPU core has spent in each mode. rate(...[1m]) is the average per second over the last minute, for each core. For the idle mode that is the fraction of time idle, so one minus the average across cores is the fraction busy. The [1m] window must hold at least two scrapes, so it needs more than 15 seconds of data.
  • The third subtracts the boot time from the current time.

The numbers you see are your server's own. The only wrong results are null or an empty line, which mean Prometheus has not collected enough data yet. Wait a minute and run the block again. You can try the same expressions in the browser at /graph, where the Graph tab plots them over time.

Step 8 · Reload the configuration after a change

When you add a target, edit the file, check it, then reload. Do not restart: a reload keeps the running server and its data:

Bash
promtool check config /etc/prometheus/prometheus.yml
sudo systemctl reload prometheus

If you made a mistake, promtool says so and the running server is untouched. To confirm that a reload worked, query prometheus_config_last_reload_successful: the value 1 means it did.

Step 9 · See it from your laptop, and scrape other servers

Both programs have no login. That is why they listen on 127.0.0.1. Prometheus's web interface can run queries and, with the right flags, change things, and Node Exporter reveals which software and kernel you run. Do not publish either on the internet.

To open the Prometheus page from your laptop without exposing it, tunnel through SSH. On your laptop:

Bash
ssh -L 9090:127.0.0.1:9090 you@your-server

Then open http://127.0.0.1:9090 on the laptop. Replace you@your-server with your login. The tunnel lasts as long as the SSH session.

To monitor other servers from this one, each of them runs Node Exporter, and you add them to targets. On each of those servers, change --web.listen-address to its private network address, for example 10.0.0.5:9100. Then open the port to the Prometheus server only, never to everyone. Replace the addresses with your own, and look at the UFW tutorial for the rest of the firewall:

Bash
sudo ufw allow from 10.0.0.2 to any port 9100 proto tcp

Step 10 · Keep it updated

Hand-installed programs get no updates from apt, so apt upgrade will never touch them. You are responsible for security fixes and new versions. Subscribe to the release feeds on the two GitHub release pages. To upgrade, set the new version, repeat the download and checksum steps, and replace the program while the service is stopped:

Bash
NODE_VERSION=<new version>
# download, check and unpack as in the install step, then:
sudo systemctl stop node_exporter
sudo install -o root -g root -m 0755 "node_exporter-${NODE_VERSION}.linux-${ARCH}/node_exporter" /usr/local/bin/node_exporter
sudo systemctl start node_exporter

Do the same for Prometheus with prometheus and promtool. Read the release notes before upgrading across a major version, because configuration flags and behaviour can change. If you would rather have apt manage updates and accept older versions, install prometheus and prometheus-node-exporter from the Ubuntu archive instead.

Step 11 · Remove everything

If this was a trial, remove what you created. This stops the services, deletes the units, programs, users and the collected data. The last line deletes the work folder:

Bash
sudo systemctl disable --now prometheus node_exporter
sudo rm /etc/systemd/system/prometheus.service /etc/systemd/system/node_exporter.service
sudo systemctl daemon-reload
sudo rm /usr/local/bin/prometheus /usr/local/bin/promtool /usr/local/bin/node_exporter
sudo rm -r /etc/prometheus /var/lib/prometheus
sudo userdel prometheus
sudo userdel node_exporter
rm -rf ~/monitoring-install

Skip this step if you are continuing to the next tutorials, which build on this setup. The Grafana and Alertmanager tutorials do not remove Prometheus or Node Exporter, so come back to this step after the last of them.

Next steps

Need this done for you?

Padding does this work for growing companies, from the first diagnosis to a system that keeps working.

Platform Engineering

Get new tutorials

New tutorials, about twice a month.