Skip to main content

Web servers and networking · Advanced · 14 min

How to Load Balance Traffic Across Servers with Nginx

By Padding · Updated 15 Nov 2026
Ubuntu 26.04.1 LTS · Nginx 1.30

You will spread requests across three backends with an Nginx upstream block, weight them, keep a spare in reserve, and watch Nginx route around a backend that stops answering.

On this page

One app server has a ceiling, and it is also a single point of failure. A load balancer sits in front of several copies of the app, spreads the requests and stops sending traffic to a copy that has failed. Nginx does this with an upstream block, with no extra software.

In this tutorial you will start three small backends on one machine, balance requests across them, change the balancing method, and stop a backend to see Nginx work around it. In production the backends are on different servers, so you write their private addresses where this tutorial writes 127.0.0.1.

The open-source Nginx used here checks health passively: it notices failed requests and stops using that backend for a while. Active health checks (health_check) are a feature of the commercial Nginx Plus, and are not covered.

Step 1 · Install Nginx and create three backends

Install Nginx from nginx.org first, as in /tutorials/install-nginx-on-ubuntu-26-04. Python is used for the backends because it needs nothing else; Ubuntu includes python3.

Write a backend that answers with the port it listens on, so you can see which one served a request:

Bash
mkdir -p ~/backends
cat > ~/backends/backend.py <<'PY'
import sys
from http.server import BaseHTTPRequestHandler, HTTPServer

port = int(sys.argv[1])

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        body = f"backend {port}\n".encode()
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass

HTTPServer(("127.0.0.1", port), Handler).serve_forever()
PY
for port in 9001 9002 9003; do
  nohup python3 ~/backends/backend.py "$port" > /dev/null 2>&1 &
done
sleep 1
for port in 9001 9002 9003; do curl -s "http://127.0.0.1:$port/"; done

Each backend prints its own name: backend 9001, backend 9002, backend 9003.

Step 2 · Define the upstream and proxy to it

An upstream block names a group of servers. A proxy_pass that points at the group name spreads requests over it. By default, Nginx uses round robin: each request goes to the next server in turn.

Keep the upstream in its own file, so later steps change one small file. The server block that uses it goes in another.

Bash
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
    zone app_backends 64k;
    server 127.0.0.1:9001;
    server 127.0.0.1:9002;
    server 127.0.0.1:9003;
}
CONF
sudo tee /etc/nginx/conf.d/lb.conf > /dev/null <<'CONF'
server {
    listen 80 default_server;
    server_name _;

    location / {
        proxy_pass http://app_backends;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
CONF
sudo rm -f /etc/nginx/conf.d/default.conf
sudo nginx -t

Apply it:

Bash
sudo systemctl reload nginx

This example assumes a machine whose only job is to balance. It deletes the stock default.conf and makes this block the default_server for port 80, so it answers every name the server receives. On a server that already hosts sites, do neither. Keep default.conf, remove default_server, and give the block a real server_name, such as app.example.com.

The upstream block lives at the same level as server blocks. The name app_backends is yours to choose.

The zone line puts the group's state in shared memory. Nginx runs several worker processes, and without a zone each worker keeps its own count, so the rotation is uneven and a server marked as failed is only avoided by the worker that saw the failure. Add a zone to every upstream you rely on.

Check it worked

Send six requests:

Bash
for i in 1 2 3 4 5 6; do curl -s http://127.0.0.1/; done | sort | uniq -c

Each backend answers twice: 2 backend 9001, 2 backend 9002 and 2 backend 9003.

Step 3 · Weight the servers and keep a spare

If one machine is twice as powerful as another, give it a bigger share with weight. A server marked backup gets traffic only when all the others are unavailable.

Change the upstream file:

Bash
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
    zone app_backends 64k;
    server 127.0.0.1:9001 weight=3;
    server 127.0.0.1:9002;
    server 127.0.0.1:9003 backup;
}
CONF
sudo nginx -t
Bash
sudo systemctl reload nginx

A reload applies the new settings while the old workers finish their requests, so no connection is dropped.

Check it worked

Bash
for i in $(seq 1 8); do curl -s http://127.0.0.1/; done | sort | uniq -c

Out of eight requests, backend 9001 answers six and backend 9002 answers two, which matches weights of 3 and 1. backend 9003 answers none, because it is the backup.

Step 4 · Choose a balancing method

Round robin is the default. Two other methods suit different workloads. Add the directive as the first line inside the upstream block.

  • least_conn; sends each request to the server with the fewest active connections. Use it when requests take very different amounts of time.
  • ip_hash; picks the server from the visitor's IP address, so the same visitor reaches the same server. Use it when the app keeps session data in memory. It is a stopgap: a shared session store works better and lets you use any method. ip_hash does not accept the backup parameter, so Nginx refuses a config that combines them.

Try ip_hash:

Bash
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
    zone app_backends 64k;
    ip_hash;
    server 127.0.0.1:9001;
    server 127.0.0.1:9002;
    server 127.0.0.1:9003;
}
CONF
sudo nginx -t
Bash
sudo systemctl reload nginx
Bash
for i in 1 2 3 4 5 6; do curl -s http://127.0.0.1/; done | sort | uniq -c

All six requests come from one address, so they reach the same backend, and one line appears with a count of 6. Now go back to the weighted group with a backup for the next step:

Bash
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
    zone app_backends 64k;
    server 127.0.0.1:9001 weight=3;
    server 127.0.0.1:9002;
    server 127.0.0.1:9003 backup;
}
CONF
sudo nginx -t
Bash
sudo systemctl reload nginx

Step 5 · Handle a backend that fails

Nginx marks a server unavailable after it fails max_fails times within fail_timeout. The defaults are 1 failure and 10 seconds. While a server is marked unavailable, Nginx sends no traffic to it. You can set both on each server line, for example server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;.

When a connection to a backend fails, Nginx retries the request on the next server in the group. The directive proxy_next_upstream sets which failures count. Its default covers connection errors and timeouts. Retrying a request that was sent to a backend that may have already acted on it, such as a POST, is not safe, and Nginx does not retry non-idempotent requests by default.

Stop the first backend and send more requests:

Bash
pkill -f 'backend.py 9001'
sleep 1
for i in $(seq 1 6); do curl -s -o /dev/null -w '%{http_code} ' http://127.0.0.1/; done; echo
for i in $(seq 1 4); do curl -s http://127.0.0.1/; done | sort | uniq -c

Every request still returns 200. Nginx noticed the failed connection to port 9001, tried the next server and then stopped using it. Port 9002 now serves everything, and the backup on 9003 stays idle until 9002 fails as well.

Check it worked

Look at the Nginx error log for the failed connection:

Bash
sudo grep -c 'Connection refused' /var/log/nginx/error.log

The count is at least 1. The log lines read connect() failed (111: Connection refused) while connecting to upstream. Watch this log in production: it is how you learn a backend has gone.

Troubleshooting

  • no live upstreams while connecting to upstream: every server in the group is marked unavailable, and no backup is left. Check your backends first, and the firewall between the balancer and them.
  • Requests do not rotate: the ip_hash method, a hash directive or one weight much larger than the rest is in effect. Re-read the upstream block.
  • Users get logged out at random: the app keeps session data in the memory of one backend. Move the session to a shared store such as Valkey or the database, or use ip_hash as a stopgap.
  • Backends see the load balancer's address, not the visitor's: read X-Real-IP or X-Forwarded-For, as the config above sets them.

Next steps

  • Return to the single-backend version in /tutorials/nginx-reverse-proxy-for-a-node-app.
  • Add HTTPS at the balancer, once, with /tutorials/free-https-with-lets-encrypt-and-nginx.
  • Add response headers at the balancer with /tutorials/add-security-headers-in-nginx.

Need this done for you?

Padding does this work for growing companies, from the first diagnosis to a system that keeps working.

Platform Engineering

Get new tutorials

New tutorials, about twice a month.