Web servers and networking · Advanced · 14 min
How to Load Balance Traffic Across Servers with Nginx
By Padding · Updated 15 Nov 2026
Ubuntu 26.04.1 LTS · Nginx 1.30
You will spread requests across three backends with an Nginx upstream block, weight them, keep a spare in reserve, and watch Nginx route around a backend that stops answering.
On this page
One app server has a ceiling, and it is also a single point of failure. A load balancer sits in front of several copies of the app, spreads the requests and stops sending traffic to a copy that has failed. Nginx does this with an upstream block, with no extra software.
In this tutorial you will start three small backends on one machine, balance requests across them, change the balancing method, and stop a backend to see Nginx work around it. In production the backends are on different servers, so you write their private addresses where this tutorial writes 127.0.0.1.
The open-source Nginx used here checks health passively: it notices failed requests and stops using that backend for a while. Active health checks (health_check) are a feature of the commercial Nginx Plus, and are not covered.
Step 1 · Install Nginx and create three backends
Install Nginx from nginx.org first, as in /tutorials/install-nginx-on-ubuntu-26-04. Python is used for the backends because it needs nothing else; Ubuntu includes python3.
Write a backend that answers with the port it listens on, so you can see which one served a request:
mkdir -p ~/backends
cat > ~/backends/backend.py <<'PY'
import sys
from http.server import BaseHTTPRequestHandler, HTTPServer
port = int(sys.argv[1])
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = f"backend {port}\n".encode()
self.send_response(200)
self.send_header("Content-Type", "text/plain")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
HTTPServer(("127.0.0.1", port), Handler).serve_forever()
PY
for port in 9001 9002 9003; do
nohup python3 ~/backends/backend.py "$port" > /dev/null 2>&1 &
done
sleep 1
for port in 9001 9002 9003; do curl -s "http://127.0.0.1:$port/"; doneEach backend prints its own name: backend 9001, backend 9002, backend 9003.
Step 2 · Define the upstream and proxy to it
An upstream block names a group of servers. A proxy_pass that points at the group name spreads requests over it. By default, Nginx uses round robin: each request goes to the next server in turn.
Keep the upstream in its own file, so later steps change one small file. The server block that uses it goes in another.
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
zone app_backends 64k;
server 127.0.0.1:9001;
server 127.0.0.1:9002;
server 127.0.0.1:9003;
}
CONF
sudo tee /etc/nginx/conf.d/lb.conf > /dev/null <<'CONF'
server {
listen 80 default_server;
server_name _;
location / {
proxy_pass http://app_backends;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
CONF
sudo rm -f /etc/nginx/conf.d/default.conf
sudo nginx -tApply it:
sudo systemctl reload nginxThis example assumes a machine whose only job is to balance. It deletes the stock default.conf and makes this block the default_server for port 80, so it answers every name the server receives. On a server that already hosts sites, do neither. Keep default.conf, remove default_server, and give the block a real server_name, such as app.example.com.
The upstream block lives at the same level as server blocks. The name app_backends is yours to choose.
The zone line puts the group's state in shared memory. Nginx runs several worker processes, and without a zone each worker keeps its own count, so the rotation is uneven and a server marked as failed is only avoided by the worker that saw the failure. Add a zone to every upstream you rely on.
Check it worked
Send six requests:
for i in 1 2 3 4 5 6; do curl -s http://127.0.0.1/; done | sort | uniq -cEach backend answers twice: 2 backend 9001, 2 backend 9002 and 2 backend 9003.
Step 3 · Weight the servers and keep a spare
If one machine is twice as powerful as another, give it a bigger share with weight. A server marked backup gets traffic only when all the others are unavailable.
Change the upstream file:
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
zone app_backends 64k;
server 127.0.0.1:9001 weight=3;
server 127.0.0.1:9002;
server 127.0.0.1:9003 backup;
}
CONF
sudo nginx -tsudo systemctl reload nginxA reload applies the new settings while the old workers finish their requests, so no connection is dropped.
Check it worked
for i in $(seq 1 8); do curl -s http://127.0.0.1/; done | sort | uniq -cOut of eight requests, backend 9001 answers six and backend 9002 answers two, which matches weights of 3 and 1. backend 9003 answers none, because it is the backup.
Step 4 · Choose a balancing method
Round robin is the default. Two other methods suit different workloads. Add the directive as the first line inside the upstream block.
least_conn;sends each request to the server with the fewest active connections. Use it when requests take very different amounts of time.ip_hash;picks the server from the visitor's IP address, so the same visitor reaches the same server. Use it when the app keeps session data in memory. It is a stopgap: a shared session store works better and lets you use any method.ip_hashdoes not accept thebackupparameter, so Nginx refuses a config that combines them.
Try ip_hash:
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
zone app_backends 64k;
ip_hash;
server 127.0.0.1:9001;
server 127.0.0.1:9002;
server 127.0.0.1:9003;
}
CONF
sudo nginx -tsudo systemctl reload nginxfor i in 1 2 3 4 5 6; do curl -s http://127.0.0.1/; done | sort | uniq -cAll six requests come from one address, so they reach the same backend, and one line appears with a count of 6. Now go back to the weighted group with a backup for the next step:
sudo tee /etc/nginx/conf.d/upstream.conf > /dev/null <<'CONF'
upstream app_backends {
zone app_backends 64k;
server 127.0.0.1:9001 weight=3;
server 127.0.0.1:9002;
server 127.0.0.1:9003 backup;
}
CONF
sudo nginx -tsudo systemctl reload nginxStep 5 · Handle a backend that fails
Nginx marks a server unavailable after it fails max_fails times within fail_timeout. The defaults are 1 failure and 10 seconds. While a server is marked unavailable, Nginx sends no traffic to it. You can set both on each server line, for example server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;.
When a connection to a backend fails, Nginx retries the request on the next server in the group. The directive proxy_next_upstream sets which failures count. Its default covers connection errors and timeouts. Retrying a request that was sent to a backend that may have already acted on it, such as a POST, is not safe, and Nginx does not retry non-idempotent requests by default.
Stop the first backend and send more requests:
pkill -f 'backend.py 9001'
sleep 1
for i in $(seq 1 6); do curl -s -o /dev/null -w '%{http_code} ' http://127.0.0.1/; done; echo
for i in $(seq 1 4); do curl -s http://127.0.0.1/; done | sort | uniq -cEvery request still returns 200. Nginx noticed the failed connection to port 9001, tried the next server and then stopped using it. Port 9002 now serves everything, and the backup on 9003 stays idle until 9002 fails as well.
Check it worked
Look at the Nginx error log for the failed connection:
sudo grep -c 'Connection refused' /var/log/nginx/error.logThe count is at least 1. The log lines read connect() failed (111: Connection refused) while connecting to upstream. Watch this log in production: it is how you learn a backend has gone.
Troubleshooting
no live upstreams while connecting to upstream: every server in the group is marked unavailable, and no backup is left. Check your backends first, and the firewall between the balancer and them.- Requests do not rotate: the
ip_hashmethod, ahashdirective or one weight much larger than the rest is in effect. Re-read theupstreamblock. - Users get logged out at random: the app keeps session data in the memory of one backend. Move the session to a shared store such as Valkey or the database, or use
ip_hashas a stopgap. - Backends see the load balancer's address, not the visitor's: read
X-Real-IPorX-Forwarded-For, as the config above sets them.
Next steps
- Return to the single-backend version in
/tutorials/nginx-reverse-proxy-for-a-node-app. - Add HTTPS at the balancer, once, with
/tutorials/free-https-with-lets-encrypt-and-nginx. - Add response headers at the balancer with
/tutorials/add-security-headers-in-nginx.
Need this done for you?
Padding does this work for growing companies, from the first diagnosis to a system that keeps working.
Get new tutorials