Azure Load Balancer: the layer-4 traffic cop for your VMs

You have three identical VMs and one stream of incoming connections. Something has to stand at the front, wave each new connection toward a healthy machine, and quietly stop sending work to any machine that has fallen over. That something is Azure Load Balancer — fast, blunt, and working at a lower layer than you might expect.

Azure Load Balancer is a service that, in Microsoft's plain words, "distributes incoming network traffic across backend virtual machines (VMs) or virtual machine scale sets." It is the single point of contact for clients: traffic hits the load balancer's frontend, and the service hands each flow off to an instance in the backend pool according to your rules. Simple job, done at enormous scale — it can handle millions of flows.

Layer 4 is the whole personality

The single most important fact about Azure Load Balancer is where it operates: layer 4 of the OSI model — the transport layer, the world of TCP and UDP. It does not read HTTP. It does not know or care whether a request is for /login or /images, and it cannot make decisions based on a URL path, a cookie, or a host header. It sees connections — source, destination, ports — and it distributes them. That is a feature, not a limitation: staying at layer 4 is exactly what lets it be a pass-through load balancer with ultralow latency and huge throughput for any TCP or UDP application, not just web traffic.

It does not read your requests. It reads your connections. That is why it is fast, and why it is the wrong tool when the routing decision lives inside the HTTP.

Public or internal: two kinds of front door

A load balancer's frontend comes in two flavors, and picking the right one is most of the design:

• Public load balancer — has a public IP and load-balances traffic from the internet to your VMs. It also does double duty for outbound: it can translate your VMs' private IPs to a public IP for connections they originate. This is the front door for an internet-facing tier.
• Internal (private) load balancer — lives entirely inside a virtual network with a private IP, spreading traffic that is already inside your network. Use it in front of an internal tier — a database or app layer that a web tier talks to, or a service reached from on-premises over a hybrid link — where you never want a public IP at all.

Health probes: the part that earns its keep

Distributing traffic evenly is easy. Distributing it only to instances that are actually working is the valuable part. Azure Load Balancer uses health probes to continuously check each backend instance, and it sends new flows only to the ones reporting healthy. When a VM stops answering its probe — it crashed, it is mid-reboot, the app hung — the load balancer stops routing new connections to it without you lifting a finger, and resumes once it recovers. That quiet, automatic exclusion of the sick instance is what turns a pool of VMs into a service that stays up while individual machines come and go.

Where Application Gateway takes over

The question that separates people who get this from people who guess: "Load Balancer or Application Gateway?" The answer is the layer. If your routing decision depends on the connection — spread TCP or UDP across a pool, forward a port, balance a non-HTTP protocol — Load Balancer (layer 4) is right, and it is faster and cheaper. If your routing decision depends on the HTTP itself — send /api to one pool and /images to another, terminate TLS, apply a web application firewall, do cookie-based session affinity — you need Application Gateway, the layer-7 sibling. Many real architectures use both: Application Gateway out front for smart HTTP routing, Load Balancer behind it for raw transport-layer spread. They are teammates, not rivals.

The takeaway

Azure Load Balancer is a layer-4 service that takes a stream of TCP or UDP connections and spreads them across a backend pool of VMs, using health probes to skip the instances that are down. Choose a public frontend for internet traffic and an internal one for traffic that stays inside your network. Reach for Application Gateway instead the moment the routing decision needs to read the HTTP. "Standard Load Balancer for fast transport-layer spread across a zone-redundant pool, Application Gateway when I need path-based routing or a WAF" is the answer of someone who chose by the layer, not by the logo.