Congestion control, and why low latency is a separate problem

There is no field anywhere in an IP packet that says how fast the sender is allowed to go. No router hands out an allocation. No exchange takes place at the start of a connection in which the network states the capacity available and the sender agrees to stay within it. A sender that wants to know its share of a path has exactly one way to find out, which is to send and see what happens.

That leaves the sender with two pieces of evidence, and only two. Acknowledgements either come back or they do not, and the ones that come back take a certain amount of time. From those two observations every congestion control algorithm ever deployed has to reconstruct a picture of a path it cannot see. That reconstruction is what congestion control is: the rules a sender follows to decide how fast it may send, based on signals it has to interpret rather than facts it has been given.

Both signals are indirect, and both can mislead. A gap in the acknowledgements might mean a router queue filled up and had to discard something, which is genuine congestion. It might equally mean a bit error on a radio link corrupted a frame, which has nothing to do with capacity at all. Round trips getting longer might mean a queue is building ahead of the bottleneck, or it might mean the path changed and the traffic is now taking a longer route. The sender cannot tell these cases apart from the evidence available to it, which is why every algorithm in this module is, at bottom, a different bet about which explanation is more likely.

The previous module, on TCP reliability and flow control, introduced the two windows that bound a TCP sender and kept them carefully separate. The receiver's advertised window says how much the receiver can absorb. The congestion windowis the sender's own private estimate of how much the network in between will tolerate, and the sender is limited by whichever of the two is smaller. Everything in this module is about the second one: nobody tells the sender what it should be, so the sender has to keep guessing and revising.

The reason this matters is that the bottleneck is shared. Picture a household line carrying a video call, a photo backup and a large operating system update at the same time. The link has one capacity, and three senders in three different data centres are each independently deciding how hard to push into it. None of them can see the other two. None of them has been told what fraction is theirs. If each simply pushed as fast as its own hardware allowed, the queue at the narrow point would stay permanently full, every packet would be delayed behind every other packet, and a rising share of what did get through would be retransmissions of things that had already been sent. The link would be fully occupied and delivering very little useful work. That failure mode has a name, congestion collapse, and avoiding it is the reason restraint is built into the transport rather than left to applications.

This is the framing that most operational confusion comes from missing. Congestion control is not error handling. It is not a recovery routine that runs when something has gone wrong. It is a continuous, distributed agreement between senders who will never communicate with each other, mediated entirely by what the network does to their packets. A sender backing off is that agreement being honoured, and a transfer that never sees a congestion signal is a transfer that never found out where the limit was.

Having established that a sender only ever infers capacity, the practical question becomes which inference it makes and how it turns that into a number. Almost every machine you will ever measure answers that question the same way, so the loss-based answer is the one to understand first.

CUBIC is specified in RFC 9438, published in August 2023, and it is the default congestion control algorithm in both Linux and Windows. That single sentence carries more weight than it looks. A default is what runs unless somebody deliberately changes it, and changing it is a decision that leaves a trace in configuration rather than something that happens by itself. So when you open a throughput graph without knowing anything else about the two endpoints, CUBIC is the reasonable first assumption about what drew the shape you are looking at.

Its behaviour is easiest to hold as a story about memory. CUBIC remembers the window size it was using when congestion was last detected. It grows quickly back towards that remembered size, then flattens out and lingers around it, spending a while confirming that the path really can still take that much. Only after that does it start pushing above the remembered size, cautiously at first and then faster, probing for capacity that might have been freed up since. When a packet is lost it cuts the window, records the new high-water mark, and begins the cycle again.

The shape of that growth is where the name comes from and where the design intent lives. RFC 9438 replaces the older linear increase with a cubic function precisely because linear growth scales badly on fast links with long round trips. On a path with a great deal of capacity and a long delay, an algorithm that adds a fixed small amount per round trip can spend an implausibly long time climbing back to a window it was already using a moment ago, and the link sits underused throughout. The cubic curve recovers quickly when it is far below the remembered window and slows right down as it approaches it, which is the behaviour you would design by hand if you wanted fast recovery without immediately overshooting.

If you have ever wondered why throughput graphs of bulk transfers so often look like a row of teeth, this is the answer. The climb, the plateau, the probe above and the sudden collapse are the visible trace of one algorithm repeatedly asking the same question and getting the same kind of answer.

Two consequences follow from CUBIC using loss as its signal, and both of them set up the rest of the module. The first is that CUBIC can only learn from damage. A queue has to actually overflow before the sender receives any information at all, which means that on a device with a large buffer the sender stays ignorant while the queue fills and the delay through it grows. By the time the loss signal arrives, the latency problem has already been happening for some time. The second is that CUBIC treats every loss as congestion. On a wireless or long-distance link where packets are sometimes lost to interference or bit errors, the algorithm reads a corrupted frame as a full queue and cuts its rate for a reason that does not exist.

Both of those weaknesses come from the same root, which is that CUBIC only has one piece of evidence and treats it as unambiguous. The obvious alternative is to gather more evidence and reason about it, and that is exactly what the second algorithm on the figure above does.

BBR takes a different position on what the sender is trying to do. Rather than pushing until something breaks, it continuously estimates two properties of the path: how much bandwidth the bottleneck can actually deliver, and the shortest round-trip time the path exhibits when nothing is queued. Those two numbers are a model of the path, and the sender paces its sending against that model instead of reacting to individual losses.

Pacing is the operative word. A loss-based sender fills its window and lets the packets leave as fast as the interface will emit them, which puts a burst into the bottleneck queue whether or not the queue needed one. A model-based sender spaces its packets out to match the rate it believes the bottleneck can absorb. If the model is right, the bottleneck stays busy and the queue in front of it stays close to empty, which is the combination that CUBIC structurally cannot produce.

The model has to be maintained, so BBR periodically disturbs it on purpose. It lifts its rate briefly above the estimate to find out whether more bandwidth has become available, then returns. Separately it reduces its rate for a moment to let any queue it has created drain, so it can re-measure the true minimum round trip rather than a round trip inflated by its own traffic. Those two deliberate excursions are visible in the right-hand lane of the figure above: the pulse over the estimated bandwidth line, and the dip below it.

Read the two lanes of that figure side by side and the difference is not about which one is faster. It is about what each one accepts as evidence. The left lane reacts to something the network did to its packets. The right lane reacts to its own measurements of the network. That is why the same link can produce a sawtooth from one sender and a near-flat line from another, and why comparing throughput between two hosts without knowing what each of them is running is not a comparison of anything in particular.

Now the point the figure's status band exists to make, because it is the fact most often stated wrongly. CUBIC is a published standard: RFC 9438 is an RFC and can be cited as one. BBR is not. Version 3, the version described in current work, is specified in draft-ietf-ccwg-bbr, which is an IETF Internet-Draft. A draft is a working document with no formal standing. It can be revised, superseded or abandoned, and it becomes a standard only if and when it is published as an RFC. Writing "RFC BBRv3" in a design document, a runbook or a supplier requirement is not a small slip of notation. It claims a stability the document does not have, and it commits an organisation to something whose specification may not say the same thing next year.

This is not an argument against using BBR. It is a reasonable engineering choice in many situations, and on paths where loss is a poor proxy for congestion it is the better one. It is an argument for describing it accurately, because the difference between a standard and a draft is exactly the difference between a behaviour you can hold a supplier to and a behaviour you are choosing to depend on at your own risk.

The IETF has been explicit about how much scrutiny this area needs. RFC 9743, published in March 2025, sets out what anyone proposing a new congestion control algorithm is expected to evaluate before it goes anywhere near the public internet, so that a new algorithm works efficiently without harming everything else sharing the same links. The existence of that document tells you something on its own. Standards bodies do not write guidance on how to specify new members of a category unless new members keep arriving. Congestion control is not a settled part of the stack that was finished decades ago; it is an active area with a real risk of a badly behaved newcomer taking more than its share.

CUBIC and BBR are both the sender's side of the bargain, and both are constrained by the same thing: they can only work with the signals the network chooses to give them. The network's side of that bargain is what a queue does when it starts to fill, and that is where the largest latency problem on the consumer internet lives.

Memory became cheap, so equipment vendors gave their devices generous buffers. The reasoning was sound in isolation. Dropping a packet felt like a failure, holding it felt like a service, and a bigger buffer meant fewer drops on any measurement you were likely to take. What that reasoning missed is that a packet held is a packet delayed, and that a queue which never overflows is a queue that never sends the one signal a loss-based sender is waiting for.

Bufferbloat is the result. A buffer deep enough to absorb a sustained overload does not prevent congestion, it converts congestion into delay. Throughput measurements come back looking healthy because the link genuinely is carrying its full rate. Utilisation graphs look normal. What has actually happened is that a standing queue has formed in front of the bottleneck, and every packet, including the small interactive ones, now has to wait behind everything already sitting in it. The video call stutters, the remote session feels detached from the keyboard, and the page takes an age to start rendering, all while the numbers on the dashboard say the connection is fine.

The mechanism also explains why the intuitive remedy fails, which is worth being precise about because it is a decision people spend real money on.

The route out of that trap starts with a change to the signal itself. Explicit Congestion Notification uses a pair of bits in the IP header to let a router mark a packet rather than discard it. The mark says that the queue is building. The receiver echoes it back to the sender, the sender slows down, and the congestion has been communicated without anything being thrown away or retransmitted. That separation of the congestion signal from packet loss is what makes a low latency design possible in the first place, because it means the sender can be told about a queue while the queue is still short.

L4S is the architecture built on that idea, and its name is a specification of intent rather than a brand: low latency, low loss and scalable throughput, all three at once. It was published in January 2023 as three RFCs with three distinct jobs. RFC 9330 sets out the architecture and what the parts are for. RFC 9331 defines the ECN protocol that L4S traffic uses, which marks far more readily than classic ECN did. RFC 9332 defines dual queue coupled active queue management, the mechanism in the network that keeps L4S traffic in its own shallow queue alongside everything else without either kind of traffic starving the other.

Put those three together and the sequence at a congested bottleneck changes completely. Traffic that has opted into L4S goes into the shallow queue. As soon as that queue begins to build, packets in it are marked. The sender sees the mark and trims its rate within a single round trip, before the queue has grown long enough for anyone to feel it. Nothing was dropped, so nothing has to be retransmitted, and the queue never became the delay. Traffic that has not opted in continues to use the classic queue and behaves exactly as it did before, which is what makes incremental deployment possible at all.

Compare the two panels of that figure with the earlier discussion of CUBIC in mind. In the before panel, the sender is doing what a loss-based algorithm must do: pushing until something drops. The buffer, doing what it was designed to do, keeps accepting. The delay arrives first and the loss signal arrives last, which is precisely the wrong order for anyone trying to hold an interactive session open. In the after panel the order is reversed. The signal arrives while the queue is still short, so the sender acts before the delay is ever created.

The deployment picture follows the architecture. L4S is not a switch on a single box, because it needs the sender to use a compatible congestion control, the receiver to echo the marks and the bottleneck to run the dual queue. Where it has reached ordinary consumers is through the low latency modes of DOCSIS, the cable broadband standard, which is why the interoperability testing between Comcast, Apple, Google and NVIDIA described at the start of this module is the substantive news rather than the publication of the RFCs. Three of those four parties do not sell the queue and the fourth does not sell the endpoints. Nobody can deliver this alone.

That also sets a realistic expectation for your own estate. If your users are on connections whose bottleneck equipment does not implement the dual queue, L4S is not available to you no matter what your servers support, and the answer for now is the older one: keep the buffers at your own bottlenecks short and let active queue management hold them there. What has changed is that queueing delay is now treated as a first-class problem with a standardised answer, rather than as an unfortunate property of busy links.

A sawtooth, a standing queue and a draft specification are now three separate things. All three reach you the same way, as an ordinary line on a monitoring dashboard or an ordinary sentence in a design document, and all three are routinely read as something they are not. What follows turns them into the questions worth asking when one of them lands on your desk.

Start with the claim that does the most damage, because it is stated confidently in support tickets every day.

That gives you the first diagnostic move, which is to separate a damaged path from a busy one. Damage does not care how much traffic you are offering: the loss stays there on an idle link, it tends to be accompanied by errors counted on an interface, and it is often symmetrical in a way congestion is not. Congestion appears when the link fills, scales with how hard the link is being pushed, and vanishes when the offered load drops. Two runs of the same test at two different times of day distinguish them faster than any amount of reading of a single measurement.

The second move is the one this module exists for. When latency is high and throughput is stable at the full expected rate, you are looking at a queueing problem and not a capacity problem. Those two facts together are the signature: the link is delivering everything it should, and the delay is what is left over. A capacity shortage looks different, because throughput itself falls short of what the link is meant to deliver. Getting this the right way round decides whether the next action is a purchase order or a change to queue management, and only one of those two will work.

The third move is to establish what is actually running before interpreting a graph. A sawtooth throughput trace on a bulk transfer is CUBIC doing its job, not an intermittent fault, and it does not need investigating. A flat trace from a model-based sender on the same path is not evidence of a better network. If two services on the same link produce completely different shapes, the difference may lie entirely in their congestion control, and no amount of examining the network in between will explain it.

The fourth move is about evidence and language, and it is where the standards status from section 14.3 becomes practical rather than pedantic. If a supplier proposal, an internal design or a change request depends on a specific congestion control behaviour, write down which document defines it. CUBIC gives you RFC 9438 and a stable reference. BBR gives you draft-ietf-ccwg-bbr, a document that may change. That is a legitimate position to take, but it should be a decision on the record rather than an assumption buried in a sentence that called a draft an RFC.

None of this asks you to run packet captures or read algorithm source code. It asks you to hold three distinctions steady: loss that behaves like congestion against loss that does not, delay caused by a queue against delay caused by distance or capacity, and a published standard against a working document. Every one of those is a question you can ask from a monitoring dashboard and a couple of well-timed tests.