A Wire-Compatible TCP Implementation for Low-Latency Applications

Abstract:

Despite alternative transport protocols more suitable to latency-sensitive applications, TCP remains the de facto standard for Internet traffic, including many low-latency, interactive applications. As such, applications often tweak the TCP protocol to better suit their needs but run the risk that any change to TCP’s wire format can cause reachability issues or complete failure. This work presents a modified TCP implementation to reduce end-to-end latency without modifying TCP’s wire format.

Bufferbloat Resilient TCP (BBR) is a sender-side modification that detects and mitigates instances of so-called “bufferbloat” in a TCP connection. Large network buffers coupled with TCP’s additive increase often result in standing queues and drastically higher round-trip times. BBR throttles the TCP sender by identifying bufferbloat through the correlation between sending rate and delay. A live production deployment shows BBR improves average end-to-end latency, bandwidth efficiency and retransmissions by orders of magnitude for a video serving platform.

Random and congestive packet loss increase the latency of subsequent packets, not just the lost packets, due to TCP’s strict in-order delivery model. Unordered TCP (uTCP) is a receiver-side change that exposes to applications out-of-order data segments that the OS normally delays in socket buffers. This small change enables applications to build generic unordered datagram transports and improves end-to-end latency for interactive applications. Evaluations show that uTCP can approximate the latency performance of UDP for various application workloads.

Ph.D. Thesis: PDF