What is ECMP?
ECMP (Equal-Cost Multi-Path) is a routing mechanism that spreads traffic across several equal-cost paths. We explain flow hashing and per-flow packet ordering.
Last updated:
ECMP (Equal-Cost Multi-Path) is a routing mechanism that lets a router install several paths of identical cost to the same destination in its forwarding table and spread traffic across them simultaneously. Instead of picking one best route and discarding the rest, the router uses all equal-cost links at once. This raises throughput and resilience. ECMP works with link-state protocols (OSPF, IS-IS) and with BGP, where it must be enabled explicitly.
How ECMP works
When the routing process finds two or more routes to the same prefix with the same metric, it installs all of them as parallel next-hop entries in the forwarding table (FIB). The number of simultaneous paths is limited by the platform. It is typically 8 to 64, and more on modern silicon. The decision about which link a given packet leaves on is made in hardware at forwarding time, so ECMP adds no extra load to the control plane and scales to line rate.
Flow hashing and packet ordering
To avoid reordering packets within a single connection, ECMP load-balances per flow rather than per packet. The router computes a hash over selected header fields and uses it to pin a flow to a specific link. All packets of the same flow take the same path, so the ordering that TCP expects is preserved.
- The typical 5-tuple: source IP, destination IP, protocol, source port and destination port.
- A 3-tuple variant (source, destination, protocol) is sometimes used for non-TCP/UDP traffic.
- Per-flow hashing preserves ordering but can distribute unevenly with a few large flows (the elephant-flow effect).
- Many platforms add configurable fields, such as MPLS labels or the IPv6 flow label, to improve balance.
- Per-packet load balancing distributes perfectly evenly but breaks ordering and is rarely used for TCP.
ECMP in BGP and where it is used
In BGP only a single best route is installed by default, so multipath has to be enabled explicitly (maximum-paths). For routes to count as equal, they must share the same AS_PATH length, the same MED, local preference and origin. A multipath-relax option on many platforms loosens this by accepting equal-length AS_PATHs learned from different neighbouring autonomous systems. ECMP is widely used for layer-3 link aggregation, in leaf-spine data-center fabrics, and to spread traffic across multiple upstream sessions in parallel.
ECMP and AS202520 SkyPass
ECMP is a practical way to add throughput and reliability without buying a single, costlier link. On the AS202520 SkyPass network, with PoPs in Warsaw and Wrocław and peering at THINX, TPIX, WRIX and 1-IX, ECMP lets traffic be balanced evenly across multiple parallel BGP sessions within our IP transit and peering connectivity. A multihomed customer gains greater aggregate capacity and a smooth shift to the remaining paths if one link fails.
Frequently asked questions
How many ECMP paths does a router support?
It depends on the platform. The typical range is 8 to 64 parallel paths per prefix, though newer silicon can handle more. The limit is set with the maximum-paths parameter.
Does ECMP reorder packets?
Not when it uses per-flow load balancing. All packets of one flow (the same 5-tuple) take the same path, so ordering is preserved. Reordering can only occur with the rarely used per-packet mode.
Why is traffic across ECMP links not perfectly even?
Because balancing is per-flow, based on a hash function. With a few large flows (elephant flows) the hash may pin them to the same link, which causes imbalance despite multiple available paths.
Does ECMP in BGP need to be enabled?
Yes. By default BGP installs only a single best route. Multipath is enabled explicitly (e.g. maximum-paths), and the routes must be equal: same AS_PATH length, MED, local preference and origin, among others.