Low Latency Trading Systems: The Complete Guide to Infrastructure, Inference and Execution Speed

From the trigger that starts the clock to the order that stops it: how a modern low-latency trading system is built, where AI inference sits inside it, and why the industry keeps arriving at the same conclusion — that being fast is no longer enough on its own.

Introduction

In electronic markets, everyone sees the same information at the same time. Being first to react is what matters, because an opportunity disappears the moment an order is placed — that order changes the very conditions that created the opportunity in the first place.^[1] This guide is for proprietary trading firms, market makers, and the quant and trading-technology teams who build and own the infrastructure behind that reaction. It sets out what low-latency trading actually involves, how a modern low-latency trading system is built end to end, where AI inference sits inside it, and how CPU, GPU and FPGA compare as places to run that inference. It closes with practical steps for reducing latency and an evaluation checklist for technical buyers.

What is low-latency trading?

High-frequency trading involves placing high volumes of orders at lightning speed to take advantage of extremely rapid market movements, and in this world, speed is the biggest advantage.^[1] Underneath that speed sits a basic trading loop, run millions of times a day: watch the market for new information such as a price change, decide what that information means and which trade to make, and send the trade to the exchange before competitors do.^[1] For some trades, even nanoseconds count. The fastest trading systems in production today operate in the realm of sub-nanosecond timing, where measurement noise itself becomes part of the engineering problem.^[1]

Low-latency trading is that loop, engineered to run as fast as a given strategy demands: a trading activity, not a single piece of technology — the technology exists to serve it.

How latency in trading is measured

The industry’s own name for the part of the loop that matters most is the hot path: a sequence of events in a trading application starting with an external trigger — a market event — and ending when the responding order leaves the system. The time spent on that sequence is called trigger-to-order latency, or hot path latency.^[1]

One way to picture it: a market event arrives and a clock starts; inside the network interface card, the system decodes the message, decides what to do, runs a risk check, and serialises the outbound order; the order leaves and the clock stops.^[1] Because everyone on the market sees the same event at the same moment, the system that completes that loop first is the one that captures the opportunity.

That’s also why the measurement discipline behind a benchmark result matters as much as the headline number. STAC-ML, the benchmark used to compare inference deployments for exactly this kind of workload, reports latency as an audited 99th-percentile figure against a named hardware configuration, not a single average number chosen for the marketing.^[3]

Anatomy of a low-latency trading system

A trading application can be broken down into three layers — signals, strategy and execution — with a risk envelope wrapped around the whole thing.^[1] The sections below walk through that structure, plus the physical infrastructure underneath it.

Market data ingest

Exchanges send market data to every participant simultaneously over UDP multicast, so no one gets a head start on the raw feed itself; participants send their own orders back over TCP.^[1] Decoding that data is the first step inside the hot path — the “decode” stage of the trigger-to-order sequence.^[1]

Signal generation

Once market data has arrived, it feeds the signals layer: services that collect prices and order-book information and run the calculations — pricing algorithms and machine-learning pipelines among them — that get made available to the strategy layer deciding how to trade.^[1] Increasingly, that signal-generation step is itself an AI model, and it tends to take one of two forms: a slower, richer model that digests data on a slower cadence and fires a fast trigger the moment it recognises a scenario worth acting on, or a faster, distilled model running close to the decision point that watches signals in real time and acts on them directly.^[1]

Latency-sensitive hot paths

Not every trigger in a trading system carries the same urgency, and separating them out matters. Four hot paths recur across most trading strategies:

• Quote/requote: continually update the prices shown to the market.
• Aggress/take: the trigger-to-order path — detect an opportunity and execute the trade.
• Retreat/reprice: after completing an order, reprice everything connected to it.
• Cancel/pull: cancel a resting order before it can be traded against.^[1]

The retreat path shows why each one matters on its own terms. The moment a trade fills, that trade changes the price of the next quote on offer, and the whole surface connected to it — in the case of an S&P option, potentially thousands of related options — has to be repriced and republished before a faster competitor can trade against the now-stale price. Ten years ago that retreat process took seconds; through optimisation at every level of the stack, it now takes nanoseconds.^[1]

Order execution and colocation

Execution is deliberately narrow: once a decision is made, the execution step is only permitted to send the trade, with no additional logic running there — a hard separation of concerns.^[1] Because everyone sees the same market event at the same moment, physical distance to the exchange’s matching engine is itself a latency cost. That’s why serious market makers run servers directly inside colocation facilities, as close to the exchange as physically possible — running dozens of such co-locations across the venues they trade on.^[1]

Measuring the pipeline

At the fastest end of the spectrum, measurement is hard in its own right: once a trading system’s response time reaches sub-nanosecond territory, the noise in the measurement can be as large as the thing being measured.^[1] Monitoring has to be sensitive enough to catch changes far smaller than that — sensitive enough, in one real production incident, to detect a change in latency caused by nothing more than a cable that was a little longer than it needed to be. At these margins, some changes look statistically indistinguishable from noise until several days of monitoring confirm whether they’re real.^[1]

Where AI inference fits

For a long time, being faster than the competition was the whole game: lower latency than everyone else meant catching arbitrage opportunities before anyone else could. That’s no longer true. Once most serious participants have access to low-latency infrastructure, latency becomes the floor everyone stands on rather than what separates the winners — and the differentiator becomes the model behind the trade.^[1] Today, there is easily twice as much engineering effort going into models and data as into further shaving latency.^[1]

That shift shows up as the same two patterns described above: a slow model with a fast trigger, and a fast model running near the edge of the network.^[1] Either pattern only works if the inference step itself is fast and predictable enough to sit inside the hot path without becoming its slowest link — which is precisely the workload STAC-ML’s Tacana suite is built to measure: LSTM inference running against live market data, evaluated the way a sliding window works, re-evaluating a fixed history of ticks every time a new one arrives, and reusing computation rather than recomputing it from scratch.^[3]

Why GPU inference can become a bottleneck

NVIDIA has made the case for GPU inference in exactly this setting. In a technical post announcing single-digit-microsecond 99th-percentile latency for LSTM inference on its GH200 Grace Hopper Superchip, on the STAC-ML Markets (Inference) Tacana benchmark, NVIDIA described the result as matching or exceeding the FPGA-based results on the board at the time.^[3] Audited under SUT ID SMC250910, that GH200 result came in at 4.70 microseconds p99 for the smallest model tested (LSTM_A), 7.10 microseconds for LSTM_B, and 15.80 microseconds for LSTM_C.^[3]

Those numbers hold up on their own terms. What they no longer describe, once a newer audited FPGA result exists on the same benchmark, is the fastest board on the table — and that gap is exactly the kind a hot path built for nanosecond-scale trigger-to-order timing can’t afford to carry.

CPU vs GPU vs FPGA: what matters in practice

A trading application typically starts life as an ordinary, well-optimised C++ program running on general-purpose CPU. It stays there until CPU context switching and the kernel’s own network stack become the slowest, highest-variance parts of the pipeline — at which point pinning cores and bypassing the kernel to talk to the network card directly buys back a further layer of speed, before hardware becomes the next lever.^[1]

How to reduce latency and increase throughput

Reducing latency in a trading system tends to follow the same progression, one bottleneck at a time.

1. Measure the hot path first. Expect the measurement itself to be hard at the sub-nanosecond end — build in enough monitoring to tell a real regression apart from noise before changing anything.^[1]
2. Pin CPU cores to the process handling the hot path, so it can’t be pre-empted by anything else running on the same core.^[1]
3. Bypass the kernel and work with the network card directly once the kernel’s own network stack becomes the bottleneck.^[1]
4. Move the decision logic into an FPGA once software and kernel-level optimisation are exhausted: enumerate the market patterns the model needs to respond to, precompute a response for each one, hold those responses behind fast gate logic, and release the matching response the instant a live pattern arrives.^[1] Myrtle’s own audited VOLLO submission shows what that looks like end to end — 1.89 microseconds p99 for LSTM_A, audited under SUT ID MRTL260323.^[3]
5. Consider custom silicon once FPGA clock speed becomes the constraint: an ASIC trades away the FPGA’s reconfigurability for gigahertz-range clock speeds, at the cost of a fabrication cycle measured in months rather than minutes.^[1]
6. Own the physical path to the exchange, not just the logic deciding what to send down it: dedicated or leased fibre, leased wavelength capacity, and — because radio through air travels 30–40% faster than light through an optical cable — microwave and shortwave radio links for the legs where a few milliseconds are worth the reliability trade-off.^[1]
7. Test hardware logic exactly like software. FPGA logic can run through the same CI pipeline and automated tests as everything else, and be reviewed the same way source code is reviewed, so a hardware change doesn’t get to skip the scrutiny a software one would face.^[1]
8. Wrap the whole system in a risk envelope. Every strategy should sit inside a risk-management system that can block trades and stop individual strategies, watching for outputs that fall outside expected parameters regardless of what the underlying model wants — with automated, post-trade monitoring anywhere the hot path itself can’t afford the latency of a human check.^[1]

 

Evaluation checklist for technical buyers

When evaluating a low-latency trading system, or an inference platform that will sit inside it, the following questions help surface tail behaviour, integration friction and operational realism.

Conclusion

Latency is the floor now, not the moat: once every serious participant has access to low-latency infrastructure, the differentiator becomes the model behind the trade.^[1] But a model’s edge is not like a piece of software. It exists because a price was wrong, it decays every time it’s used, and betting more against it than the edge supports doesn’t scale the return — it wipes out the position.^[2] Between those two facts sits the part of the system this guide has focused on: the hot path that has to turn a signal into an order fast enough, and predictably enough, that the edge is still there when the order arrives. Getting that step right — measured, audited, and tested with the same rigor as everything else in the stack — is what lets an edge survive contact with the market before it decays away.

Sources and further reading

1. Orosz, G., Klaric, I. and Spevack, J.,
   “Software engineering at a proprietary trading company: Optiver,”
   The Pragmatic Engineer, 11 August 2026.
2. O’Sullivan, S.,
   “Quant trading ≠ software company,”
   FT Alphaville, Financial Times, August 2026.
3. myrtle.ai,
   “Setting a New Record for Single-Digit Microsecond Latency Inference in Capital Markets,”
   24 April 2026, including:
   • STAC Report, myrtle.ai VOLLO (Rev C), SUT ID MRTL260323, 24 April 2026 —
     stacresearch.com/MRTL260323
   • STAC Report, myrtle.ai VOLLO (Rev B), SUT ID MRTL230426, 3 May 2023.
   • STAC Report, NVIDIA GH200 Grace Hopper Superchip, SUT ID SMC250910 —
     docs.stacresearch.com/SMC250910
   • NVIDIA Technical Blog,
     “Achieving Single-Digit Microsecond Latency Inference for Capital Markets,”
     2 April 2026.

VOLLO is a trademark of myrtle.ai. STAC and all STAC names are trademarks or registered trademarks of the Securities Technology Analysis Center, LLC. This guide draws on publicly available reporting and commentary alongside myrtle.ai’s own audited benchmark results; readers should consult the full sources referenced above before making any performance or purchasing decision.