Cambridge, UK, December 15th, 2022 – Myrtle.ai, a recognized leader in accelerating machine learning inference, today announced its VOLLO™ product has achieved the lowest latencies yet measured for the STAC-ML™ Markets (Inference) benchmarks. Created by the STAC Benchmark Council™, which includes the world’s largest global banks, brokerages, exchanges, hedge funds, proprietary trading shops, and asset managers, as well as more than 50 leading technology vendors, these benchmarks are a tool used to objectively compare different platforms for latency, throughput, efficiency and quality in machine learning (ML) inference.
VOLLO achieved latencies as low as 24 microseconds with a throughput over 170k inferences/second1. The unrivaled latency and throughput enable users to make more intelligent decisions using more complex models faster than in the past, giving them a competitive advantage in trading, risk analysis, quotes and many other trading-related activities. Designed for simple and quick installation on co-located servers, VOLLO also enables reductions in rack space and energy consumption, two premium resources in such locations.
VOLLO runs on a standard form factor Intel FPGA-based PCIe accelerator card, up to 4 of which can be installed in a 1U server. VOLLO can be configured with customer-trained models, utilizing model architectures from the LSTM model zoo, enabling users to deploy a range of workloads specific to their application requirements via an export process from standard Machine Learning tool flows. It can support up to 12 parallel models per FPGA accelerator card installed in the system, enabling a maximum of 48 parallel models for a system with 4 cards installed.
A key enabling technology is the Intel Agilex F-Series FPGA. “Intel Agilex FPGAs are a superb delivery platform for our new fintech inference solution”, said Peter Baldwin, CEO of Myrtle.ai. “Agilex FPGAs have enabled us to quickly achieve great performance on these new machine learning benchmarks from STAC. Hardened bfloat16 support and sufficient on-chip RAM makes it ideal for accelerating the recurrent networks in the STAC benchmarks, wherever they are deployed, as those workloads evolve and scale”.
As a Titanium Partner in the Intel Partner Alliance, Myrtle.ai has been supported by Intel in the development of VOLLO. “These results demonstrate what can be achieved when you join Intel’s leadership FPGAs with Myrtle’s AI/ML accelerator expertise,” says Jim Dworkin, Vice President and General Manager of the Cloud & Enterprise Acceleration Division in Intel’s Programmable Solutions Group. “We’re pleased to work with Myrtle on pairing its remarkable technology with Intel Agilex FPGAs for the benefit of our customers, and we thank STAC for its leadership in defining relevant benchmarks for the industry.”
“STAC benchmarks are specified by financial firms based on their business needs, and they designed this benchmark to compare inference performance across multiple architectures,” commented STAC® President, Peter Nabicht. “Myrtle.ai is the first company to obtain audited STAC-ML benchmark results using FPGA technology, providing financial firms with valuable data points as they seek out the best solution for their needs.”
VOLLO is available today and evaluations can be arranged. For more details go to myrtle.ai/fintech or contact Myrtle.ai today at fintech@myrtle.ai.
More information about STAC and the STAC Benchmark Council can be found at www.STACresearch.com. The full benchmark results are available in the STAC Report (SUT ID MRTL221125) at www.STACresearch.com/MRTL221125.
1 STAC-ML.Markets.Inf.S.LSTM_A.4.LAT.v1 and STAC-ML.Markets.Inf.S.LSTM_A. 4.TPUT.v1
