Our first series of posts aims to introduce a few high-level principles which guide our thinking. These principles are more a mindset than specific technical recommendations, but they play a large role in and undergird a lot of our thinking. The three we will examine are:

• Constraints + Discipline = Innovation + Flexibility
• Simple Designs
• Fear

To illustrate how these play out in practice, the coming posts will examine them through the lens of a major evolution of our trading systems. They will examine a significant upgrade to the Chicago Mercantile Exchange’s iLink Architecture and our incorporation of Field Programmable Gate Arrays (FPGAs) into our trading systems. This post explains the context. What is the CME and why does it matter to Optiver? What were the implications of CME’s previous architecture? Why was this iLink change so important?

The CME and Optiver

The Chicago Mercantile Exchange (CME) is the world’s largest options and futures exchange. A wide variety of financial products trade there: agricultural futures, energy products like Oil and Natural gas, fixed income products like US Treasury bonds and Eurodollar futures, and the S&P 500 future. The S&P 500 is the world’s most important stock index, and the CME’s S&P 500 product suite is a very liquid, high-volume market in which to trade that index. It constitutes an important part of Optiver’s US business activities.

Optiver’s primary role in the financial markets is as an Options Market Maker. For any given product (like the S&P 500), there are thousands of options which have highly correlated prices. As an options market maker, our goal is to facilitate trading in as many of those options as we can, at the best prices possible, continually. To do so we leverage a few different trading strategies, all of which balance three factors:

• Theoretical Price: Our best estimate of the current value of a financial instrument like an option. We do not only calculate this for an individual financial instrument; we must also consider how these instruments relate to each other. We must ensure our theoretical prices maintain the proper correlations amongst tens to hundreds of thousands of options.
• Spread: Financial instruments actually have two prices at any point in time. The bid price is the price at which people are willing to buy. The ask price is the price at which people are willing to sell. Our goal is to make the difference between those two prices (the spread) as small as we can so that when customers come to the market to trade, they trade with us.
• Update Speed: The markets change continually. Every new trade or price update can have an impact on the price of an option. The government releases various economic indicators. CEO’s and world leaders make important announcements on social media. Financial institutions make large trades. Once this information is publicly available, all prices in the market are out of date and the race to update them begins.

CME’s iLink Architecture

Given the size and importance of the CME, we (and many of our competitors) poured a tremendous amount of resources into understanding and tuning our trading systems to work very well on the Chicago Mercantile Exchange. The core of an exchange is the “matching engine”. This is the central component which generates trades when orders to buy and orders to sell match with the same price. CME’s iLink Architecture had multiple pathways leading to their matching engine. When an order was submitted it would proceed from server to server within CME’s system until it reached the matching engine. At that point of arrival, all orders would get in line, and be matched accordingly.

These multiple pathways led to non-deterministic behavior: we could send two messages 30 microseconds apart*, and the later message could overtake the earlier message. The implication was that if we could respond to market events in under 30 microseconds, our price update had a statistically significant chance of succeeding. While 30 microseconds is very short for human reaction time or web page rendering, a trading system can do a lot in that time. We had designed our system around the fact that speed, while an important constraint, was not the primary constraint. It could be balanced with quite a bit of sophistication in pricing, maintaining correlations, exhaustive limit checks, and broad coverage of events which may merit price updates.

iLink and MSGWs

In December of 2013, CME announced plans for a major upgrade to their architecture. The key change that caught our eye was Market Segment Gateways (MSGWs). MSGWs would provide:

• FIFO ordered message handling per market segment, and
• Elimination of variability associated with multiple sessions across multiple gateways

This changed our constraints in two key ways. First, CME said it would be eliminating the parallel paths to the matching engine. No longer could we rely on our message being able to randomly jump in front of faster competition because our path was faster at this moment than theirs. Second, these new MSGWs promised FIFO ordering of messages. The first message to arrive at the MSGW would be the first message processed by the matching engine, and CME was guaranteeing quite a bit of fidelity in its ability to order incoming messages. Together, these guarantees meant that our 30 microsecond maximum response time was about to drop substantially. While latency and speed were important before, these changes elevated them to primary importance. We could no longer trade speed for sophistication and flexibility. While we knew we could get software response times under 5 microseconds, achieving response times far below that eliminated software-only solutions from the competitive landscape. To survive we would have to rethink the designs of our automated trading systems. Achieving lower latencies would require leveraging hardware to eliminate software from the critical path as much as possible.

* These latency numbers are accurate, but they are also intentionally imprecise.

David Kent, Chief of Staff – Technology

David is a Stanford Computer Science alum and spent several years as a developer at Amazon.com. He joined Optiver as a Software Engineering Lead in 2009 and has led many of Optiver’s software development teams. He is presently Chief of Staff for the Optiver US Technology Group.