Solving problems at scale

The allure of “problems at scale” is significant for researchers aspiring to transition from academia to the private sector. At Optiver, we are constantly scaling up in every dimension – adding more features, models, financial exchanges on which we trade; and expanding our range of products, asset classes and geographic colocations. Additionally, we are increasing our data capacity, computing power, throughput and reducing latency. Scaling up is our top priority. We focus on expanding capabilities rather than merely optimizing within existing constraints.

• Scale of models: In the constantly changing environment of the capital markets, the demand for large-scale models is relentless. These models are complex and must generalize well across different scenarios. Innovation and improvement are perpetual, making every aspect of our work an ongoing engagement with research questions.
• Scale of data: As data increasingly becomes recognized as the new oil, Optiver provides access to an almost unlimited supply of data across various contexts and varieties. This access is exceedingly rare and sets us apart from traditional machine learning research in the community.
• Scale of implementation: At Optiver, the solutions we develop must also be scalable. This introduces a unique set of challenges, requiring implementations that not only meet current needs but are also adaptable to future demands and expansions.

The ML opportunity

Market data exhibits a structure distinctly different from domains traditionally associated with deep learning research, such as vision and language. This uniqueness presents numerous opportunities for groundbreaking research. Currently, we’re exploring custom deep learning layers and architectures that capitalize on the distinct properties of market data.

Our challenges include analyzing multi-dimensional time series data with sequence lengths in the tens of millions. This complicates the learning of long-term contexts from both research and engineering perspectives. Additionally, the highly non-stationary nature of the data, combined with a very low signal-to-noise ratio, poses challenges in developing robust models that generalize effectively to unseen data.

Currently, our data modeling involves petabytes of information, but there’s potential to use much more. This is only constrained by our engineering capabilities, which are expanding each day. On the deep learning front, we have the freedom and flexibility to pursue full innovation. There is significant potential to discover new signals, or “alpha,” which are unattainable with our current systems.

However, we are not looking to replace traditional models; rather, we are investing more in them, with significant efforts to train and infer at scales 10 to 100 times greater than before. Our approach to machine learning inference, paired with ultra-low latency execution, sets us apart from major tech companies, where the focus is generally on batching, throughput and maintaining millisecond-level timing. Our targets are measured in microseconds and nanoseconds and we are constantly striving for tighter timing.

We develop both general and specialized models for a diverse array of products, from stocks and treasury bonds to oil futures. Our research and development efforts are not limited to broad strokes or extremely detailed aspects; there is room for initiative and innovation across the entire spectrum.

Research in the trading industry

The trading industry represents a multidisciplinary problem for researchers and engineers to innovate faster than the market. We release new functionalities and model changes daily, creating very short feedback loops that quickly show the merit of your work. You can code something today, test it thoroughly, and see it deployed in live trading the very next day, allowing you to immediately gauge its effectiveness.

You’ll find yourself among a diverse group of like-minded engineers and researchers, all focused on solving problems and making thoughtful advances efficiently. We do not rush ideas that are not fully developed. Intellectual and scientific rigor are maintained and revered in this environment. There’s ample space for owning problems, exploring new ideas and advancing in your field, all within a highly collaborative and transparent culture.

There is also an opportunity to pursue your own ideas and make significant impacts on our research stack. We hold no concept as sacred; everything can be improved and we are always seeking to innovate. Each vertical within our company offers you direct impact on the business and, more importantly, immediate feedback on your contributions. This aspect of our industry – the ability to rapidly test and receive feedback on research ideas – proves exceptionally compelling.

If you are interested in further exploring career opportunities at our firm, we encourage you to visit the below links and apply to our open roles.