Dan Barnes
Dan Barnes

Analytics, models and algorithms – pressure builds on FX Market Data platforms

Market data is the lifeblood of the FX trading community; and despite a fall in trading volumes last year, Dan Barnes investigates how trading automation could generate growth that would swamp existing data infrastructures.

First Published: e-Forex Magazine 51 / Features / April 2013

The foreign exchange markets have not been sheltered from the bearish outlook of 2012; daily average market volumes fell year-on-year to US$4.3 trillion from US$4.7 trillion in 2011 according to research firm Celent. Nevertheless growth is predicted for 2013, driven by increased levels of high-frequency trading and the launch of new electronic trading venues. The higher speed of trading and potential for exponential growth based on HFT strategies is putting pressure on the ability of market participants to process market data. 

“We’re seeing an increase in the number of market data sources, the firms participating in the markets, and as a result the sheer volume of quotes is increasing,” says Nick Deacon, senior director, EMEA at SAP. “We’re starting to see the knock-on effect of market data volumes increasing from various aggregators and sources. That means that firms have to beef up their infrastructures. What they are finding is that not only do they have to deal with an increase in the volume of market data but they need to focus on the quality of market data. Where their algos and pricing engines are using live market data to make trading and execution decisions they need to ensure that the data feeding through to those engines is of the highest quality.”

Sreekrishna Sankar, senior analyst at Celent and author of the report ‘Institutional FX Market: Changing business models and evolving market structure’ has observed that HFT as a driver has been pushing against the downturn in volumes in the dealer-to-client (D2C) segment which itself reflects the severe economic situation for many firms.

“The manual segment and the non-HFT algorithmic traders have been struggling to compete in a HFT-centric market, where gaming and similar strategies have been on the rise,” he wrote. “The drop in order/fill ratio to as low as 5% for certain platforms and industry estimates of around 20% reflects the impact of gaming strategies. The emergence of new platforms catering to the spot market has been driven by the need of the banks and the traditional traders to have a fair and transparent platform where they can compete on a level playing ground with the other industry participants, including the hedge funds, which practice high frequency trading strategies.”

As HFT strategies took off in the cash equity and derivative asset classes, the growth in trading volumes was significant, providing considerable support during the downturn as traditional firms reduced activity. Its increase in the FX market likewise creates a potential for a higher speed of growth than has been seen in the before. The lowered fill rates that occur due to the higher number of cancelled orders, a feature characteristic of HFT strategies, frustrates manual traders in the meantime.

Nick Deacon  “Now firms are not only dealing with more data but 
it is of differing types from different sources.”

Nick Deacon

“Now firms are not only dealing with more data but it is of differing types from different sources.”

Furthermore, there is a widening of the currency pairs that traders of all sorts need access to, asserts Ryan Moroney, product manager for market data and enterprise solutions at trading technology provider CQG, making access to a broad range of market data crucial to support operations.

“Recently you’ve seen in the news that the Chinese currency is becoming one of the top 14 in terms of volume and validity as a globally traded currency,” he says. “One of the things we have noticed in the last six months is the interest in rupees. In the futures world the Dubai Gold and Commodities Exchange is hitting new records every month because of the Rupee-US dollar contract, to the point that CME and ICE have launched contracts. If you look back to 2009-10 everyone was trading euros and US dollars. Although overall volumes have come down since then, the market has also become more diverse.”

With new markets launching, the potential for arbitrage trades grows. The ability to access multiple providers has increased and although spreads have tightened over the last twelve months; if a venue’s spreads are off, it is very easy to arbitrage between them and other markets. 

“You always saw tight spreads in the institutional world but in the retail world that wasn’t the case, says Moroney. “When you look at eSpeed, Currenex or HotSpot their spreads have become much tighter, closer to what you would see on EBS. We see the market growing.. We’re actively moving into a route of offering more FX platforms and electronic trading systems that people can use with our spreading and aggregation product to trade. To aggregate effectively firms need as much data as they can get, so they can take advantage of whatever different opportunities there may be. That is our focus. We are trying to grow what we do and we are working with liquidity providers to have some of the normalisation done on their side so it takes some of the guess work of decoding out of the process.”

Ryan Moroney“If you’re not already in the data aggregation game, 
pulling data together is a big problem.”

Ryan Moroney

“If you’re not already in the data aggregation game, pulling data together is a big problem.”

Value in data

Louis Lovas, director of solutions at OneMarketData, says that understanding data is a game changer but that data management across a multitude of growing markets can be a messy business. 

“Tighter spreads and increased competition pushes firms to capture and store more and more data from numerous sources,” he says. “Poorly designed or legacy systems can easily translate to spending inordinate amounts of time and resources processing data, outfitting new storage and scrambling to deploy new systems.”

“If you’re not already in the data aggregation game, pulling data together is a big problem,” says Moroney. “Everyone supplies their data in a different format. For us that is not really hard because we have been aggregating data from different sources for more than 30 years. We are looking at growing our FX exposure but we are used to taking something in an arbitrary format, standardising it and sending it along. If you are just getting into the business of aggregating multiple sources it is a major challenge. In addition to normalising the format you are having to synchronise the times. You can’t be time stamping when you receive the prices, you have to stamp when it originated.”

High-frequency and event-driven trading both require low-latency access to data so that automated trading systems are able to react in time to exploit circumstances to make the required margin demanded by a strategy.

“Fast access to market data is of critical importance for strategy decision time,” says Lovas. “The immediacy of pricing data often from multiple sources where books need to be consolidated and possibly currency converted. This is moving closer to being full hardware enabled. This is important to specific trading styles such as spreads and pair trading – to prevent legging.  HFT looks to exploit the price imbalances that lend themselves to arbitrage opportunities. Proximity hosting can be a vital aspect of this.”

“One significant challenge is managing scale,” he observes. “As trading firms diversify the demand to capture and store more and more data from numerous sources means dealing with the idiosyncratic nature of a fragmented market. That places enormous demands on IT infrastructure. Thankfully, the improvements in compute power and storage per dollar have made the consumption both technically and economically possible.” 

It is not only FX spot traders that require the data; often derivatives traders will have an FX leg to their trades which demands accurate and timely information. CQG’s core customers are futures traders and firms trading options. Their demographic is typically looking at someone interested in trading currencies and also traders hedging quant position trading commodities such as corn, who may need to mitigate any currency risk between the Brazilian Real and the US dollar.

Louis Lovas “As trading firms diversify the demand to capture and store more and more data from numerous sources means dealing with the idiosyncratic nature of a fragmented market.”

Louis Lovas

“As trading firms diversify the demand to capture and store more and more data from numerous sources means dealing with the idiosyncratic nature of a fragmented market.”

The heart of the business

Establishing the right technology at the heart of the business is imperative whether it is seen as a cost of doing business or a potential source of profit. Celent has identified the focus for technology deployment in the FX space on superior pre-trade capabilities, “…in terms of both content generation and distribution. Superior analytics tools as well as mobility will become important for customer satisfaction.”

At present the level of technology employed by firms varies considerably, with larger more established businesses typically struggling with a greater legacy of outdated systems than smaller firms.

“Often we see firms with older technologies, which will have originally come from the traditional market data providers,” says Deacon. “The whole FX market place has become much more disparate with different sources coming in from different venues. Now firms are not only dealing with more data but it is of differing types from different sources. Typically what many firms will have is a market data infrastructure where they will typically route feeds into that receiving infrastructure. That is then used to distribute data to the downstream consuming systems so they may be pricing systems, proprietary trading systems etc. Each of those different systems will potentially consume different classes of market data. They not all take exactly the same information streams, they will have their own specific data that they will consume. So an important function for the infrastructure is to distinguish what the various downstream systems need and to ensure that those systems receive the most timely and accurate data.”

Larger firms will typically have bigger infrastructures in place which may not just be supporting the FX operations, but may also be supporting many other areas where FX information is consumed. For those firms it is a tougher challenge to adapt to the latest technologies. However tools like complex event processing can enable them to improve their environment.

Deacon continues, “They need to upgrade their infrastructures not only to deal with different sources but also to deal with bursts of activity you can potentially see in the market. Where a major event triggers extreme activity around the FX trading area then they need to ensure their systems can handle those bursts of activity as this is where where the money is made or lost. It’s crucial when firms are dealing with extremes in the market that the firms are dealing with the very latest information.”

Accessing data sources is being made easier through the use of simple APIs that provide and easy access to firms that aggregate data. By deploying the latest in virtualised systems there is no need to deliver the technology on-site for clients. 

“Cloud deployments provide advantages in managing the scale through higher levels of data protection, commoditisation and fault tolerance at a cost savings,” says Lovas. “Centralised storage and infrastructure for real-time and historical data and tools for analytics i.e. the fund in-a-box platform, removes the headache of trade infrastructure management and maintenance from a firm’s daily chore. Consequently firms can focus on what matters most - data analysis, model design and validation.” 

One firm which offers a cloud-based data provision is Xignite a supplier of aggregated market data, including real-time quotes for over 140 currencies. The company’s service allows for easy integration of FX data for use in mobile apps or to display on a website. 

“Even outside of the trading world companies have a need for FX data - for expense reporting, procurement, even online global pricing,” says Stephane Dubois, CEO at Xignite. “There are hundreds of thousands of firms who need real-time and historical FX rates.” 

Xignite delivers its real-time, historical and reference market data in the cloud. This method offers a significant level of flexibility, allowing users to locate just the slice of market data they require and easily integrate it into any application, using just about any technology. 

“We try to simplify access to the market data,” he explains. “When we first started the company we had to use legacy technology and feed FX information out of a terminal. FX data is not that complex and we felt it should not be that difficult to integrate it into an application. We try to make access extremely simple and to do that we use the cloud and provide application programming interface (APIs) which are extremely scalable.”

An additional efficiency exists for firms who are deploying an application into the cloud as they can reduce infrastructure headaches and hosting expenses and pull the market data into their applications or databases directly over the Internet. That can be particularly useful for firms running algorithmic trading testing. 

Dubois explains, “One use of historical data is to back test trading strategies, calculating how a strategy would have performed had it actually been applied in the past.” 

There are limits however. The cloud cannot support very low latency trading down to the microsecond level, although Dubois asserts it is not hard to get millisecond performance. That means for firms focused on HFT or those that require high-speed reactions to market events, a more physical solution is required to deliver data. 

Sreekrishna Sankar“The emergence of new platforms catering to the spot market has been driven by the need of the banks and the traditional traders to have a fair and transparent platform where they can compete on a level playing ground with the other industry participants, including the hedge funds, which practice high frequency trading strategies.”

Sreekrishna Sankar

“The emergence of new platforms catering to the spot market has been driven by the need of the banks and the traditional traders to have a fair and transparent platform where they can compete on a level playing ground with the other industry participants, including the hedge funds, which practice high frequency trading strategies.”

What is at risk?

The ability to accurately test a system’s capabilities should not be underestimated, nor should the risk for firms that are unable to manage support the processing required to manage the scale of incoming data is significant.

“If you feed those engines with poor quality data then you will make poor decisions and potentially expose yourself in the market,” warns Deacon. “It is essential that firms keep up to date and ensure that the flow of market data through their systems is not only accurate but also that they can keep up with burst volumes and that no backlog of market data builds up. Your systems must be feeding the very latest rates to your engines.”

Strategy modelling and trade performance analysis are dependent on accurate, clean data in order to generate alpha and manage risk, notes Lovas, with FX data proving crucial to other asset classes.

“Most important is the challenge to achieve the needed timeliness and quality in the data dump for trade modelling, back-testing and investment decisions. Front office solutions target price discovery and analysing market trends. Data quality must be attained because of what is at stake – profit making and lowering, controlling risk,” he says. “FX rates can be sourced from liquidity providers across a fragmented market, constructing a Best Bid Offer (BBO) is an exercise left to the user, there is no NBBO. Yet, the most important factor in determining a bond yield curve is the currency in which the securities (e.g.  Government bond) are based.”

He continues, “Yield curves are closely scrutinised as a predictor for future interest rate changes and economy growth. Curves are a reflection of inflation risks and the effectiveness of monetary policy and heavily influence investment decisions for the fixed income analyst. Upward sloping, positive yield curves indicate economic growth therefore investors may receive a better rate in the future. Conversely, if the curve get inverted it can be a sign of upcoming recession. Curve building is very dependent on clean/scrubbed rates for curve validation. The cleansing process would ensure valid data – rate-adjusted instrument prices determined by factoring out anomalies, i.e. they are within 1.5 standard deviations of recent past.”

Stephane Dubois “In the FX market, you typically have only a partial view of the pricing data. There is no consolidation of market data and there is demand for this.”

Stephane Dubois

“In the FX market, you typically have only a partial view of the pricing data. There is no consolidation of market data and there is demand for this.”

Standardisation is key 

CQG has to make sure that all of its data is integrated and actionable so that a client trading corn in Brazil will have the ability to match up that data with the exchange of the dollar and the real. The firm makes sure that, through its analytics platform, it is not only getting the user data for the corn prices and the Brazilian real prices, but also providing the ability to put them together to make a decision based upon all of those dynamics. 

“There are a lot of challenges dealing with data coming in different formats and parts of the world and it can take different times to reach you,” explains Moroney. “So do the HFT guys who care very much about timing so mixing those things together correctly is crucial.” 

Typically different firms handle high speed data in a number of different ways. Some will ensure their infrastructure is upgraded to always be able to consume the data at the highest rate it can be received. Others will use technology like complex event processing to prioritise the data, to filter out the ‘background noise’ data and then deal with the more critical pricing data, effectively conflating the flow of data down to a lower level,  acting as a filter up as passing the higher priority data.

There are a number of different vendors looking at distributing market data, some focussed on the low-latency space, other slightly more traditional but offering a wide range of choice. 

“A lot of the different firms use old message formats but many are moving towards newer, more standardised formats such as FPML for describing their different types of instruments,” says Deacon. “That is becoming an area where firms are focusing because of you have a standard message format you and your client can connect much more easily, where if you have a proprietary format it is much more of a challenge for your customer to connect with you. We also see many clients using point-to-point connectivity where they really want to the optimum performance from connecting two systems. They might avoid going through a traditional market data delivery infrastructure and actually connect the systems up directly together.”

Format is not only a challenge in the receiving of data, it is becoming a bigger issue as clients utilise a broader range of technology, often mobile and with a range of operating systems.

“A few years ago everyone was using Windows-based platforms,” notes Moroney. “Now you have to deliver data so that people can use it in a cloud-based solution or on an iPad. So you have to create a more flexible way of sending market data so you aren’t limited to using Microsoft platforms. That has taken effect in the last three years, delivering data efficiently through different technologies that are optimised for different platforms. Instead of trying to make the data generic we try to make the interface that sends out the data pretty flexible and then we create specific interfaces for different platforms and optimise for that platform so we are not compromising the integrity of the data.”