High Frequency Trading: All You Need To Know

In the aftermath of Michael Lewis’ book “Flash Boys” there has been a renewed surge in interest in High Frequency Trading. Alas, much of it is conflicted, biased, overly technical or simply wrong. And since we can’t assume that all those interested have been followed our 5 year of coverage of a topic that finally has earned its day in the public spotlight, below is a simple summary for everyone.

To be sure, the thinking behind HFT is hardly revolutionary, or even new. Although today HFT is closely associated with high speed computers, HFT is a relative term, describing how market participants use technology to gain information, and act upon it, in advance of the rest of the market. Near the advent of the telescope, market merchants
would use telescopes and look out to the sea to determine the cargo hold of incoming merchant ships. If the merchant could determine which goods were soon to arrive on these ships, they could sell off their excess supply in the market before the incoming goods could introduce price competition.

That said, the real proliferation of technology in trading, started in earnest in the 1960s with the arrival of the NASDAQ, the first exchange to heavily use computers.

Ironically, while some form of HFT has been around for a long time, its true “potential” was first revealed in October 1987 with the first whole market flash crash, which resulted from an exponential propagation of program trading, which like right now with HFT, nobody truly understood. And even though some thought that Black Monday would have taught traders and regulators a lesson, it merely accelerated the incursion of computerized and algorihmic trading into regular markets, to such an extent that HFT now accounts for nearly three quarters of all exchange-based trading volume, while dark pools and other “off exchange venues” – or more markets that are not readily accessible to most – account for up to 40% of all total trading by volume up from 16% six years ago.

The rough chronology of algorithmic trading, of which HFT is a subset, is shown in the timeline below.

Over the past decade, following regulatory initiatives aimed at creating competition between trading venues primarily as a result of the overhaul of the National Market System Regulation (or reg NMS), the equities market has fragmented. Liquidity is now dispersed across many lit equity trading venues and dark pools. This complexity, combined with trading venues becoming electronic, has created profit opportunities for technologically sophisticated players. High frequency traders use ultra-high speed connections with trading venues and sophisticated trading algorithms to exploit inefficiencies created by the new market structure and to identify patterns in 3rd parties’ trading that they can use to their own advantage.

For traditional investors, however, these new market conditions are less welcome. Institutional investors find themselves falling behind these new competitors, in large part because the game has changed and because they lack the tools required to effectively compete.

In brief: The role of the human trader has evolved. They must now also understand how various electronic trading methods work, when to use them, and when to be aware of those that may adversely affect their trades.

Market venue competition began with the Alternative Trading System regulation of 1998. This was introduced to provide a framework for competition between trading venues. In 2007 the National Market System regulation extended the framework by requiring traders to access the “best displayed price” available from an automated visible market. These regulations were intended to promote efficient and fair price formation in equities markets. As new venues have successfully competed for trade volume, market liquidity has fragmented across these venues.

Market participants seeking liquidity are required by regulatory obligations to access visible liquidity at the best price, which may require them to incorporate new technologies that can access liquidity fragmented across trading venues. These technologies may include routing technology and algorithms that re-aggregate fragmented liquidity. Dark Pools – trading platforms originally designed to anonymously trade large block orders electronically – began to expand their role and trade smaller orders. This allowed dealers to internalize their flow and institutional investors to hide their block orders from market opportunists.

The use of these technologies can lead to leaking trading information that can be exploited by opportunistic traders. Information is leaked when electronic algorithms reveal patterns in their trading activity. These patterns can be detected by HFTs who then make trades that profit from them. Competition for liquidity has encouraged trading venues to move from the traditional utility model, where each side of a transaction would be charged a fee, to models where the venues charge for technological services, pay participants to provide liquidity and charge participants that remove liquidity. Many trading venues have become technology purveyors.

Broker-dealers have realized that they are often the party paying the trade execution fee, which is used by the venues to pay opportunistic traders a rebate for providing liquidity. To avoid paying these fees and internalise their valuable uninformed active flow, especially from retail customers, broker-dealers have also established dark pools. By internalising their flow or, in many cases, selling it to proprietary trading firms, they can avoid paying the trading fees that the venues charge for removing liquidity from their order books.

The irony is that in their attempt to streamline and simplify the market with Reg ATS and Reg NMS, regulators have created the ultimate hodge podge of trading venues, information leakage nodes, and countless opportunities to frontrun both institutional and retail order blocks.

Before we continue, let’s take a look at perhaps the most critical and misunderstood concept around, one which HFT advocates are quite happy to (ab)use without really understanding what it means.

There is more: as we explained back in August 2009, the correct term to focus on isn’t liquidity, but Implementation Shortfall, also known as Slippage, which is the toll HFTs collect from investors – this is, on average, the cost of spread and frontrunning. Implementation Shortfall (IS) Costs – comprised of 2 pieces: Timing Delay Costs – Any delay cost incurred between the Initial Decision (Open on Day 1) and the Broker Placement Price. Think of this as the cost of Seeking Liquidity; and Market Impact Costs – Price change between the time the Order is placed with the Broker and the eventual trade price. (those curious to learn more about the nuances can do so at this link).

Why is liquidity so critical? Because it goes hand in hand with the concept of the modern exchange, since the measure of consummated liquidity is a key variable in determining the successfulness of any trade venue. It also goes to show why HFTs never operate in a vacuum but in explicit symbiosis with exchanges. It was Zero Hedge who pointed out in 2012 that HFT is a critical component of exchange revenue streams, ranging anywhere between 17% and all the way up to 32%.

It is this inextricable link between the venue and the algos that dominate the venue, that has led many to suggest – correctly – that one of the key culprits for HFT proliferation is the dominant exchange business model, known as the Maker-Taker model, in which the liquidity provider is paid (in practical terms it means paying those who provide liquidity with limit orders even it the limit orders are merely “flashed” subpenny orders frontrunning a major order block), while charging liquidity takers (those who take away liquidity with market orders). This is summarized in the panel below.

No matter the reason, one thing is certain: the use of HFT has exploded.

With the equity markets becoming electronic and prices quoted by the cent (as opposed to the previous eighths of a dollar), the traditional, “manual”, market makers have found it difficult to keep up with the new technologically savvy firms. The playing field has been tilted in favor of HFTs, who use high speed computers, low-latency connectivity and low latency direct data feeds to realize hidden alpha… or as some call it – frontrunning.

HFTs can follow active, passive or hybrid strategies. Passive HFTs employ market making strategies that seek to earn both the bid/offer spread and the rebates paid by trading venues as incentives for posting liquidity. They do this efficiently across many stocks simultaneously by utilizing the full potential of their computer hardware, venue-provided technology and statistical models. This strategy is commonly known as Electronic Liquidity Provision (ELP), or rebate arbitrage.

These ELP strategies can also be signal detectors. For example, when ELP strategies are adversely affected by a price that changes the current bid/ask spread, this may indicate the presence of a large institutional block order. An HFT can then use this information to initiate an active strategy to extract alpha from this new information.

Active HFTs monitor the routing of large orders, noting the sequence in which venues are accessed. Once a large order is detected, the HFT will then trade ahead of it, anticipating the future market impact that usually accompanies sizable orders. The HFT will close out their position when they believe the large order has finished. The result of this strategy is that the HFT has now profited from the impact of the large order. The concern for the institutional investor, that originally submitted the large order, is that their market impact is amplified by this HFT activity and thus reduces their alpha. The most sophisticated HFTs use machine learning and artificial intelligence techniques to extract alpha from knowledge of market structure and order flow information.

The ubiqituous presence of HFT also means that one of the key considerations when placing an order is “smart order routing” which take into account such concepts as latency arbitrage and order size. This is furhter simplied in the panel below.

Which brings us to the topic of whether all HFT does is simply frontrunning, legal as it may be, and allowing firms like Virtu to post “liquidity providing”,”trading” profits on 1,237 of 1,238 trading days. The answer – no. At least not explicitly. The full list of HFT strategies, broken down by their impact on various stakeholders is shown below. Again, at least on paper, some strategies are beneficial if mostly to the retail investor. The biggest question, however, is – is there such a thing as a retail investor left at a time when market trading volume has fallen to decade lows, and where HFT now comprises the bulk of lit volume.

And while on paper HFT does provide benefit, the reality is that in practice the consequences of HFT are almost unique negative. Putting aside the ethical implications of whether one views frontrunning as legal or not, the far bigger unintended consequences of HFT is that it has made trading venues inherently far more unstable and prone to sudden and unexplained crashes. Indeed, the market has suffered several adverse events as a consequence of the new fragmented, for-profit, market venue environment. In some cases, these events resulted from the unpredictable interaction of trading algorithms; in other cases they were the result of software glitches or overloaded hardware.

KNIGHT CAPITAL LOSS – OVER $450 MILLION + WAVES OF ACCIDENTAL TRADES

A software malfunction from Knight caused waves of accidental trades to NYSE-listed companies. The incident caused losses of over $450 million for Knight. The SEC later launched a formal investigation.

GOLDMAN SACHS – $10S OF MM + TECHNICAL GLITCH IMPACTS OPTIONS

An internal system upgrade resulting in technical glitches impacted options on stocks and ETFs, leading to erroneous trades that were vastly out of line with market prices. Articles suggest that the erroneous options trades could have resulted in losses of $ 10’s of millions. Goldman Sachs stated that it did not face material loss or risk from this problem.

NASDAQ – 3 HOUR TRADING HALT DUE TO CONNECTION ISSUE

Due to a connection issue NASDAQ called a trading halt for more than three hours in order to prevent unfair trading conditions. A software bug erroneously increased data messaging between NASDAQ’s Securities Information Processor and NYSE Arca to beyond double the connection’s capacity. The software flaw also prevented NASDAQ’s internal backup system from functioning properly.

NASDAQ – DATA TRANSFER PROBLEMS FREEZE INDEX FOR 1 HOUR

An error during the transferring of data caused the NASDAQ Composite Index to be frozen for approximately one hour. Some options contracts linked to the indexes were halted, though no stock trading was impacted. NASDAQ officials state that the problem was caused by human error. Although the market suffered no losses, this technical malfunction – the third in two months – raises considerable concerns.

* * *

Which brings us to the culmination of 50 years of changing technology, namely the changing investor-broker relationship.

Traditionally, investors spent their efforts seeking alpha and brokers were charged with sourcing liquidity. Liquidity could be sourced via the upstairs market or the stock exchange. The stock exchange operated as a utility that consolidated liquidity. Beyond generating alpha, the only decision for an investor was choosing a broker to execute their trades. Today, investors are still concerned with generating alpha. However, the trading process required to execute their alpha strategies has become more complex. The consolidated utility model has been replaced by a market that is highly fragmented with for-profit venues vigorously competing for liquidity which is provided primarily by HFTs.

This new environment puts brokers in a difficult position. They have a fiduciary responsibility to provide best execution to their clients. This requires them to invest in new technology to source liquidity and defend against HFT strategies. And because many of these venues now pay rebates for liquidity, which is quickly provided by HFTs, brokers are usually left having to pay active take fees to the venue. And at the same time that brokers are incurring these costs, investors are pressuring them to reduce commissions.

These pressures on brokers’ margins are creating conflicts of interest with their clients. By accessing venues with lower trading fees, or attempting passive order routes of their own, brokers can reduce their operating costs. However, these trade routes are not necessarily best for the investors.

Sophisticated investors now demand granular execution information detailing how their order flow was managed by their broker so they can ensure they are receiving the best execution. While brokers provide aggregate performance reports, investors can build a more complete analysis, including broker performance comparison by using more granular information.

Summarized visually – Before:

And After:

* * *

So putting it all together, what is the current state of the market? Ironically, when one strips away all the bells and whistles of modern technology, it all goes back to a concept as old as the first market itself – namely alpha, or outperforming the broader market.

In order to find hidden alpha, it is important to first understand where market participants are with respect to information utilization. The light grey area in the chart below represents the typical institutional investor, playing the role of the “ostrich” or “compiler”, either choosing to ignore the changes around them or to use information only for basic compliance tasks. Most HFTs belong in the light blue “commander” stage; they take command of the information around them and let it guide their business. Taking advantage of the information opportunity, and finding hidden alpha, requires a firm to move up the stages of adaptation.

COMPLEXITY: This measures the sophistication of the use of information in directing action. Whether the information is trade data or newsfeeds, it can be put to use in more or less sophisticated ways, from simple arithmetic to complex statistical methods coupled with strong strategic understanding. Arithmetic uses aim at providing no more than basic accounting measures of values, volumes and gains and losses. Statistical methods aim to identify patterns in information that can be used to guide trading. Strategic understanding introduces game theory, anticipating the reaction of other market participants when an investor employs a particular trade strategy.

FREQUENCY: Each trade an investor makes provides an opportunity to learn. Gathering information from every trade, as opposed to a select few, helps give the investor a better understanding of how those trades may perform in the future. The more frequent the analysis, the more relevant the findings will be.

ITERATION: Findings serve a purpose only if they are acted upon. The key is to use information to guide actions whose outcomes are then analyzed and the findings reapplied. This creates a continuous iterative loop that drives towards ever greater efficiency.

BREADTH: Knowledge sharing with similar objectives (e.g. institutional investors trading large blocks) could lead to a more efficient investment implementation process for all participants. Working together, institutional investors can share block order implementation experience and data, as a utility. The result of this could help participating institutional investors defend against market impact losses and protect proprietary strategies.

HFT firms will likely plateau at stage 4, “commander”, as they are less likely to share any information in a utility concept; trade execution is their proprietary intellectual capital. Institutional investors, on the other hand, have the potential to reach stage 5, “optimizer”. For institutional investors, their proprietary intellectual capital usually lies within their investment decisions, not their trade implementation routes. Institutional investors are thus more willing to collaborate with eachother to work against trade strategies that cause them market impact.

Regardless of an investor’s disposition towards trading strategies; leveraging advanced technology or committing to more traditional trading strategies, it is important to realize that advanced technology trading is today’s reality. Investors need to strongly consider taking the appropriate steps to protect against the potential negative repercussions of, as well as position themselves to find the hidden alpha within, today’s advanced market.

* * *

So the bottom line: HFT is legal frontrunning… but also so much more. In fact, like the TBTF banks, HFT itself has become so embedded in the topological fabric of modern market structure, that any practical suggestions to eradicate HFT at this point are laughable simply because extricating HFT from a market – which indeed is rigged but not only by HFTs at the micro level, but more importantly by the Federal Reserve and global central banks at the macro – is virtually impossible without a grand systemic reset first. Which is why regulators, legislators and enforcers will huff and puff, and… end up doing nothing. Because if there is one thing the TBTF systemic participants have, is unlimited leverage to collect as much capital due to being in a position of systematic importance in a market, rigged or otherwise.

Finally, if push comes to shove, and the presence of HFT is threatened, watch out below, because if HFT’s presence, glitchy as it may have been, led to the May 2010 flash crash and the subsequently unstable market which has exhibited at least one memorable crash every single month, then the threat of pulling the marginal trader which now accounts for 70% of all stock churn and volume (if certainly not liquidity) would have consequences comparable to the Lehman collapse.

* * *

Finally, for all those still confused by HFT, here is the ultimate simplification.