One of the first signs of trouble came at 9:41 p.m. ET on Thursday, when most of Wall Street was closed and traders were still enjoying the US Thanksgiving holiday.
“Due to technical problems,” CME group Inch. told clients in a one-line email that its futures and options “markets are halted.”
The problem turned out to be a cooling system at a data center complex in suburban Aurora, Illinois, some 50 miles (80 kilometers) from Chicago, which serves as a major hub for the trillions of dollars in derivatives traded each day. Interior temperatures soared past 100F (38C) despite the cold weather, according to people familiar with the matter.
At the time, information from CME indicated that the outage at the facility — operated by private equity-owned CyrusOne — would be brief, according to a person with direct knowledge of the situation. As a result, he decided against transferring to a backup facility near New York.
But soon the magnitude of that fateful decision will become painfully clear. Despite a flurry of emails from CME assuring clients that the problem would be resolved “shortly,” the outage continued and brought down vast swaths of the global financial system for hours. From Tokyo to London, and finally New York, trading in everything from gold and oil to bets on the direction of US interest rates came to an abrupt halt.
Even after trading mostly resumed on Friday, disruptions extended into the US session, with CME Direct, the exchange’s trading platform, offline for most of the day.
The outage highlighted the vulnerability of increasingly integrated global markets that rely on a handful of dominant exchanges. It also raises questions about the contingency plans of CME, one of the world’s largest derivatives exchanges, and its heavy reliance on a data center it sold in 2016 to CyrusOne, a company now owned by KKR & Co. and Global Infrastructure Partners.
The shutdown “shows how concentrated the futures markets really are — there just aren’t many alternative venues for the main products,” said Thomas Texier, head of clearing at Marex Group Plc, a London-based brokerage.
The 10-hour outage surpassed the one that hit the CME in 2019 and again underscored how integral the stock market has become to global markets. On average, in October, the volume of derivatives trading amounted to more than 26 million contracts each day, according to the group.
Although it came on a day when US trading was expected to be muted – due to Thursday’s holiday – it nevertheless proved unsettling for investors around the world who had to make month-end adjustments by shifting positions from one contract to another.
In Singapore, one oil trader said that when the first warning was issued, they thought it was a scam because trades and quotes were still being released. But a few minutes later the screen suddenly froze and they were kicked off the Nymex platform. A merchant in London thought there was a problem with the Wi-Fi connection.
“We had to trade some cash government bonds today and they were noticeably thinner and wider,” said James Athey, portfolio manager at Marlborough Investment Management Ltd. “At the end of the month, the day after Thanksgiving, the CME is down. It’s not an ideal combination.”
By the end of the trading day on Friday, CME had resumed all of its trading operations, including CME Direct. A representative for the exchange declined to comment beyond the updates it provided to clients throughout the day.
CyrusOne said in a statement that the problem was caused by a machinery failure that affected the systems used to cool its computer systems and that it was “working 24 hours a day to restore normal operations as quickly and safely as possible.” They said they have successfully restarted several cooling systems with limited capacity and installed temporary cooling equipment to supplement their operations.
It’s unclear what exactly happened to the CyrusOne’s cooling system. But the data center has a redundancy system and offers free cooling when temperatures drop below 30 F, according to information on CyrusOne’s website.
The 450,000-square-foot Aurora complex has served as the primary digital operations center for CME for nearly two decades. It is well-known among high-frequency traders and Wall Street firms, who have long vied for positions on the site to gain an edge over competitors by shaving fractions of a second off the time it takes to trade.
In 2016, CME decided it wanted to move away from infrastructure ownership and sold the site to Dallas-based CyrusOne. As part of the deal, CME agreed to lease space from CyrusOne for 15 years so it could continue to host computers at the location that support its market, essentially outsourcing its day-to-day operations. KKR and Global Infrastructure Partners agreed to buy CyrusOne in 2021.
The center continued to serve as a key hub for traders around the world, and the impact was felt far and wide. In London hours, for example, trading in US Treasury futures was halted, gold had erratic moves and US crude and palm oil on Bursa Malaysia were also affected.
Even when the trading system was restored, some market makers were reluctant to engage in trading until they could be sure the problem had been resolved, according to people familiar with the matter.
Others saw the event as a lifeline during a typically slow day in the US.
“I woke up thinking my Wi-Fi was down,” said Ritik Katte, chief investment officer at MCD Capital, a London-based investment firm. “Liquidity is lower than usual, so the Thanksgiving holiday seems to be extended.”
