Over a month has passed since the National Stock Exchange suffered a major embarrassing outage. It affected millions of investors. What it is that happened? Why are the details still not available to the public. Have we learned any lessons from it, asks SUCHETA DALAL.
——
THE finance ministry’s intervention has probably led to a few quick announcements by the capital market regulator on the embarrassing outage at the National Stock Exchange (NSE) on 24th February. The Reserve Bank of India (RBI) Bulletin for March also seemed to contradict NSE’s claim about ‘instability of telecom links’ and said the problem was at its clearing corporation.
After the 24th February market outage, there was a month-long silence from NSE and the Securities and Exchange Board of India (SEBI). Then two separate communications on 22nd March were issued after SEBI’s governing board and the Technical Advisory Committee (TAC) met, without really throwing the light on core issues. SEBI laid out a long list of do’s and don’ts for exchanges to deal with technical glitches and disasters, including plans for unannounced live trading sessions with just four hours of notice after April and 45 minutes’ notice after July 2021. It also called for clear communication protocols for announcing problems and resumption of trading.
On the failure to invoke the main DR (disaster recovery ) at Chennai, NSE has stayed silent for a month and reiterated that it “considered all the available alternatives on hand” and decided that getting the main system back on track was the best option. SEBI’s governing board has now asked the Exchange to ‘determine’ why it failed to switch to the DR and ‘fix individual responsibility’ for it in 21 days.
It will be interesting to see if it does.
Now, let us examine whether we are any wiser today, after the two communications, about what happened on 24th February on issues that matter to investors.
Risk Management and Safety
NSE’s press release starts with broad statements such as the Rs 900 crore per year that it has spent on technology, how other global exchanges (notably Australia, New Zealand, Japan, Germany and UK) faced outages too; and that its daily trading volumes in equity and equity derivatives segments are up 122% and 79% in 2021 compared to 2019.
NSE has, indeed, managed to improve trading volumes and its own profits, but building larger volumes and increased technology-spending doesn’t matter if it does not lead to investor safety as well.
The fact that it has had nearly 22 brokers expelled or declared defaulters in a shorter period (November 2019 to March 2021) reflects poorly on NSE’s risk management and supervision, especially when failure to act in time has caused investors to lose several hundred crore of their savings.
Even as I write this column, NSE has expelled Bezel Stock Brokers Private Limited (on 25th March), after Reflection Investments on 12th March and Sampoorna Portfolio in December 2020.
It withdrew the trading rights of Arcadia Share and Stock Brokers in February.
Specific Failure
On the specific problem of 24th February, NSE says, “There is synchronous data replication” between its primary site and the near-DR a couple of kilometres away. This is designed ‘to ensure no data loss’, in case of primary site failure, and the data is also supposed to be replicated at its DR site at Chennai, which is expected to take over when the main site fails. As we know, nothing worked that day.
NSE now attributes the problem to “instability in (telecom) links from both service providers primarily due to digging and construction activity along the path between the two sites.” It also points to “unexpected behaviour of the Storage Area Network (SAN) system, with the primary SAN becoming inaccessible to the host servers.” This affected the risk management system at NSE Clearing, impacting clearing and settlement, index and surveillance systems. It claims to have removed the SAN software which did not confirm to its design requirements, says the Exchange.
One of our readers, PM Bhate, put it most simply for those who trade on the Exchange: “Why was the software accepted in the first place? What prevented NSE from demanding that the failure logic should be documented and communicated to NSE so that its appropriateness and compatibility with NSE’s setup could be confirmed?” He suggests that this is equal to a car company absolving itself after an accident by saying, “We deeply regret the death of passengers but, you see, the airbag vendor-supplied airbags that did not confirm to our stated design requirements.”
Updating Indices at Near-DR
The NSE release still does not answer key issues. If the problem was between telecom links at NSE’s head-office and near-DR, then the main site should have worked perfectly well if de-coupled from the near-DR at Kurla. There would have been no-disaster/outage and the telecom links would have been quietly fixed. Clearly, this did not happen for other reasons, which are unknown as yet. What exactly is housed at the near-DR facility at Kurla which is crucial to the functioning of the main exchange? Neither NSE nor SEBI have thrown light on the issue.
The Hindu BusinessLine and CNBC have reported that NSE’s indices were being updated at the Kohinoor near-DR for some time now and we know that the problem began when indices stopped updating at 10.07 am. NSE shut the exchange a good one hour and 40 minutes later. When asked, NSE told the paper “We will not be able to comment on the technical details of the incidents at this stage.”
Moneylife posed the same question to the NSE. We also asked, if data travels to the near-DR and back for index calculation, wouldn’t this give an unfair advantage to high-frequency traders whose servers are within the exchanges, since their algorithms would calculate the indices faster and put them way ahead of all other investors. NSE emphatically denied that data travels out of the main site. It said, “NSE Indices system is housed in Primary data center at BKC along with all other systems.
There are no computations that happen at near-DR site for any of the applications. Index dissemination system provides a multicast-based dissemination of index values to all participants.”
But if the index calculation happens at the main building, it still does not explain why the massive outage on 24th February began with index calculation going for a toss nearly 100 minutes before the Exchange shut down. Interestingly, NSE’s communication had said, “We are also exploring alternate solutions to de-risk dependency of critical applications to a single storage device.” We wonder whether index calculation is one of these critical functions.
Curiously enough, the digging and construction activity that affected NSE does not seem to have affected broker connectivity or impacted any of the major banks and corporates at the Bandra-Kurla complex, including SEBI and the diamond bourse.
Failure To Communicate
How about NSE’s failure to provide adequate information to investors? Here again, NSE stubbornly sticks to its stand saying that it decided to reopen the market at 3:17 pm “only after there were visibility and clarity on resumption of services and any prior communication would not have been appropriate. We reiterate that we could not have communicated sooner…” The question then is: Why was the market restarted at all, knowing full well that mandatory squaring off had already inflicted losses on investors? What about investors who suffered losses? Neither SEBI nor NSE is in any discussion about compensating these investors from the investor protection fund of the Exchange, since they are not at fault in this case. For a start, investors who have lost money ought to lodge complaints with NSE and SEBI and demand compensation.
It is pertinent to point out that during the Emkay fat-finger trade on 5 October 2012, SEBI had issued a stern warning to NSE for restarting the Exchange without informing the regulator and also for failing to bring all systems to an immediate halt, with the result that several orders, which were inside the system, continued to be matched when it restarted. Apparently, no protocols were prescribed by the regulator, despite that incident.
It is also surprising that neither SEBI nor TAC have questioned NSE over the lack of decision-making, which led to a slow shut down of the Exchange, nearly 1.40 hours after indices stopped getting updated. Will these questions be answered through a slow investigation and adjudication process over the next five years?
Interoperability between Exchanges
NSE insists that ‘interoperability’ between clearing corporations worked. Having said that, it admits that ‘updation of collateral’ is key to making this work, which neither exchange seems to have paid attention to.
At a webinar organised by Moneylife Foundation on 11th March, the failure to update collateral was initially blamed on the high cost of updating broker front-office software. This was publicly denied by Keshav Samant, CEO (chief executive officer) of 63 moons, who clarified that brokers’ systems were upgraded free on the ODIN software, which has an 80% market share, to provide interoperability. It is the choice of the trading member whether to turn it on or not, he said.
While interoperability was mandated more than two years ago, this suggests a shocking failure to ensure last-mile checks at the broker end. Surprisingly, 10 days later, neither NSE nor SEBI has specifically asked brokers to ensure that they can switch positions to another exchange if there is another outage. NSE’s release only says that it is working at addressing updation of collateral as part of strengthening interoperability.
One month later, are we any wiser about whether or not things will be handled differently if there is another trading stoppage in the near future?
This article was first published by Moneylife.
(Sucheta Dalal is an award-winning journalist based in Mumbai. She was conferred the Padma Shri for Journalism. She is the managing editor of Moneylife magazine. The views expressed are personal.)
