Wall Street watchdog moves to cloud, big data, to boost capabilities

[June 21, 2014]  By Herbert Lash
 
 NEW YORK (Reuters) - A Wall Street regulator has begun to migrate its surveillance and other watchdog duties to cloud computing, which in combination with crunching big data, will dramatically boost the organization's capabilities, two officials at the agency said.

The Financial Industry Regulatory Authority began using cloud computing at the beginning of this year in a roughly 30-month roll-out that will save FINRA $10 million to $20 million annually while making the agency operationally far more agile.

By moving to the cloud, FINRA will gain increased processing capacity and more space to store data, while reducing costs because the service is only used when needed, Steven Randich, chief information officer at the agency, said in an interview.

Technology is key to FINRA's mission to oversee brokerage firms, monitor the U.S. stock market and protect investors from potential fraud. Wall Street's industry-funded watchdog monitors trading in almost 6 billion shares daily for abusive activity.

In the past, increasing capacity and storage cost more money. FINRA, which processes 25 billion "market events" a day, had hit the limits of what is commercially available, Randich said.

A market event can be a quote or a trade, or anything related to an order from its creation to its clearing.

"We could have redesigned our system to scale across a lot of big machines, but economically that's not really on the table for us," he said. "By moving to the cloud we get dramatic processing and storage scale at commodity prices."

A major benefit has been speed. Some of the more complex queries FINRA runs on its data could take several hours in extreme cases, but they are now are done in a few seconds, Randich said.

"It's measurably, measurably faster," said Thomas Gira, who is in charge of market regulation at FINRA. "It might take us to another level in the sense of timeliness."

Gira said the new technology will allow FINRA to run surveillance patterns more quickly and to do more data analysis, although he acknowledged behavior in the electronic marketplace is constantly changing, requiring the authority to continually upgrade its own capabilities.

The early migration has focused on FINRA's order audit trail system, or OATS, which accounts for a large part of processing all of a day's market-generated data.

The hundreds of surveillance patterns FINRA runs need to be written on the new platform, then tested against the old code to ensure the results are the same. Once migrated, they will run faster, Gira said, noting the results so far have been "pretty dramatic."

(Reporting by Herbert Lash; Editing by Dan Grebler)

[© 2014 Thomson Reuters. All rights reserved.]

Copyright 2014 Reuters. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.