How big data is changing financial journalism

Hacks/Hackers London met last night and looked at examples at how big data is changing financial journalism.

The meetup of journalists and technologists, which took place at Bloomberg, heard from speakers from Reuters, Bloomberg, the Financial Times and OpenCorporates.

• Case study 1: Bloomberg

Marianne Bouchart is web producer EMEA for Bloomberg News and is responsible for kick-starting data journalism projects within the organisation.

She demonstrated how journalists at the financial news organisation have a vast resource available by way of the Bloomberg Terminal. This is the Terminal that 310,000 people pay around $20,000-a-year to access as a subscriber to the Bloomberg Professional service (there is more background on the Terminal here).

Journalists use the database as a way of finding links between people and companies when writing text-based stories and can use the data for interactives.

Bouchart gave a couple of examples on interactives, including this interactive of 'The Fed's secret liquidity lines' to banks during the financial crisis - which uses data from 480 csv files.

Another example of an interactive is the 'Bloomberg billionaires index', which allowed readers of the news site to explore the data.

"Bloomberg is committed to taking a position in the world of data visualisation," Bouchart said.

• Case study 2: Reuters

Sam Arnold-Forster and Himanshu Ojha explained how Reuters tells financial stories through data, from flat graphics, to interactives, through to a six-month investigation into income inequality in the US.

Ojha, who described himself as "a journalist who came to data", worked on the in-depth investigation headlined 'The unequal state of America: a Reuters series'.

Ojha said that Reuters wanted to carry out investigative research that no other outlet had done – which was difficult considering the number of news stories and research that have been done on inequality.

Most of the data came from the US census bureau, which Ojha and team analysed on a state-by-state basis.

The data team found stories within the data and then sent reporters out into the field to find the human element of the story, finding the people who were living on food handouts.

A data set alone equals a fun dinner party fact, whereas data and other data equals a storyHimanshu Ojha, data journalist, Reuters

By focusing on data you "sometimes kill the story", Ojha said.

His advice to fellow data journalists was to question the data. "It's important to have a decent editor who questions everything," he said.

He also found "making friends" with academics and researchers in the field of inequality an invaluable source, explaining how he called contacts "all over the world" asking them to look at the findings.

"Send data well before you publish," he advised, and "build in time for fact-checking", which in the case of the Reuters study took three or four weeks.

Ojha also shared another thought for those considering delving into data. "A data set alone equals a fun dinner party fact, whereas data and other data equals a story."

• Case study 3: Financial Times

Emily Cadman and Martin Stabe explained how data stories are investigated and turned into interactives at the Financial Times.

No one has the job title of 'data journalist' as everyone does data, Cadman, who is head of interactives, said.

She explained that a typical team is made up of four people: a reporter, who is an expert in the field, often with 20 years' experience; an interactive journalist; an interactive designer; and an interactive developer.

Projects range from a couple of hours to investigations lasting weeks or months, and much thought is given to what an interactive will add to the storytelling process.

Cadman was very clear in her view that developers should not become journalists and vice versa, but that each person should focus on his or her own expertise.

So without having the mine of data that Bloomberg and Reuters journalists have available, how does the FT find data? Cadman explained that it frequently partners with academics and other sources of data.

Martin Stabe gave examples of how there is often a need for "better data". A binary measure such as the number of children receiving free school meals demands "better variables", Stabe said.

The FT has had success in getting this better data by asking the source of that data, often government departments.

Another way the FT accesses data is through scraping. Stabe gave an example of how they researched Cayman Islands directors with "an opaque industry".

Stabe explained how by scraping "filings" and linking to other data, the team could identify the people who are directors of hundreds of hedge funds.

So where is the Financial Times heading with its interactives? Stabe explained that one of the changes will be in "a shift from 'interactive' to 'news apps'", a term favoured by many in the US including National Public Radio and the Chicago Tribune. The difference between the two is that news apps involve more page views by the reader as they explore the story.

Stabe also said there would be increased integration into the FT's mobile offerings (there is more background on mobile and the FT here).

There will also be a "greater focus on tools for reporters", and a move to "automate production of routine graphics". That way the team can spend more time "on the really cool stuff".

Stabe also shared a thought (he attributes to journalism professor Steve Doig) that "at it's best data journalism is social science on a deadline" in that you take a hypothesis which you seek to prove or disprove.

• Case study 4: OpenCorporates

After hearing from three financial news outlets, Chris Taggart spoke about OpenCorporates, which launched in 2010.

Taggart said that OpenCorporates is the world's largest open database of companies, explaining that there is an aim to get data on every company and "legal entity" in the world.

Around 50 million are listed so far – and it is a great resource for journalists, providing a page on each company or entity and powerful search options.

In addition to company data, OpenCorporates adds data from a wide range of sources, linking that to the company concerned. The database pulls in notices such as health and safety violations.

Taggart explained that they get data from sources such as government releases and Companies House, as well as screen scraping sites.

Taggart also reinforced his view of the importance of open, suggesting that a lack of open data in the lead up to the financial crisis had a negative impact.

• There is more detail in this liveblog here, written by Henry Taylor and George Arnett, journalism students at City University.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).