WHAT'S NEW?
  • Great news for journalists from Nepal: Our Nepali edition is online! READ MORE
  • Are you missing some topics in this manual? Send us an email! READ MORE
  • Apply now for a fellowship of the Asian Investigative Journalism Conference READ MORE

Data-mining is arguably the most objective process to help you arriving evidence. Think about which lead is more likely to put you on the right track: a complaint from one hospital patient about thieving nurses or a database from the Health Ministry on disciplinary hearings and dismissals as a result of complaints about theft over the last five years? As with all information, you should always be mindful that even statistics can be manipulated and used to misinform, but efficient 'mining' of databases has exposed immensely important stories over the last decade.

International data can provide even more relevant results. For example, development aid donors sometimes publish reports on how they spent their money in any given year. By collecting such data from donors that are active in your country and analysing them, you can tell stories with headlines like ‘Donors (to our country) spent most aid money training our civil servants’.

Database mining does not always have to be about finances, either. Social network analyses have produced stories on terrorist networks, political party supporters, and the most influential and richest people in particular communities. These networks can be members of a certain profession, a geographic community or prominent people in a political party. You can combine data on how much they earn, who they work with and meet that paints a social network picture, which tells you something about their influence in society.

During the Summer School of the Centre for Investigative Journalism Jonathan Stoneman hold a session about dealing with Big Datasets. Watch the entire lesson here:

It is important to compile all necessary data yourself using the databases of journalists' and other organisations. In their databases, journalists or organisations store information, sometimes tagged by topic. These databases can contain articles, researches, studies and also contacts. Those data can then be used to obtain more background research. In the U.S. and Europe, investigative journalists have established centres that produce databases for mining that journalists around the world can use.

Watch this video with data-mining expert Giannina Segnini, Director of the Master of Science Data Journalism Program at the Columbia Journalism School, giving her tips on how to investigate with data.