Big Data is Disrupting Food, Part 2 – Foodborne Illness Data


Last week I talked about how Big Data is a positive disruptor to drive changes in food production. Today I want to conclude this mini-series by talking about Big Data in food consumption and about foodborne illness data in particular. If you are like any of the 1 in 6 people that get sick from what you eat, you may find this an interesting topic. I’ve definitely had my fair share of it so I wanted to delve a bit more into it. As an open data advocate, I think the more data people have about the food they consume and restaurants they visit, the better. This data and the public response to it affects our food supply.

Most of the officially reported data about foodborne illness is a lag indicator, a way to measure the alleged sickness after it has already happened. The Center for Disease Control and Prevention has several surveillance data sets available in Excel and PDF format online for free. When I started exploring the FoodNet and FOOD Tool data, I found them a bit confusing, incomplete, old, and in formats that make analysis more difficult. The FoodNet has several types of surveys with the most recent ones being from 2009. The FoodNet annual reports range from 1997-2014 and are in my favorite PDF format – a budding data scientist’s nightmare.

The CDC’s FOOD Tool data has a nicer user interface with data visualizations of outbreaks per state, year and month from 1998-2015. It also has the raw data in Excel format that could be analyzed by year, state, type of bacteria, location and type of food carrying the bacteria if known. It’d be great if the CDC had plans to make some of this data publically available in API format but I know very well the limited resources federal agencies face when trying to make these types of open data advances.

Neither of the aforementioned data sets is what would be considered Big Data but more the traditional small data and statistics. I fortunately stumbled upon HealthMap which is using Big Data from social media to supplement these traditional data reporting. From their map it looks like about six state health departments are participating in this project along with the Boston Children’s Hospital. There’s a lot of research in 2016 using the spatial and the real-time nature of social media to predict events and to find out what’s happening. HealthMap looks like one possible vehicle that uses Big Data to be able to predict foodborne illnesses before they happen.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.