April 3, 2012 brian

The Possibility of Big Data

I’m getting more interested these days in big data, and, naturally, its potential applications in economic development, workforce development, and regional planning. Companies like Infochimps in Austin are launching really interesting products geared toward big data management in the private sector. The public sector is jumping on board as well, with interesting projects such as the Community Health Data Initiative at HHS. While some local governments, such as San Francisco and New York City, seem to be serious about open data, I haven’t seen much discussion yet in economic and workforce development circles about how big data or open data can improve outcomes in our field.

This is a conversation we need to start having. The possibilities seem endless. Imagine a location-based service on your phone, for example, that would enable you to find local businesses nearby that pay all of their employees a living wage. Despite reading nearly every study I can find on the impacts of living wage ordinances on local economies, I still have mixed feelings about them. In Texas, I think it’s a non-starter, even in places like Austin. So why not do the next best thing and empower consumers with more information and let the market do the rest? You’d need a compelling value proposition to overcome the hesitation I imagine most businesses would feel releasing that sort of information to a third-party in a way that could be validated, but I think this would be a worthy experiment with minimal startup cost. Progressive-minded consumers in cities like Austin wield a convincing amount of discretionary income, so hopefully data collection would get easier over time as the program demonstrated return on investment.

I imagine there are plenty of applications in the workforce development world, too. For example, every state collects data on educational attainment, professional certifications, and occupational experience from unemployment insurance claimants when they sign up for benefits. In most cases, that valuable information just sits in state databases, or perhaps gets rolled up to federal agencies to generate aggregate national statistics. Why not investigate the feasibility of posting that data online in machine-readable format in a way that eliminates the possibility of violating confidentiality, such as what HHS is now doing at the federal level? I’d bet there is an entrepreneur somewhere out there who could figure out a way to combine that data with online job postings to get unemployed people back to work faster than most government-run programs. And I’m sure there are countless other applications that, as practitioners involved in the day-to-day management of economic and workforce development programs, we can’t even imagine.

Which is really the point. I had the good fortune of sitting in on a presentation by Todd Park, the newly appointed U.S. CTO, last year when he was still the CTO at HHS. Todd was responsible for open data projects at HHS and was talking to his counterparts in other agencies about how to engage in open government. This is paraphrasing, but his advice was something along the lines of: “You are civil servants with the responsibility of protecting the public’s interest. You are not big data experts, and you are not entrepreneurs. Just post the data, do your best to protect confidentiality, and then let the market go to work. You’ll be amazed at what people will come up with that will help your agency achieve its goals of public health, environmental quality, or whatever they may be.”

Sign me up.

Tagged:

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>