Talking Open Data with NYC Stakeholders: A Wrap-Up

Written by Marilla Li, Outreach Manager for The GovLab’s Open Data 500 project.

Open government data – that which is released in open formats for anyone to download, use and redistribute – has tremendous potential to increase government transparency and accountability. Open data gives technologists, researchers and the public an opportunity to generate new insights to improve government service delivery and craft applications to benefit the public good.

New York City has been a pioneer in embracing open data, offering a number of datasets in machine-readable formats to the public and working directly with the community in its efforts to do so.

Last month, the GovLab had the pleasure of hosting a Stakeholders Forum on Open Data run by New York City’s Transparency Working Group – a coalition of civic and civic tech organizations supporting efforts to make NYC government more open and accountable. The event took place in collaboration with the NYC Department of Information Technology and Telecommunications (DoITT), the city agency responsible for providing IT services, infrastructure and telecommunication to New York City.

Some key takeaways from the forum include:

• The Mayor’s Office of Data Analytics (MODA) plans to update the NYC Open Data Plan by July 2014 and spread deadlines until 2018 for all agencies to make their data “open.”
• Stakeholders wish to see more metadata released in order to better understand the data’s original purpose and collection method, i.e. create visible “life cycles” for Open Data.
• Productive dialogue around the “life cycles” of Open Data is currently impeded by major searchability issues with the NYC Open Data Platform and portal.
• Diverse ranges of opinion remain among agencies regarding opening up data, owing mainly to privacy concerns.
• The City remains limited in responding quickly and appropriately to all community requests and feedback regarding Open Data.

The Forum opened with John Kaehny, Co-Chair of the Working Group and Executive Director of Reinvent Albany, introducing some of the key stakeholders present and paving the agenda for a productive session aimed at moving the dialogue forward on how key stakeholders may shape and improve the future of Open Data in NYC. Specifically, Kaehny noted the session’s goals were to:

• Review plans for upcoming benchmarks of the NYC Open Data Law;
• Highlight concerns and brainstorm solutions for data usability; and
• Explore the Socrata NYC Open Data Platform and portal.

Updates to NYC’s open data plan expected by July 2014 – will offer agencies extended deadlines

Nick O’Brien from MODA briefed the group on the status of implementing the NYC Open Data Law, which became official code in March 2012 and prompted release of the first Open Data Plan iteration last September. While the current plan lists data sets to be posted and contains many manually posted sets provided before the capacity to automate became real, O’Brien highlighted that the ultimate goal is one central, automated hub where all Open Data released by agencies can reside.

While over 1100 datasets currently live on the NYC Open Data Platform, O’Brien acknowledged that not just automation, but regulation and access pose ongoing challenges for both data users and providers. O’Brien reminded forum participants that the Mayor’s Office appointed a Chief Open Platform Officer last April and expressed hope that it marks a turning point in ensuring that “enforcement has enough teeth so that agencies would not skirt regulations.”

Finally, O’Brien highlighted some upcoming benchmarks, including an update to the Open Data Plan expected by July of this year. The update will address comments from the open data community and general public and will feature implementation deadlines spread across the next four years to ensure that NYC remains technically capable of managing both a feedback loop and a quality control loop.

Data usability issues on open data portal spur calls for more metadata

Noel Hidalgo from BetaNYC and Dominic Mauro from Reinvent Albany presented to the forum next, addressing some current data usability concerns, including:

• A lack of transparency in displaying the metadata – i.e. the internal “hierarchy” that lets users “see what’s going on inside and see different facets of the data”;
• Ongoing resistance from certain agencies toward opening up its data; and
• Searchability of the Open Data Portal, which still churns out “a lot of noise” in addition to what users actually searched.

The group then brainstormed initial solutions to some of these challenges, including:

• The creation of a feature to show users where datasets are coming from;
• Crafting recommendations to post on the Portal for how to use a given dataset; and
• Continued efforts to work closely with agencies in order to encourage them to open up their data.

Multiple attendees expressed a desire to interact more with the data creators and creation processes to deepen their understanding of the context of the data’s originally intended usage. However, attendees stated that many users’ struggles to navigate the open data portal serve as a major impediment to even starting such interactions.

Open data platform and portal allows for customization, but navigation challenges persist

After this dynamic discussion, Chris Whong from Socrata provided a user walk-through of the NYC Open Data Platform and portal housing the data. He stressed the importance for the portal to enable any level of user to publish and use data and he provided examples of searching for different data pieces in the platform, noting that “someone with no tech skills should be able to view, filter, and interact as one might without any extra software.”

Whong emphasized the most customizable feature on the portal – a custom application programming interface (API) built so that the data can be “sliced a lot of ways” to be more usable and more contextualized to each data user’s needs. Whong acknowledged, however, that even with all the filtered and unfiltered abilities, both the portal and data prove difficult to navigate in its current, fledgling form.

Notably, the forum brought together a variety of open data stakeholders, including representatives from:

• Citizens Union
• League of Women Voters NYC
• New York Public Interest Research Group
• New York Civil Liberties Union
• BetaNYC
• Reinvent Albany
• Manhattan Borough President’s Office
• City Council Committee on Technology
• Council Committee on Government Operations
• Mayor’s Office of Data Analytics
• NYC DoITT
• Socrata

Stay tuned for possible future collaborations between the Transparency Working Group and The GovLab. For updates and to learn more, follow Reinvent Albany on Twitter @Reinvent Albany, and the GovLab @TheGovLab.