New York’s analytics chief on unlocking the power of open data

New York City now has 4.6 billion rows of data available on its open data platform, with more being added all the time.

This is thanks to one of the strongest open data laws in the US that makes most data open by default, says Martha Norrick, Chief Analytics Officer for the City of New York.

Upcoming additions will include a dataset from the Department of Health on urban apiaries where people keep bees and another from the Department of Environmental Protection on sludge produced at wastewater treatment plants.

Norrick told Cities Today: “We work with agencies to review any data that they’ve published on their website each year and any data that they’ve released to the public via Freedom of Information Act requests.

“Generally speaking, if the government is producing a dataset, the public is entitled to see it and we work very closely with agencies to make sure that all that data ends up on [the open data portal].”

Over 11 years since New York’s Open Data Law passed, the city’s work is evolving.

“I think the focus is really shifting from getting as much data as possible out there to also include making sure that the data we’re publishing is high quality, well documented and accessible to people,” says Norrick. “We’re really focused on improving the user experience with open data.”

The Mayor’s Management Report, for instance, used to include just the number of datasets that were published but now incorporates the usership of open data and the quantity of data involved.

The city has a six-person open data team in the Office of Technology and Innovation as well as 100 data co-ordinators within agency departments and a growing cohort of open data ambassadors, who are members of the public that volunteer their time to support others in using open data.

Public requests are managed through an open data helpdesk, which recently fielded its 10,000th enquiry since 2017. People can request a dataset, provide feedback on data or ask a question.

Human experiences

Historically, some government open data efforts have failed to live up to their potential but Norrick says that in New York, “the juice is worth the squeeze”.

The open data platform has over a million unique users annually.

“I think we’re lucky to have such a passionate group of New Yorkers that really deeply care about the city that we all live in together,” Norrick comments.

“[Open data] is an important tool for our citizens to hold government accountable. It’s an important way for people to understand what’s really happening in the city.”

Norrick recalls that one of her favourite innovative applications of open data was an article in the New York Times that analysed dog registrations as a way of understanding gentrification in the city.

Open Data Week is another way the city aims to engage people. The next edition in March 2024 will be the eighth so far and Norrick says the occasion attracts “thousands of people” from New York, across the country and even abroad. It includes public events such as exhibitions, conferences and workshops.

Last year, an artist used open data and an undulating fabric-based installation to visualise sea-level rise.

“Those artistic pieces can really tell a story,” says Norrick.

She adds: “One of the things that I think sometimes gets lost about data is that it really records human experiences.”

New York’s latest Open Data Progress report also highlights tools and services created using open data, including Spatial Equity NYC, which enables users to visualise inequalities in public space, public health, mobility, and the environment across Community Board and City Council districts.

Revel, an electric mobility company, uses New York’s open data for its day-to-day operations and strategic decision-making. One application is automatically detecting when users drive where mopeds aren’t allowed or drive the wrong direction down one-way streets. Another is modelling taxi and rideshare usage under different conditions.

“I’m constantly amazed by what people use open data for,” Norrick comments. “People are so creative. One of the best parts of my job is getting to see what people do with this incredible resource.”