Why London is pivoting its data approach

London’s plan to overhaul its Datastore demonstrates a marked shift away from the early days of smart city open data. Sarah Wray talks to Chief Digital Officer Theo Blackwell about the new direction.

London is replacing its Datastore open data platform with a new Data for London hub that will act as a ‘central library catalogue’ for data held by both the public and private sector.

The new infrastructure and name represent a fresh approach in London, reflecting changes in both technology and broader thinking.

Rather than ingesting ever-more data and hoping someone finds it useful, the focus with Data for London is on sourcing data based on needs and linking to where it is held.

“The Datastore outgrew its original functionality,” Theo Blackwell, London’s Chief Digital Officer, told Cities Today, noting that there were an increasing number of requests from people to develop services that would also include the sharing of non-open data.

“If you want to plot where you’re going to build schools in London, you need open data and private data from the Department for Education on who the pupils are and where they live in order to create a heatmap,” explains Blackwell.

“Open data couldn’t solve the whole problem of the kinds of things that were really useful for the city.”

Alongside open data, London’s High Streets Data Service incorporates data from partners, including aggregated and anonymised spend data from Mastercard and footfall data from O2.

In addition, as London tackles issues such as air quality, it needs to be able to use various types of sensor and real-time data, which the Datastore wasn’t built for when it launched in 2010.

Human element

Rather than splashing out on an off-the-shelf platform with all the latest whistles and bells, the city has allocated £500,000 (US$605,000) for the cloud-based infrastructure which will be developed by the Greater London Authority with a technical partner. This is a departure from the software-as-a-service Datastore.

“With software-as-a-service you have to hope the roadmap of the company you’re buying from chimes with the roadmap that you want to see,” says Blackwell.

“Our objective now is to build something with a light core, and then the future functionality as dictated by the users would be built in a modular way. That could be buying best in class or building something ourselves – we’ve got the option to be a bit more mixed economy on the build.”

The initiative also means a much bigger push on engagement around data through events such as ‘unconferences’ and design sessions, but more importantly through integrating data much more into wider discussions about net zero, congestion and other priorities.

“I think everybody who’s really experimented with open data has realised that you can’t just have passive platforms where you publish and great things will happen,” says Blackwell.

“That only happens to a limited extent; you need to invest a lot in curation.”

As part of moving from passive to active, Data for London will include data governance resources such as templates for data-sharing agreements. It will also share interpretation and analysis, models and re-usable code to make the data as useful as possible to more public servants, researchers, businesses and citizens.

A new Data for London governance board will provide guidance on ensuring data is as useful as possible, and that data sharing is legal, ethical, and secure.

In the early days of open data, there was little insight into who the users were. Data for London will give owners more control.

“This is quite a journey away from what was conjured up in people’s minds at the beginning of the smart city revolution, which was to create a place that would effectively harvest data,” says Blackwell.

“The pivot is also in saying that the role of the human in this is really important, because they’re setting out the data questions that need to be answered, the methodology on how a data service should be created, bringing people together, and we’re reducing the friction on all of the governance agreements so that data can be shared effectively, but responsibly.”

Cities to regions

More cities could move in a similar direction as they form regional collaborations to address key challenges that cross municipal borders. The approach could also help smaller cities and towns that may need bigger datasets to draw meaningful insights related to issues such as tackling mental and physical health, social problems and more.

Blackwell comments: “If you’re talking about circular economy, net-zero or reducing congestion, the same challenge applies: how do you link data that’s held in different administrative silos and bring it together and get the most value from it?

“We think that this is applicable and is not anywhere near the back-breaking cost of some of the smart city platforms that cost tens of millions.”

The investment, he says, will be in building a bigger team and the cost of data projects themselves.

“We are much more focused on the purpose of that data rather than the fact that we’ve got lots of it.”