How London is using data matching to tackle homelessness

London is using lessons from a pan-city homelessness data project to shape its next three-year innovation strategy, as it seeks to move collaborative technology projects from experimentation into wider deployment.

The Strategic Insights Tool, developed from 2023 and subsequently rolled out across the UK capital, connects information from councils, outreach teams and homelessness services. It now draws on records from 45 organisations to help decision makers understand how people move between the streets, housing services and accommodation.

The initiative reflects a wider city innovation strategy currently being developed by the Greater London Authority (GLA). The yet to be published strategy includes four elements that cover digital infrastructure, collaboration, artificial intelligence, and trust and inclusion.

“The first part of that stack is infrastructure, and that has three components,” Theo Blackwell, Chief Digital Officer, Greater London Authority, tells Cities Today. “The first is improving the city’s data infrastructure through the Data for London programme, so how the city joins up and connects data across 32 boroughs and major institutions like Transport for London.”

The strategy also includes developing the London Office of Technology and Innovation (LOTI), which brings together London’s boroughs, into more of a research and development vehicle for innovation at scale. Other priorities include connectivity, data centres and green innovation, predictive analytics, generative and agentic AI, autonomous vehicles and drones.

The Rough Sleeping Insights Project demonstrates how this collaborative approach can be applied to a complex public service. LOTI led the work with the GLA, London Councils, Bloomberg Associates, Faculty AI, boroughs and homelessness service providers.

The project began with a problem that technology alone could not solve. Information about people sleeping rough was distributed among numerous organisations and systems, preventing public services from seeing an individual’s journey as a whole.

“Homelessness doesn’t respect administrative boundaries and we didn’t have a big-picture view of people’s pathways through the rough sleeping system,” Blackwell says. “Some of the data was held by our outreach workers, some was held by the boroughs themselves, and a lot was held by various voluntary and community organisations. A lot of this information was also quite personally sensitive.”

The tool draws on three main sources. The Combined Homelessness and Information Network (CHAIN) records contacts between outreach workers and people seen sleeping rough; In-Form is a case-management system used by accommodation and hostel providers; and the Homelessness Case Level Information Collection (H-CLIC) holds council data on statutory homelessness applications and interventions.

Someone might appear in several systems without those records being routinely connected. This limited the ability of decision makers to understand movements between the streets, council housing services, hostels and other accommodation. It could also require people to repeat their circumstances whenever they encountered a new service.

The project team began with a seven-week discovery phase. It mapped the rough sleeping ecosystem, conducted interviews and workshops with prospective users, defined their needs and examined samples and structures from the available datasets.

“Working with Faculty AI, the private sector and the London boroughs, we brought people together using design thinking and LOTI’s outcome-based methodologies,” he says. “That involved a lot of problem identification and iteration with experienced professionals, many of whom are non-digital and non-data people.”

Building the first version

A minimum viable product was developed over six weeks in 2023. LOTI and Faculty AI initially worked with Camden, Hillingdon, Lambeth and Westminster, alongside four homelessness service providers.

The first version concentrated on a limited set of functions that could be tested intensively before wider investment. Users received online tutorials, a written guide and a recorded demonstration. Their feedback, together with technical problems identified during testing, informed subsequent development.

The tool ingests information through application programming interfaces or manual uploads. Because participating organisations structure their information differently, the records are cleaned and mapped into a common data model.

The project team selected Splink, an open-source data-linking package developed by the Ministry of Justice, after testing several approaches. Its probabilistic matching model considers attributes including similar but not identical names, dates of birth, telephone numbers and National Insurance numbers.

Records are connected only when the model calculates at least an 85 percent probability that they relate to the same person. LOTI reports that it identifies around 91 percent of genuine matches, although performance varies according to the quality of the source data.

The system is intended to identify patterns across groups rather than support decisions about individual cases. Matched journeys are aggregated and displayed through visualisations that authorised users can filter to examine different experiences.

“We’ve created a system that can see what is happening to people throughout their rough sleeping experience,” Blackwell says. “It gave us a London-wide view for the first time, gave boroughs a view outside their own borough for the first time and allowed experienced professionals to see the big picture.”

He says the evidence has also allowed practitioners to challenge assumptions formed when decisions had to be made without joined-up information.

Building trust around sensitive data

Following the pilot, a second phase extended the tool across London and brought in additional service providers. This required stakeholder briefings, the onboarding of further datasets, new user access and a review of information governance arrangements.

The most demanding part was not choosing the technology but obtaining approval to share sensitive personal information across councils, charities and service providers. Information governance was therefore treated as a dedicated workstream from the beginning.

LOTI developed guidance covering lawful bases for processing and sharing information and the privacy information organisations should provide to individuals. Documentation created for the nine pilot organisations was reviewed before the project expanded to 42 organisations during the second phase.

Testing also showed that people could sometimes be identifiable when filters reduced a visualised group to a very small number. The partners agreed to suppress information relating to groups of five people or fewer.

“If you were a member of the public, you might assume that the state, because it collects all this data, already has this big picture,” Blackwell says. “But it’s yet another example of how data is often kept in silos and deemed too difficult or too sensitive to remove from those silos. We’ve had a really strong proof point that wrangling this is possible.”

Moving towards prevention

The insights tool is designed to help London measure progress towards making rough sleeping rare, brief and non-recurrent. It can also compare interventions, inform service commissioning and reveal gaps in processes or data collection.

LOTI is now examining how to add further datasets and extend the work from rough sleeping to homelessness in its wider forms. The work demonstrates the need to start with a defined service problem and get early participation from practitioners with agreement on how information can be shared safely.

The project also demonstrates that cities do not necessarily need the latest generative AI to produce useful results. LOTI and Faculty AI used established probabilistic data-matching methods showing that proven technology can deliver practical results without adding unnecessary complexity.

These lessons will be shared when London hosts the City Innovation Network Leadership Forum on 7–8 October. Senior city officials will discuss how governments can progress from individual trials to deploying technology across public services.

“The big challenge is how we equip our city leaders with more knowledge and confidence around technology,” Blackwell says. “How we create proper AI functions in city government that can honestly look at deployment at an enterprise level, rather than just an experimentation level, is really important.”

Main image: Smutkoalex | Dreamstime.com