What happened?
In 2022 a data scientist joined the housing department of a London borough council. The team was tasked with managing temporary accommodation, supporting homeless applicants and prosecuting rogue landlords. Within months the analyst discovered a tangle of broken databases, duplicated records and a chronic lack of oversight. Tenants were recorded as having impossible ages, the number of children in a household varied wildly, and crucial data on rent payments and employment status was missing or contradictory.
Efforts to clean the data required staff to manually correct individual entries, a process that could not keep pace with the volume of new errors. The council spent £2.8 billion on temporary accommodation across England in the last financial year, yet could not produce reliable figures on how many tenants it actually housed or how many were in work.
"This is one of the countless times my working life has been made worse by decisions about databases which I had no involvement in," a senior analyst said.
Procurement decisions were made by managers who did not understand the difference between a database and a website, and no one could identify who owned the various IT systems. Even the contract for the homelessness tracking software was up for renewal without anyone in the department aware of it.
Why does it matter?
The chaos described is not unique to one borough; it mirrors problems reported by hundreds of local authorities across the country. This matters because the UK government has just announced that mayoral authorities will gain new powers to call in planning applications and manage public assets. To exercise those powers effectively, mayors will rely heavily on the data and expertise of local councils.
If councils cannot provide accurate, timely information, the devolution agenda championed by Andy Burnham risks being undermined. Mis-reported housing benefit figures, unknown tenant numbers and missing employment data make it impossible to assess the impact of policies or allocate resources efficiently. Moreover, the lack of a unified data strategy hampers the ability to track outcomes, a key requirement for any accountable public service.
What happens next?
Experts suggest that the only realistic solution is to build new, interoperable systems rather than trying to patch the existing ones. This would involve establishing common identifiers across housing, benefits and employment databases, and ensuring that data owners are clearly defined and accountable.
In the short term, the government could mandate a data audit for councils that receive new mayoral powers, setting minimum standards for data quality and system integration. Training programmes focused on data engineering and analytics would need to be funded, addressing the current skill gap where senior roles are filled by individuals with limited technical expertise.
Long-term, the success of the devolution reforms may depend on creating separate, well-resourced data hubs that operate independently of the traditional council bureaucracy. By allowing legacy systems to "wither on the vine," as the former analyst put it, new structures could deliver the joined-up services that the reforms promise.
Until such reforms are implemented, the risk remains that mayoral authorities will inherit the same fragmented information landscape, limiting their ability to make evidence-based decisions for the communities they serve.