Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

Predicting rent arrears: Leveraging data science in the public sector

Jonathan Leslie (Pivigo), Tom Harrison (Hackney Council), Maryam Qurashi (Pivigo)
16:3517:15 Thursday, 24 May 2018
Data science and machine learning, Emerging technologies and case studies
Location: Capital Suite 10/11 Level: Beginner
Secondary topics:  Financial Services
Average rating: *****
(5.00, 5 ratings)

Who is this presentation for?

  • Business managers, managers in the public sector, and data scientists

What you'll learn

  • Explore a recent project in which a team of data scientist trainees helped Hackney Council devise a more efficient, targeted strategy to detect and prioritize tenants at risk for falling into arrears


Social housing is provided by local councils and housing associations with the goal of providing secure, low-rent housing options to those most in need. Occasionally tenants experience difficulties in paying their rent on time and fall into arrears. This situation hurts the tenant and council alike: falling behind on rent payments can be stressful to the tenant, and the lost revenue can put councils under substantial financial burden. Following the forthcoming rollout of Universal Credit by UK central government, the number of tenants falling into arrears is likely to increase substantially.

Hackney council, like many other local authorities in the UK, has struggled with lost revenue from rent arrears. Of its 20,000 tenants receiving social housing, a third are in arrears, with further money owed by former tenants. Currently, all management of arrears cases is retrospective. However, this system is stretched by limited resources and is not able to discriminate between those who are likely to pay back their debt quickly and those who are more at risk of entering a state of long-term arrears, which accounts for the majority of arrears debt. If a more intelligent system could be implemented to differentiate the high-risk cases from relatively benign, temporary ones, the council could make better use of its resources and be less vulnerable to persistent lost revenue.

Jonathan Leslie, Maryam Qurashi, and Tom Harrison discuss a recent five-week data science project involving a partnership between Pivigo and Hackney Council in which a team of data scientists-in-training devised an innovative way to help Hackney Council predict those tenants who are most at risk for going into long-term arrears. The team identified that no single factor had strong predictive power. Instead, they observed that subtle combinations of factors were associated with risk and that these combinations could vary considerably. Using this insight, the team produced a predictive model that calculated the risk of arrears for each tenant. They then extended this model to predict an arrears trajectory for each, differentiating between short-term and long-term arrears risk. This, then, allows the council to determine which of the arrears cases pose the greatest risk and should, therefore, be prioritized for targeted intervention.

This work underscores the power of data science and how even a small team of data scientists can bring real value to an organization in a short period of time. This can be especially significant in the public sector, where limited budgets often mean that the council has to make difficult decisions about how best to use the resources they have. By providing the tools to help officers make those decisions in a more informed way, data science can help the council increase efficiency, decrease lost revenue, and, in this case, provide more assistance to tenants before their debt becomes unmanageable.

Photo of Jonathan Leslie

Jonathan Leslie


Jonathan Leslie is the head of data science at Pivigo, where he works with business partners to develop data science solutions that make the most of their data, including in-depth analysis of existing data and predictive analytics for future business needs. He also programs, mentors, and manages teams of data scientists on projects in a wide variety of business domains.

Photo of Tom Harrison

Tom Harrison

Hackney Council

Tom Harrison is digital transformation manager for housing services, neighborhoods, and the housing directorate at Hackney Council.

Photo of Maryam Qurashi

Maryam Qurashi


Maryam Qurashi is a data scientist at Pivigo, where she works with data scientists and organizations who are seeking to become more data driven. This means dealing with both the realities and possibilities of how to design and scope a data science project. She’s very interested in and curious about questions of ethics and moral philosophy as applied to data science and technology. Maryam became involved in the data science community in London as an S2DS fellow, following a career in academia as a microscopy image analyst.