Description of the programme

Participants recruited from a YouGov panel of Royal London customers were asked to test one of three mobile phone budgeting apps – MoneyWise (from Haringey Citizens Advice Bureau), Spending Tracker or Toshl – or complete a generic, simple pen and paper budgeting method for three months. The trackers are all available as free smartphone apps (although at the time of writing, Toshl is discontinued). Overall user numbers of the apps are not given. However, 797 customers entered into the project (by completing a first questionnaire and downloading an app) from across the UK between July and November 2016. The intervention was undertaken in participants’ own time as part of their daily lives.

The study

The study, which was commissioned by Royal London aimed to explore whether using a budgeting tool could help people manage their everyday money better. It was conducted between July and November 2016 and included quantitative and qualitative elements. Participants were assigned to test one mobile phone budgeting app or a generic, simple pen and paper budgeting method and asked to keep a record of all their spending using that method for three months. Of the 797 people from across the UK who started the trial, only 278 completed it (by completing a questionnaire on their budgeting habits and how they manage their finances at the beginning and end of the study) and used the budgeting method assigned to them. Qualitative feedback was received via anonymous online forums from 49 participants on their prior money management methods and the budgeting tools trialled in the quantitative study plus one other (Santander Spendlytics), and 10 case studies selected from these to represent the different budgeting methods (qualitative results are not summarised here).

A year after the initial evaluation took place, YouGov conducted a follow up study with 37 people who had taken part in the experiment and had used either one of the apps or pen and paper for the full duration of the three-month trial. A three-day online forum was held to establish what impact the trial had made a year after they took part.

What are the outcomes?

• Keeping track of spending
• Planning spending and keeping to a budget
• Keeping up with bills
• Motivation to save

Key findings

• 49% reported that using their assigned budgeting method was helpful in monitoring what they spent.
• 34% reported that using the budgeting method improved their ability to keep track of spending.
• 28% said using the budgeting method improved their ability to create a budget.

• 23% said using the budgeting method helped them stick to a budget.
• The number of participants who reported having a clear idea about how to create weekly or monthly budget increased from 61% pre-trial to 67% post-trial.

• 37% reported that since using the budgeting method they had better understanding of their income and expenditure.
• The number of people who said they hardly ever or never run out of money increased from 71% pre-trial to 74% post-trial.
• The number of people that could pay an unexpected bill of £300 increased from 37% pre-trial to 41% post-trial.

• 26% reported that using the budgeting method made them more likely to discuss household finances with their partner/family/household.

• 26% reported that using the budgeting method made them more motivated to save for unexpected expenses.
• 51% reported that budgeting had a positive impact on how they thought about their long-term finances.

, the pen and paper method scored more highly on a number of measures than the apps. However, pen and paper users were self-selecting and tended to be older so may have had more time to engage with the task:

• 74% of pen-and-paper method users said it was good at encouraging them to interact with their finances, compared with 53% of app users overall.
• 69% said it was good in terms of overall experience, compared with 47%.
• 75% said it was good for ease of use, compared with 56%.

• Only a small number of those involved in the follow up study were still using the same budgeting tool as in 2016. Some participants were using 'modified' versions of the tools, particularly with regard to the pen & paper method and some participants had taken elements they liked from the app to design their own system or tweak the system they were using previously
• Of those who stopped using the app, many did so because they preferred their own system of budgeting, some felt the tool lacked certain beneficial features and others struggled with the practicality of the app
• Most participant do now have some sort of budgeting system in place, whether new or pre-existing
• 14 of 37 felt that participating in the trial made an impact on how they plan, organise and manage their money, 14 felt that it did not and 9 were unsure
• 16 of 37 felt that taking part in the trial made a difference to how in control they feel about their financial situation, 17 said it did not and 4 were unsure
• 18 of 37 felt the trial had a lasting impact on them, reporting feeling more focused on budgeting and more motivated to do so as well as being more aware of their spending. 13 were not sure of the impact and 6 felt there had been no lasting impact.

Points to consider

• High levels of sample attrition and initial self-selection into the study could serve to bias the achieved sample to those who enjoyed and benefited from the app, and who may differ systematically from all customers who could potentially benefit from budgeting help. As such, the results may tend to overstate the positive impact of the interventions.
• All outcomes are based on self-report measures, and many are retrospective only (despite the pre/post- data collection study design) which may also tend to overstate the impact of the interventions.

• Although the resulting sample is self-selecting and non-random, there has nonetheless been no attempt to estimate the statistical significance of change or differences attributed to the interventions.
• The findings should be treated as indicative of the sample only and not the population from which they are drawn.

• The findings are highly relevant at a time when interventions intended to improve budgeting are common (in the UK and elsewhere).

Full report

Looking after the pennies - full report

Follow-up report