Context

The Financial Foundations national goal of the UK Strategy for Financial Wellbeing aims for two million more children aged 5 to 17 to receive a meaningful financial education by 2030. The Money and Pensions Service already has financial literacy measures for children aged 7 and over. However, there is a gap for those aged under 7, which is important because children’s money habits and attitudes towards money start to form before the age of 7.

Arlo's Adventures is an innovative measure of financial literacy which uses a comic strip and interview script to engage 4-to-6-year-old children and assess their knowledge and experience of money. Following the design and successful small-scale, pilot testing of Arlo’s Adventures by Loughborough University for MaPS in 2022 (see Arlo design and small-scale test), the current study was commissioned by MaPS for larger-scale quantitative testing of the measure.

The study

This report describes the results of research undertaken by a team led by Loughborough University to test the validity of the Money and Pension Service’s measure of financial literacy of 4 to 6-year-old children called Arlo’s Adventures. The aims of this research were two-fold:

• to assess the feasibility of employing the Arlo’s Adventures measure at scale by testing different interview formats (in-person, video call and parent-led) in diverse populations of children
• to establish the validity of Arlo’s Adventures as a measure of young children’s financial literacy, by:
  • assessing inter-rater reliability in the scoring of children’s answers
  • running exploratory and confirmatory factor analyses to understand the underlying constructs captured
  • correlating the measure with other factors such as children's numeracy skills, demographic characteristics, and parental attitudes and behaviours.

The study recruited a sample of 382 participants aged four to six from primary schools in England, Wales, Scotland and Northern Ireland between May and December 2023. Over-recruitment was undertaken in the sampling design to ensure sufficient sample numbers in important subgroups, in order to support the aim of testing the feasibility of using the financial literacy measure at scale in diverse populations.

Key findings

Key findings relate to the feasibility of using Arlo’s Adventures at scale and the validity of Arlo’s Adventures as a financial literacy measure

Feasibility of using the measure at scale

• One-to-one in-person interviewing within primary schools was determined to be the most feasible route to use of the measure at scale.
• Other approaches raised risks around recruitment and validity.

Validity as a measure of young children’s financial literacy

• Inter-rater reliability was found to be good (Intraclass Correlation Coefficient (ICC) was 0.959 [95 CI 0.852-0.989]).
• Exploratory and confirmatory factor analysis revealed a five-factor structure for the Arlo’s Adventures measure. This had a very good fit to the model (CFI = 0.94, TLI = 0.93, RMSEA = 0.053 [90% CI = 0.046, 0.061], SRMR = 0.044).
  • The five constructs describing the factors were: ‘Transaction methods’, ‘Getting money and making money decisions’, ‘Saving money’, ‘Where money comes from’ and ‘Spending money’.
  • These mapped well onto other evidence relating to curriculum design for financial education and children’s cognitive developmental in relation to money.
• Variation in children’s numeracy skills accounted for 31% of the variance in the Arlo’s Adventures measure.
  • There were, however, limited or no relationships between children’s levels of financial literacy measured using Arlo’s Adventure and school-area deprivation, parental attitudes and behaviours.
  • There were small but significant differences in children’s financial literacy by gender and having English as a first language.

Points to consider

Methodological strengths or limitations

• The methods of testing for inter-rater reliability and the analytical methods adopted in the study were rigorous and robust.
• The authors note that data for parental responses were only available for a subset of 110 participants.
• The finding that there were no or only limited correlations between financial literacy levels and some of the target correlates potentially undermines the validity of the Arlo’s Adventures measure, if the theoretical assumptions for correlation are correct. In relation to the parental measures, the authors suggest that the small sample and data quality most likely explains the unexpected findings.
• It is not clear from the report if the finding that financial literacy levels were higher among children who spoke English as their first language than those who did not was meaningful in relation to their financial literacy or an artefact of poor comprehension of the comic strip.
• The study did not examine the test-retest reliability of the financial literacy measure or its sensitivity to change, and these may be areas for future research.

Applicability

• This report should be of interest to financial education providers and other organisation working with young children who wish to measure the financial literacy of their clients or beneficiaries and would consider using Arlo’s Adventures or a similar tool to do so.
• It may also be of interest to researchers and evaluators who wish to develop similar tools and can benefit from the learning from this report and the earlier report on design and small-scale testing of the measure.

Relevance

• This study is relevant given the Money and Pensions Service commitment to the Financial Foundations national goal set out in it UK Strategy for Financial wellbeing by helping to improve the measurement of knowledge and experience of money among the under 7s.

Generalisability/transferability

• Lessons learnt from this study about the challenges of testing financial literacy measures for young children are likely to generalise to other financial literacy measures for young children and, to a great extent, other age groups and similar markets.
• In similar markets, the Arlo’s Adventures measure itself should also transfer well, albeit with appropriate modification to reflect the particular financial and social contexts of those countries.