Context

Pension Freedoms, introduced in 2015, have afforded consumers greater flexibility in accessing their pension savings, but in doing so also require consumers to make more complex decisions. While Government and regulators have introduced several measures to help Defined Contribution (DC) pension savers navigate these decisions, they and the industry have agreed that there is more they can do to support pension consumers.

The Financial Conduct Authority’s 2022 Financial Lives Survey found that 51% of DC pension customers had low or very low engagement with their pension pots and existing support services. Improving consumer engagement with MoneyHelper – the free government-backed guidance service provided by the Money and Pension Service (MaPS) – is one of the areas in which consumers can be helped to make good decisions about saving, how to invest and how to decumulate.

The study

This study was undertaken by the FCA to understand how carefully-timed and behaviourally-informed messages provided to people at key moments in their lives and pension pathways might increase engagement with MoneyHelper.

Supported by learning from a review of the policy context and research literature, the study involved:

• Three online experiments which each tested varying email subject lines and email content, in a hypothetical environment, each using treatment and control groups in a randomised multi-factorial design.
  • Experiment 1 measured email opens in response to the subject line and clicks in response to a call-to-action in the email content, with a sample of 14,985 participants.
  • Experiment 2 measured comprehension and attitudes towards the subject line and email content, with 3,996 participants.
  • Experiment 3 was a re-rerun of experiment 2, involving a new sample of 3,993 participants, in an attempt to resolve a potential back-fire effect, noted below.
• A qualitative study of 80 participants to inform experiment 3, after it was observed in experiment 2 that there was a possible, negative, ‘back-fire’ effect of the aesthetics used in the treatment groups’ materials.
• A field trial in late 2023 and early 2024 with 82,117 participants from two firms, which tested the optimal subject line and email content from the experiments at potentially critical moments in people’s lives and pension pathways (such as milestone birthdays and pension account log-in), using treatment and control groups on a partially randomised basis.
  • Primary analyses examined the click rate for the email’s call-to-action link to go to the MoneyHelper website using linear regressions.
  • Secondary analyses examined the email open rate using logistic regressions.
  • For each type of analysis, several iterations of regression models were run to check the robustness of the observed results.

Key findings

Message design: The ‘social norms’ email from Experiment 3 (which was designed to overcome mistrust and inertia barriers to engagement) was selected to be the optimal email content used in the field trial. This was complemented by the most effective subject line on average of any version tested in Experiment 1, ‘Future you’ (which was designed to overcome present bias).

Baseline rates: Average open rates ranged from 42.2% to 54.5% across both firms and age groups (under 50s being aged 40-49, and all over 50s). Average click rates ranged from 1.5% to 3.7%. Both open and click rates were higher among individuals over 50 than those under 50.

In tests of the treatment effects of targeted message timing compared with control (i.e. random timing), all of the following reported differences were significant at the 5% level of statistical significance (p<.05).

At milestone birthdays: Targeting the under-50s in the treatment group around their milestone birthdays reduced the likelihood of opening emails by 2.5 percentage points compared with the control group who received messages at random times. There was no effect on open rates among the over 50s, and no effect for click rates.

When approaching retirement: There was a positive effect among the treatment group on open rates when using an approaching-retirement trigger, relative to the control, of 2.7 percentage points. The evidence was less clear in relation to click rates.

When engaging with the pension account: A positive effect of targeted messaging was observed only for account log-in (not at other points such as starting a new pension or increasing contributions). Open rates were, on average, 6.1 percentage points higher for the under 50s, and 6.9 percentage points higher for the over 50s. Click rates were 1.3 and 3.5 percentage points higher, respectively.

New calendar year: An increase in open rates of 5.9 percentage points among the under 50s was attributable to the messaging targeted at the calendar new year, but only in one of the two firms. Otherwise, there were no effects in response to messages triggered by the new calendar year.

Points to consider

Methodological strengths or limitations

• This was a large-scale and rigorous study.
• The authors nonetheless note that they did not explicitly test click rates for the redesigned emails in Experiment 3, only comprehension and attitudes towards the messages, which implies that the email content may not have reflected optimal message design.
• They note that timing treatment effects are likely to be conservative estimates, because there was some contamination (doubling-up) of treatment-group messages (triggered, for example, by milestones) among the control groups.
• The authors also note that sample numbers may have prevented some meaningful differences in the field trial from being observed (i.e. increased risk of false negative results).

Applicability

• This report is Occasional Paper 65 from the Financial Conduct Authority’s (FCA’s) series of occasional papers in financial regulation, which is one of the sources that the FCA may refer to inform its views and discharge its functions.
• The findings should be of interest to all public-interest organisations with a role in pension provision and engagement specifically, and the financial wellbeing of individuals more generally.
• The study highlights the importance of taking a design, test, and redesign approach to field materials to policymakers and researchers.

Relevance

• The study findings are highly relevant in a context in which the framework for defined-contribution pension decision-making has changed for individuals, against a backdrop of previously low participation and engagement.

Generalisability/transferability:

• There is likely to be some transferability of findings from defined contribution pensions to other, more complex, financial products.