Context

There is a discussion within certain social sciences about how in order to properly measure the impact of a particular programme or intervention, first it is required to have a measurement tool or method of evaluation that is valid and reliable. Using this as a starting point, researchers in North America have provided details of how to apply measures of validity and reliability to tools used in measuring financial wellbeing and distress.

The study

The purpose of this article is to provide details of the process used to establish validity and reliability for a recently developed instrument measuring financial distress/financial well-being. Developers used a number of different methods and tests to provide evidence that the InCharge Financial Distress/Financial Well-Being (IFDFW) Scale, an 8-question self-reported measure of perceived financial distress/financial well-being, is both valid and reliable. The methods and tests included the following:

• A Delphi study (interactive forecasting method reliant on a panel of experts)
• Pearson Product Moment correlations (the measurement of strength of a linear relationship between two variables)
• T tests (an analysis of whether two groups are statistically different from each other) for differences between financially distressed consumers and the general population
• Factor analysis (a technique used to reduce a large number of variables into fewer numbers of factors), and
• The Cronbach’s alpha statistic (A measurement of reliability, typically for statistical reliability). The article notes that the instrument is appropriate for use with both the general population and financially distressed consumers. The article also provides a review of previous literature on measuring validity in the field of social science and research.In the creation of the IFDFW Scale, developers needed to decide which personal financial concepts would be used as a measure of understanding someone’s financial distress/well-being. In order to do that made sure personal financial concepts were featured in previous conceptual frameworks or research. Through this process the developers had a starting point of 58 personal financial concepts. To refine those concepts the developers then had to make sure that the concept did the following:
• Clearly describe a distinct aspect of financial distress and/or financial well-being
• Be different enough to avoid being confused with other concepts
• Be likely to occur in a substantive proportion of the population; and
• Be likely to occur with adults whether or not they use credit cards and installment loans/leases.

Key findings

Following the further refinement of financial concepts developers were able to reduce the number of concepts to ten. That ten consisted of the following:

• Worry about being able to meet normal monthly living expenses
• Living today on a paycheck-to-paycheck basis
• Feeling about one’s current financial situation
• Stressed about one’s personal finances in general
• Feelings about level of financial stress today
• Satisfaction with present financial situation
• Ability to handle $1,000 financial emergency
• Availability of money to pay for a minor emergency
• Knowledge of personal finances
• Ability to manage money.

Subsequently the developers provided evidence that the IFDFW Scale was both valid and reliable.

This report points out that in every single assessment method used to determine the validity of the IFDFW Scale, there was clear evidence that the instrument was measuring what it was supposed to measure.

The report also states that the IFDFW Scale represents a valid and reliable measurement tool to assess levels of financial distress/financial well-being in both the general population and groups of financially distressed consumers.

Points to consider

• • This is an evaluation of a scale to measure levels of financial distress/financial well-being specifically among people in a US context, so will not be directly transferable to the UK.
• • The authors of the report are also among the developers of the scale being evaluated and are therefore very close to the topic of study.