Product testing

An introduction to product testing

Product manufacturers are expected to base their work on REAL consumer needs, characteristics (including those which characteristics of vulnerability) and objectives within an identified target market. Firms should not merely look to copy other products or services in the market.[1]? As such, the Consumer Duty rules require firms to undertake appropriate testing of their products or services.[2]? This will include scenario analyses where relevant.

The nature of product testing

In all cases, firms must undertake QUALITATIVE testing of products and services. For example, they could consider likely changes to the target market’s needs in the future and whether the product or service would continue to meet those needs. Where relevant, depending on the type and nature of the product or service and the risk of harm, firms must also conduct QUANTITATIVE testing. This could include, for example, testing how investments would perform in different market conditions.[3]

Firms should give consideration to what may happen in the FUTURE, and not just what has happened in the PAST.[4]? For example, firms should consider how the product or service is likely to function over its proposed term and, where different, the average time customers are expected to hold the product or service, so they can properly assess all potential risks to customers.[5]

What if a product fails testing?

If the results of product testing show that the product does not meet the identified needs, characteristics and objectives of the target market:

1. in relation to new products (or a significant adaptation of an existing product): the manufacturer must NOT bring the new or adapted product to the market, and
2. in relation to existing products: the manufacturer must immediately:

• cease marketing or distributing the product (whether directly or indirectly),
• cease any renewals for existing retail customers, provided that existing retail customers are easily able to move to an alternative product that provides at least the same level of benefit at an equivalent cost to the customer, whether with the firm or with another firm, and
• where the firm intends to continue to market and distribute the product, make such changes as are necessary for the product to meet the identified needs, characteristics and objectives of the target market.[6]

Testing product communications

The FCA expects firms to be able to demonstrate consumer understanding.? This is primarily achieved through testing of communications and their impact.

Firms who are responsible for the production (or adaptation)[7] of communications must be able to demonstrate:

1. that they have an approach to testing communications that delivers good outcomes,[8]
2. how they have actually tested consumer understanding, and
3. where appropriate, that (and where) improvements have been made to their communications.[9]

Firms which have direct interactions with retail customers, whether or not they are also responsible for the creation or adaptation of communications, must monitor the impact of those communications[10] and provide feedback to the originator of the relevant communication.[11]? An example would be firms that provide customer services (whether outsourced in whole or in part).

Testing should check that communications can be understood by customers, so they can make effective decisions and act in their?interests.[12]? Testing should normally be conducted BEFORE communication is made with customers.? In addition, the impact of communications should be monitored so as to assist firms in identifying whether they are supporting good outcomes for retail customers.[13]

Examples of testing

Forms testing may take include:

1. experimentation in the form of randomised controlled trials or A/B tests with real customers or online experiments,
2. surveys – asking a sample of customers for feedback and responses via a questionnaire (online or on paper),
3. interviews, or
4. focus groups.[14]

Customer communication champions

Firms may wish to train internal ‘champions’ in the principles of good customer communications. These individuals can independently review communications from a consumer perspective, and help firms develop and maintain best practice.[15]

Not all communications need to be tested

It is important to note that it is NOT necessary to test ALL communications in advance of sending them.? Rather, firms should determine which communications should be tested.[16]? In determining whether testing of a communication is appropriate, a firm should consider factors such as:

1. the purpose of the communication and, in particular, if it is designed to prompt or inform a decision, and the relative importance of that decision,
2. the context of the communication, its timing, and its frequency (for example, it is likely to be more appropriate to test communications that could impact many retail customers),
3. the information needs of retail customers,
4. the characteristics of vulnerability of retail customers,
5. whether the scope for harm to retail customers is likely to be significant, including if the information being conveyed were misunderstood or overlooked by retail customers, and
6. whether, to support good outcomes for retail customers, it is more important to communicate information urgently, rather than carrying out testing beforehand.[17]

When to test communications

Testing should usually be carried out in advance of communicating the relevant information to customers (for example, when firms are developing sales literature or telephony scripts in relation to a new product).

However, the FCA recognises that there may be times when firms need to respond to incidents at pace.? In these circumstances, they must balance considerations in relation to testing – and the associated elapsed time – with the need to intervene urgently to protect customers from harm.

In addition, it may not be possible to test certain communications, such as ad-hoc conversations during customer service?calls.[18]

Next time

Next time, we’ll be turning our attention to look at compliance with the Consumer Duty within the “Product Approval” stage, so stay tuned!

[1] FG22/5, 6.40

[2] FG22/5, 6.43

[3] FG22/5, 6.44; PRIN 2A.3.9R

[4] FG22/5, 6.45

[5] FG22/5, 6.46

[6] PRIN 2A.3.10R

[7] PRIN 2A.5.11G

[8] FG22/5, 8.44

[9] Policy Statement PS22/9, 8.11

[10] PRIN 2A.5.11G

[11] FG22/5, 8.53

[12] FG22/5, 8.40

[13] PRIN 2A.5.10R(1)

[14] FG22/5, 8.57

[15] FG22/5, 8.56

[16] Policy Statement PS22/9, 8.12

[17] PRIN 2A.5.12G

[18] FG22/5, 8.43