Impact Measurement in Open Finance

Five Thematic Priorities for Monitoring-Learning-Evaluation

Based on a review of emerging practices, five thematic areas are recommended as priorities for the first phase of a monitoring-learning-evaluation (MLE) strategy for open finance:

1. Market Innovation

Open finance seeks to increase choice between competing firms and enable new products through expanded data exchanges and improved payment systems. An MLE strategy should begin with measurement of how open finance is impacting the number and types of firms participating in the market, and the general state of competition.

2. Quality and Availability of Open Finance Information and Transactions

The quality and consistency of information exchange and payment transactions will in large part determine how much open finance improves on the current financial ecosystem. Measuring the quality and availability of information exchange and payments is already an explicit supervisory priority in most open finance regimes. Aligning these reporting requirements with analysis of the impact of data quality on things like credit risk and product diversity should be part of the supervisory strategy.

3. Consumer Use and Benefit

Impact evaluations have measured how open banking and open finance improves access to and the terms of credit products, including large effects for underserved populations. Using a combination of market level data on products on offer, terms, and firm-level data, authorities can measure how product innovation occurs in open finance, and conduct regression analysis to see how different consumer segments benefit from improved product offerings in credit and other priority product segments.

4. Market Conduct and Consumer Risks

Open finance raises several market conduct risks, in particular data protection, fraud, and debt stress. Comfort with sharing financial information and trust in open finance can be measured to understand their impact on usage—and how to increase trust and usage, taking advantage of consent requirements in regulations. Administrative data on fraud incidences can identify how open finance shifts fraud risks, and lead to more timely responses to emerging fraud issues, with built in impact evaluation of the effect of those protective measures for continuous refinement and improvement of fraud prevention solutions. Consumer surveys and account/transactional data can be used to measure whether there are any risks of debt stress arising from increased credit access in open finance.

5. Costs and Pricing of Open Finance Implementation

The funding and financial sustainability of open finance remains a hotly debated and complicated policy topic in leading open finance markets. It could be useful to measure how costs of open finance implementation shift over time, and put these in the context of the benefits for firms and consumers referenced in thematic areas 1 and 3 above. This would also help to determine pricing rules for participants to achieve a fair, if only partial, cost recovery and to sustain shared infrastructure costs of the open finance.

Key Findings on Data Sources and Sampling

The analysis reviews different data sources and sampling approaches being experimented with for impact measurement in open banking and open finance globally, including:

• Centralized dashboards: Markets like Brazil and India have developed centralized systems where the full volume of API calls and activities can be measured, enabling sophisticated analysis on uptake, usage, and quality
• Participant-level reports: Where central entities don't exist, data must be separately collected and aggregated, with trade-offs between detail, timing, and comprehensiveness
• Administrative data: Supplementing official open finance sources with government data, credit bureau data, and other official records
• Survey data: Both firm surveys and consumer surveys provide crucial qualitative and quantitative insights

Impact Analysis Methods

Four methodologies for MLE in open finance are commonly used:

1. Qualitative descriptive: Surveys and interviews of firms and consumers to understand use cases, benefits, and experiences
2. Quantitative descriptive: Volumes, usage data, performance metrics, and representative consumer survey data
3. Difference-in-difference analysis: Comparing firms or consumers who were or were not covered by open finance reforms, or those who opt in versus opt out
4. Regression analysis: Using granular account and transaction-level data to measure how different consumers experience open finance and how it impacts financial product delivery and features

Key Segmentation Approaches

Several segmentation methods are used in open finance MLE:

• Participating entity types (banks, fintechs, payment service providers)
• Customer types (individual vs. business, demographic segments)
• Data request use cases (specified purposes for data access)
• Product types (consumer vs. SME loans, payment categories)
• Types of transactions/payments (single vs. recurring, expense categories)

Global Evidence

The paper draws on impact measurement experiences from open banking and open finance implementations in Brazil, India, United Kingdom, Australia, European Union (PSD2), Spain, and other markets. Key findings include:

• Data quality issues are a major limiting factor for adoption and impact
• Open finance significantly increases fintech entry and improves their performance metrics
• Credit access improves, particularly for underserved populations and "thin file" consumers
• Interest rates on loans decrease when consumers share transaction data
• Fraud risks require ongoing monitoring and adaptive prevention strategies
• Cost recovery and pricing remain challenging policy questions requiring empirical assessment

Conclusion

The analysis demonstrates that impact measurement in open finance is still emerging, with significant variation in approaches and sophistication across markets. However, clear patterns are beginning to emerge around best practices for data collection, analysis methods, and priority areas for measurement. Authorities implementing open finance should develop comprehensive MLE strategies early, align them with supervisory requirements, and plan for increasingly sophisticated analysis as data accumulates and the ecosystem matures.