Effectiveness Reviews

The Governing Boards of regulatory bodies should regularly review the effectiveness of their organisation. When doing so, it is helpful to consider the following questions:

1. General Approach

As far as is compatible with the duties imposed by law, does the regulator leave the regulated free to live their lives as they wish and/or get on with their work, business or profession?

Do the regulated clearly understand what is expected of them?

Has the regulator done all it can to minimise the cost of compliance? Such costs include non-financial consequences such as the impact of regulation on the willingness of a business to innovate, or the willingness of volunteers to help a school, charity etc.

2. Over- or Under-Regulation?

Does the organisation achieve an appropriate balance between over- and under-regulation?

Under-regulation leaves too many loopholes that producers can avoid or evade

Over-regulation provides incentives to evade regulation, including

Note that:-

And are Inspectorates providing an effective stimulus to good management?  Or are they inadvertently reducing the ambition, innovation, responsibility and accountability of managers whose practices they have 'passed'?

3. Resources and Resource Allocation

Does the organisation have sufficient resources (money and personnel) to carry out its duties to an acceptable standard?  If not, are the deficiencies being clearly communicated to those responsible for shortfalls, including via formal annual reporting?

Does the organisation divide its resources appropriately between its four key functions?

Enabling - licensing/permissions to carry out certain activities.
Policing - enforcing compliance with legislation/regulations.
Advising - providing guidance to help entities understand and comply with legislation/regulations.
Educating & Informing - helping the public understand and exercise their rights.

4.  Enforcement

Does the organisation have the legal powers, policies and staffing required to ensure sufficient compliance with its regulations?

Does it appropriately vary its approach according to the nature and extent of the risks of regulatory failure, the nature of its regulated entities,  and their likely response to enforcement.

5. Contradictory Pressures

Has the organisation discussed and chosen appropriate responses to the contradictory pressures identified by Harvard's Professor Sparrow in his book The Regulatory Craft?:-

“Regulators, under unprecedented pressure, face a range of demands, often contradictory in nature:

6. Independence

Is the organisation satisfied with the extent of its independence from government?  The following indicators may be relevant:

A more detailed discussion of this subject may be found here.

And is the organisation satisfied with its independence from those it regulates?  Has it avoided regulatory capture

7. Accountability and Trustworthiness

Is the organisation sufficiently accountable for its decisions:

In particular, are its decision processes accessible and transparent? Does it accurately and clearly communicate the analysis that underlies its decisions? Is it candid about the compromises that may be needed?

Does the organisation appear to be trustworthy?  Does it deliver against expectation effectively, reliably, consistently, responsively?  And does it appear to operate with integrity, seeing others as equals; listening to and taking seriously their concerns, views and rights.?

Note that accountability involves explaining why the regulator has taken its key decisions. It does not mean 'obeying orders' - other than those contained in legislation and legal decisions.

A more detailed discussion of this subject may be found here.

8. Ethics

Has the organisation carried out, and implemented the recommendations of, a recent ethics health check as recommended by the Committee on Standards in Public Life (CSPL)? The health check should look at issues such as:

A helpful 2016 report by the CSPL is here.

9. Effective Regulation: Larger Organisations

Do senior staff understand the cultural and behavioural pressures that can cause catastrophic failure in the regulatory compliance of larger organisations?  These might include:

A detailed description of these pressures can be found here.

If the regulator sets targets for its regulated entities, are those targets sensible?  In the case of complaint handling, for example, are companies asked to measure whether complaints are being handled properly?  Complainants think that this is much more important than speed, for instance.   It is much better if companies take the time and devote the necessary resources to resolving complaints sensibly )and keep customers informed if this takes time) rather than rush to close a complaint quickly and then have to reopen it if the customer is dissatisfied.

10. Effective Regulation - Individuals & Smaller Organisations

Has the organisation developed a suitable range of relationships with the individuals and smaller organisations within the various elements of the typical regulatory spectrum: 

More detailed advice about the regulation of the activities of individuals and smaller firms may be found here.

11.  Governance, Targets, Performance Management etc.

Does the governing body have a good working relationship with its staff and other advisers?   

Does the organisation have performance management systems consistent with the NAO's good practice guide Performance Management by Regulators?  (And see the Notes below.)

12.  Better Regulation and Innovation

Does the organisation apply the Principles for Better Regulation and the Principles for Regulation and Innovation found in the 2022 'Closing the Gap' report?


It may not make much sense to try to quantify the effectiveness of your organisation, mainly because it is almost impossible to measure the deterrent effect, nor how this changes with numbers employed etc.  Most of the deterrent effect is probably achieved merely be establishing, and advertising the existence of, the regulator.  

Bruce Lyons commented as follows on reading the CMA's targets in 2016:

Its performance management framework commits the CMA “to achieving direct financial benefit to consumers of at least ten times our cost to the taxpayer.”  Target setting and performance measurement are an important part of performance management.  However, the precise way that the government requires the CMA to justify its funding is dangerously distortionary.

The government requires the CMA to achieve a benefit to taxpayer cost ratio of at least £10 for each £1 of taxpayer cost (i.e. budget). This ratio is completely arbitrary and without empirical foundation.  I am actually quite confident that this ratio would be comfortably met if all the benefits could be credibly measured – but they cannot. My concern is that in order to tie tangible benefits to tangible case work, ‘benefit’ is essentially measured ... by ignoring deterrence.  Think how dangerous it would be if military leaders had to justify their budget by measuring the value of wars fought, ignoring wars deterred.


Martin Stanley

Spotted something wrong?
Please do drop me an email if you spot anything that is out-of-date, or any other errors, typos or faulty links.