StatCounter

Wednesday, November 08, 2023

Cost & benefit – anti-money laundering (AML) performance measurement

Money Laundering Bulletin, 18 October 2022

So much anti-money laundering activity calls for big data analysis it is ironic, at the least, how little rigorous statistical study there appears to be of its impact, but if you can’t, don’t or won’t measure a strategy’s results, how can you know if it’s working? Paul Cochrane goes in search of key perfomance indicators (KPIs).

How effective is the ongoing fight against money launderers and terrorist financers? Anti-money laundering (AML) experts are arguing that despite the Financial Action Task Force's (FATF) increasing focus on effectiveness in mutual evaluation reports and its extensive data on jurisdictional compliance with its Recommendations, more statistical indicators would be valuable. 

NRAs – worth it?

National Risk Assessments were supposed to help provide some guidance on the risk-based approach for AML/CFT, but the World Bank has been underwhelmed by their value. Earlier this year (2022), it released a report, 'National Assessments of Money Laundering Risks: Learning from Eight Advanced Countries’ National Risk Assessments'. Its findings on NRAs from countries supposedly at the forefront of AML – the USA, UK, Singapore, Japan, the Netherlands and others – were damning.

“The experience of reviewing 11 published NRAs from leading nations is troubling,” wrote the authors. “They lack conceptual clarity, the data are highly limited, most are analytically weak or fail to explain the methodology, and the whole goal of the NRA—to inform policy decisions—is often missed or at least not made explicit in the published version.” (1)

The report is indicative of the wider problems afoot in tackling financial crime, not least in how AML measures are assessed in terms of efficiency and effectiveness, said John Cassara, an AML expert and former US Treasury Special Agent: “Whatever the financial industry is spending on compliance – roughly US$30 billion in North America alone – and compared to government spending, it is easily much more. Count up law enforcement and regulator expenses – although no one has counted that – it has to be in the tens of billions of dollars. Do the maths, how much are we spending on combating AML and what are we getting back? The numbers are terrible.”

Fractional ROI

Estimates on total money laundered or available to be laundered from the proceeds of all crime globally is estimated at US$2-6 trillion annually, said David Lewis, managing director and global head of AML advisory at Kroll, the New-York-based risk advisory firm. The cost of compliance in Europe has been estimated by European Union police agency Europol at Euro 300 billion a year, while there are estimates of global expenditure of up to US$1.3 trillion, he added.

“This compares to a fraction of 1% of the proceeds of crime being recovered by the authorities. Not a good return on investment when looked at through that lens. There are of course many problems with such comparisons but it is useful in making the point about the need for better measures and improvements in both efficiency and effectiveness of those measures,” said Lewis.

In the way of collection

The former executive secretary of FATF attributed the general problem of measuring effectiveness, costs and benefits to the “the lack of data and the lack of consistent data between countries, firms, regulators and so on. There is resistance from government officials to more systemically collect and report data given the additional burden of doing so,” he noted.

Needless to say, this is not how the system is designed. FATF Recommendation 33 states: “Countries should maintain comprehensive statistics on matters relevant to the effectiveness and efficiency of their AML/CFT [countering the financing of terrorism] systems. This should include statistics on the STRs [suspicious transaction reports] received and disseminated; on money laundering and terrorist financing investigations, prosecutions and convictions; on property frozen, seized and confiscated; and on mutual legal assistance or other international requests for cooperation.”

FATF also notes during mutual evaluation reports, “it is the responsibility of the assessed country to demonstrate that its AML/CFT system is effective. If the evidence is not made available, assessors can only conclude that the system is not effective”. (2)

Tristram Hicks, an AML and criminal justice expert in the UK, said that despite the FATF Recommendation for meaningful statistics being in place for over 30 years, there was a dearth of such figures: “FATF doesn’t provide guidance on what those statistics should be or enforce the issuing of statistics,” he said.

Apples and pears

Moreover, mutual evaluation reports only happen every 10 years, and these assessments show that “only 20% of countries evaluated by FATF are in the ‘substantially effective’ category,” said Hicks. “And as countries collect statistics in their own way, you can’t make international comparisons.”

An example of this is that Spain recently published a report on asset confiscation, showing an increase from Euro 18 million to 91 million over a five year period, but it did not state the number of people convicted. In other jurisdictions, reports are not always released to the public. “The UK passed the Proceeds of Crime Act in 2002 but only publicly reported its asset recovery results from 2015 onwards,” noted Hicks.

The World Bank report on NRAs also noted such reticence, including on the analytical methods used for assessment. “Some NRAs did nothing more than present numbers ... Japan presented numerous tables describing criminal justice processing of various kinds of cases; the relationship of those data to money laundering risks was never explained. The UK stated that it used a model developed by law enforcement agencies that cannot be found in open-source form, so no one can assess its credibility for the purpose.”

Lewis noted that this lack of transparency extends to cost-benefit analysis of AML/CFT measures. It is carried out, typically, when new laws and regulations are introduced, such as in the European Union and the UK, he said.

Best guess, at best?

Lewis cited a cost-benefit analysis of the UK’s regime when he was at HM Treasury, in 2010, that was not published: “In part, this was because industry stakeholders thought there was no representative estimate for the costs on business as it varies by so much based on so many different factors, from firm to firm and because the same measures meet several different purposes and so the cost is not just an AML cost,” he said. Lewis suggested getting information from governments on AML costs through Freedom of Information Act requests, in countries where such legislation exists.

A wider problem is that cost-benefit analysis must include preventative benefits, which financial institutions are involved in through customer checks, transaction monitoring, and the filing of STRs. However, “these are very difficult to measure,” said Lewis. “The cost benefit analysis my team did around 2010 had a go at this and there are ways of doing it but it’s a finger in the air at best.”

AML veterans think that since FATF’s establishment in 1989, more headway should be evident: Cassara argues, “We should have seen more AML progress by the metrics that matter - asset forfeitures and convictions. FATF needs to look hard at themselves: What are we doing wrong? They are too caught up in box-checking whether a country is compliant or non-compliant with the 40 recommendations. For 15 to 20 years we’ve been trying to get all countries on the same page, and work with the same FATF directives, and we have done that now, so it is time to rethink it. The bottom line is go after the criminals, and most countries are not,” he said. (3)

Frustration, a spur

Such criticism was noted by the World Bank in its report: “Conducting an NRA may also be
seen as just a box-checking exercise to meet international standards, leading to a lack of motivation to do anything more than the internationally accepted bare minimum.”

The report further highlighted some of the chronic problems with assessing risk, which is, after all, the model advocated by FATF - the risk-based approach. “Most NRAs simply repeat the FATF Guidance definitions but then do not say how, for example, threats might be measured; they simply list a series of offences.” The report also notes that countries have not agreed on definitions of what areas are high risk and low risk, which is “not as helpful as they could be”.

The report calls for setting up an international research centre focused on money laundering risk assessment “to advance understanding of how to conduct such an assessment well,” and for more databases to share information.

“To our knowledge, no country has created a database of proven money laundering transactions to determine what can be learned about threats and vulnerabilities,” it noted. The World Bank has, however, started encouraging countries that use its NRA tool to use its proceeds of crimes data collection tool, which aims to collate money laundering cases and proceeds of other serious crimes as well as develop a live database to strengthen countries' data collection and management systems. (4)

The Wolfsberg Group of 13 international banks has also issued statements on how they think financial institutions can better assess risk and demonstrate effectiveness, with one paper published in 2019, and a second in 2021.(5) Suggested steps include assessing risk in defined priority areas; implementing and enhanced risk-based AML controls; prioritising resources; better engagement with law enforcement; and publicly demonstrating AML/CFT programme effectiveness.

While such guidance and databases on proceeds of crime are welcome, ultimately more research needs to be carried out to improve the AML regime, said Hicks. “I think it is worth us all considering that as we spend billions on compliance, surely we can spare some money for research to see how it works, and do it better,” he told MLB.

Notes

1) National Assessments of Money Laundering Risks: Learning from Eight Advanced Countries’ NRAs, World Bank 2022 - https://openknowledge.worldbank.org/bitstream/handle/10986/37305/IDU02386c9b80f623045e70bf8a09a30b6162595.pdf?sequence=1&isAllowed=y

2) https://www.fatf-gafi.org/media/fatf/documents/methodology/FATF%20Methodology%2022%20Feb%202013.pdf

3) FATF was not available for comment.

4) https://pubdocs.worldbank.org/en/422681580318460585/POC-Update-04.pdf

5) https://www.wolfsberg-principles.com/sites/default/files/wb/Wolfsberg%20Group_Demonstrating_%20Effectiveness_JUN21.pdf



 

No comments: