The face of fraud has evolved dramatically over the past few years. Email scams, hacking and website impersonation are commonplace. But the evolution of Generative AI, large language models and an increasingly virtual world mean that fraud now has technological backing in a way we haven’t seen before. Some of the tell-tale signs of fraud we have been accustomed to – such as poorly worded and grammatically incorrect email approaches – have given way to word perfect messages, highly convincing deepfakes and sophisticated ransomware attacks. The advancements in technology mean that very little skill is required on the part of fraudsters to execute such scams and it has become cheaper and easier to scale the same fraud model to reach a larger number of possible targets.
In this article we explore how AI is being used in fraud, the impact this is likely to have on dispute resolution and what can be done by individuals, companies and governments to address the threat of AI-enabled used fraud.
How is AI being used in fraud?
Research conducted by PricewaterhouseCoopers LLP and Stop Scams UK in December 2023 identified that AI is being used in the following ways in connection with fraud:
- Generative text and image content – Generative AI (GenAI) is being used to create fabricated content, whether that be emails, instant messages, or other content, which is being used as part of scams. It can successfully imitate the sentence construction style of the person whose identity it exploits.
- AI enabled chatbots – sophisticated chatbots are being used as part of frauds to manipulate victims into making payments.
- Deep fake videos – deep fake videos are being used in several ways. For example, they can be used as click bait to drive traffic to websites which harvest user's personal information which is then used in payment frauds. They can also be used to bypass security controls.
- Voice cloning –voice cloning is being used in scams. For example, an employee could receive a call from somebody asking them to make a payment to a certain account, but the caller's voice has been cloned to resemble the voice of a colleague. More sophisticated AI systems can mimic the desired language, tone and intonation of the cloned individual to make the call more persuasive. Voice cloning is also being used to try to penetrate bank's biometric ID systems.
- Sophisticated targeting of victims – AI tools can be used to review large volumes of data and tailor scam content. This type of scam is known as "spearfishing". Using AI, spearfishing can be done on an automated basis and on a larger scale.
- Pressure testing – while researchers could not evidence that AI was currently being used for attacks on banks, it is expected that AI may be used in this way in the future to identify vulnerabilities than can be exploited.
While AI can be exploited for fraudulent behaviour, it also serves as a powerful tool for detecting fraud. For instance, AI systems can analyse written or spoken communication to identify unusual patterns or irregularities, effectively flagging potential scams before they escalate. This dual capability emphasizes the importance of leveraging AI technology responsibly in both prevention and detection efforts.
Recent case involving AI fraud
We are already seeing some sophisticated instances of AI enabled fraud. We recently acted for a client in Slovakia in respect of what is often referred to as an "authorised push payment scam" or "CXO fraud" where the fraudsters used deep fake technology to impersonate one of the company's non-domestic executives and instructed the client's accountant via telephone call to urgently transfer a multiple million dollar sum to the fraudsters' account in Hong Kong.
The client discovered that the transfer was fraudulent three hours after the transfer was submitted via the online banking system. Attempts were made to stop the transfer but they were unsuccessful. On receipt of the funds, the fraudsters distributed the funds to multiple accounts across different banks. We successfully assisted the client in obtaining disclosure orders to trace the funds and freezing orders to prevent further distribution. The client was ultimately successful in recovering a large proportion of the stolen funds.
The accessibility and sophistication of deep fake technology and voice cloning means that it is no longer possible to rely on the fact that you see a familiar face or hear a familiar voice – most people have some presence online which provides access to their photo and / or their voice and it only takes a small set of that data to create a convincing deepfake. Additional safeguards are therefore needed, particularly when issuing payment instructions. Businesses should review their payment approval processes to address the evolving risks, ensure their staff are properly trained on those processes and should discuss with their bank and payment service providers to ascertain what fraud prevention technologies can be deployed as an additional safeguard.
How is AI fraud likely to impact dispute resolution?
We expect the use of AI to commit fraud will impact the types of disputes that individuals and companies find themselves in, as well as the way in which disputes are resolved by courts and arbitral bodies.
As to the types of disputes:
- We expect to see an increase in claims brought by victims of fraud in which AI is a key feature, whether that be GenAI, deep fake technology and/or voice cloning. In the UK, we anticipate that many of those cases will continue to be brought in the civil courts, where victims can obtain speedy relief such as freezing and disclosure orders, but that recovery is likely to become more difficult because the more convincing the fraud is, the slower the victim is likely to be to realise what has happened, and that delay can be critical in terms of the impact on recovery prospects.
- We may see an increase in claims brought by victims of payment fraud against banks or other service providers for failing to prevent the fraud. In the UK, in October 2024, the Payment Systems Regulator introduced a new mandatory reimbursement framework in relation to payment services providers which will compensate victims of authorised push payment fraud and this provides an alternative route for victims to recover losses up to the limit of £85,000.
- We may also see an increase in reputation protection proceedings, as deepfake and GenAI can also be used by perpetrators to mimic a specific person and attribute statements or actions to them, resulting in reputational damage.
- We expect an increase in the number of disputes in which the authenticity of the underlying agreements is in issue. For example, a party may seek payment or performance under a contract in circumstances where the other party argues that the agreement has been fabricated using AI, or that their signature to the contract has been fabricated. Similar arguments may be made in relation to correspondence regarding agreements.
- Wherever a dispute involves the alleged use of AI technology, we expect to see an increase in the use of AI detection tools by litigants in order to prove that AI has in fact been used – whether that be to fabricate an underlying agreement, a payment instruction or communications between the fraudster and the victim.
What is being done in the UK to prevent AI fraud?
Tackling fraud has high on the legislative agenda for a number of years in the UK and we expect that to continue, with tech enabled fraud becoming an increasingly prominent feature. Some of the recent measures introduced in the UK to tackle fraud include:
In November last year, the UK government entered into a voluntary agreement with the technology sector to reduce fraud on their platforms and services. Its signatories include Amazon, Google, Facebook, Microsoft and others. The key actions required by companies include deploying measures to detect and block fraudulent material. A number of technology companies have already developed AI detection software and we expect as the technology being used to commit fraud continues to develop, technology companies will need to continue to invest in the development of prevention measures. However, given the voluntary nature of the charter, its scope is limited.
Ofcom is the regulator responsible for overseeing the implementation of the Online Safety Act which introduces a number of new offences for companies with an online presence for failing to prevent online harm, which includes fraud. The FCA recently highlighted the importance of the Act in fighting fraud on tech platforms, particularly APP fraud (see here).
Ofcom is implementing a number of Codes of Practice in support of the OSA and the first were published at the end of 2025. This included guidance on risk assessments to protect people from illegal harms online including fraud and financial offences.
- Economic Crime and Corporate Transparency Act 2023
The Economic Crime and Corporate Transparency Act 2023 ("ECCTA") creates a new corporate criminal offence of failure to prevent fraud, which applies to large organisations and comes into force on 1 September 2025. The new offence is designed to hold relevant organisations to account if they benefit from fraud committed by their employees or agents and the organisation did not have reasonable fraud prevention procedures in place. Guidance from the government on the ECCTA and the reasonable procedures defence was published on 6 November 2024.
Given the increased use of technology in the way fraud is being committed, if an organisation is aware that there is a risk that of AI technology being used by its employees or agents to commit fraud and does not take measures to try and prevent it, then that organisation may find itself liable in the event that fraud is committed.
For more on the ECCTA and how we can help, see here.
What is being done in Europe to prevent fraud?
The EU is leading legislative efforts when it comes to AI technology, resulting in the adoption of the first comprehensive AI regulation. The AI Act establishes safety requirements that companies must meet before placing AI products on the EU market.
The Act outright bans certain AI applications deemed to pose unacceptable risks, such as manipulative AI technologies that could exploit individuals' vulnerabilities. By prohibiting these harmful applications, the AI Act aims to protect citizens from potential fraud and abuse. However, deepfake tools will remain classified as “limited risk” AI systems thus requiring user only clearly disclose that such a content is AI-generated.
AI systems falling under the high-risk category must adhere to strict compliance requirements, including risk management, data governance, and transparency obligations. However, the AI act exempts AI systems used for the purpose of detecting financial fraud from the high-risk category, simplifying their application and use.
- EU's Digital Services Act 2022 ("DSA")
Along with the Digital Markets Act, the DSA is a major pillar of the EU's "fit for the Digital Age" initiative. It is premised on the notion that every activity deemed illegal offline should be illegal online. The Act mandates that platforms establish easy-to-use systems for users to report illegal content, including fraudulent activities. This facilitates quicker identification and removal of scams, enhancing user safety.
Both the AI Act and the DSA enhance online transparency, making it more challenging for fraud attempts to succeed and easier for them to be detected. For more information on advertising requirements under the DSA and what they mean for your business, see here.
Conclusion
It is essential that businesses, individuals and legislators keep pace with the methods used by fraudsters, and identify opportunities to use AI and new technologies as fraud protection tools, as traditional methods of detecting fraud are becoming obsolete.
"Risk assessments" are increasingly becoming a mandatory feature of the regulatory landscape for businesses and it is therefore essential that businesses understand the laws and regulations that apply to them to ensure they are compliant, but also to ensure that they have appropriate fraud prevention and protection measures in place which are reviewed regularly to address advancements in technology.
