Deepfakes in Court: An Evidentiary Quandary

Imagine being on trial for a crime you didn’t commit, only to find a video of yourself ‘confessing’— a video you never recorded.?

Deepfake technology is becoming more sophisticated, challenging our perception of what constitutes reality. No one is immune—businesses risk financial fraud, media outlets struggle with misinformation, legal systems grapple with falsified evidence, and government agencies face threats to national security. In 2024, a Chinese finance worker was tricked into transferring millions of dollars to fraudsters who had used deepfake technology to simulate the faces of his company’s Chief Financial Officer and co-workers in a video conference call.?

The question no longer revolves around whether or not deepfakes will be weaponised but how to combat them when they inevitably are.

The threat posed by deepfakes is especially severe in the judicial system, given the court's heavy reliance on evidence to prove or disprove guilt. This reliance necessitates presenting facts in various formats: video and voice recordings, digital contract surveillance footage, forensic analysis, and expert witness testimony. However, with the rise of deepfake technology, these once-trustworthy sources can now be manipulated with alarming precision.

For instance, a fabricated video could depict a defendant attending a meeting they were never present at or confessing to a crime they never committed. Worse still, an AI-generated audio clip could falsely place someone at a crime scene. Digital contracts, signatures, and even live video depositions can be altered without detection, eroding the essence of ‘evidence’ in legal proceedings.

Undeniably, ‘seeing’ is no longer ‘believing,’ and the judiciary faces an urgent challenge: How can courts separate truth from deception when AI can fabricate "evidence" indistinguishable from reality?

The Challenge

Fabricated evidence is not a new challenge to the legal system. Hence, it is only a matter of time before deepfakes make their way into the courtroom. The real problem arises in relation to accurately identifying such evidence for what it is when it is presented. Moreover, how should courts handle objections from opposing parties claiming that submitted evidence is a deepfake?

Confronting the Mirage

As this dilemma inches closer to the courtroom, the legal system must prepare for a new era—one where reality itself is on trial. Addressing this conundrum requires more than scepticism—it demands a multi-faceted approach that combines legal safeguards, forensic advancements, and proactive policies.

Countries like the United States are already enacting bold policies to safeguard the creation and use of artificial intelligence. For instance, the DEEPFAKES Accountability Act, introduced in September 2023, seeks to protect national security from deepfake-related threats while providing legal recourse for victims of harmful deepfakes. Similarly, the NO FAKES Act, proposed in 2024, aims to establish a federal framework that safeguards individuals from unauthorised digital replicas that could violate their privacy or damage their reputations. Likewise, the UK government is set to introduce restrictive policies aimed at charging perpetrators who create and share explicit deepfakes. Whether or not this impedes creativity and the ability of these nations to take full advantage of this new technological potential is another subject.

Policies

For the purposes of this discussion, it is recommended that stricter measures and policies be enforced against companies that make deepfake creation tools accessible. This will place an onus on such platforms to implement appropriate measures for the use of their works. We also suggest that governments send a clear message through policies that double down on the improper use and circulation of deepfakes.?

Forensic Analysis

In the judicial system, courts may also need to institute practice directions that require all evidence to be subjected to forensic analysis before being presented in court.?

Burden Of Proof

The burden of proving the authenticity of digital evidence may also be imposed on the parties making such submissions.?

Innovations

Innovations in deepfake detection must be prioritised, as seen in the United States, where the Department of Commerce has been tasked with developing content authentication and watermarking guidelines, ensuring AI-generated materials are clearly labelled to prevent misuse. This has spurred American-based big tech companies to innovate new measures, such as Google’s hidden watermark technique.?

Corroboration Requirements

No digital evidence should be admitted unless corroborated by at least one independent source, reducing the risk of fabricated evidence influencing judicial outcomes.

Chain of Custody Regulations

Courts should require a transparent and verifiable chain of custody for all digital evidence, ensuring transparency in its handling and preventing unauthorised alterations.

The rise of deepfakes isn’t just a technological shift—it’s a legal revolution. As technology advances, deepfakes will only continue to improve, necessitating some, if not all, of the actions highlighted above. If we don’t act now, our justice system may struggle to keep up with a world where seeing is no longer believing. #AIinCourt #DigitalEvidence