Every buyer’s guide you read online about deepfake detection software answers the same question: how accurate is this software?
The vendors’ homepages will respond with a number, typically in the range of 95% to 99%. Then these numbers are repeated in a feature table, and it’s called a comparison guide.
But when you’re shopping for a security tool, what you should really be asking is this:
How effective is this tool against someone who is specifically trying to beat it?
In this blog, you will discover what adversarial robustness means, without the formal jargon.
What the published research actually reveals, and why this threat model is far more important than simply an accuracy score on a vendor’s homepage, as well as the four most important questions to ask your vendor on your next call.
Key Takeaways
- In published research, the scoring drops when you add a small edit just to fool the specific tool (even if it’s not noticeable to the human eye)
- These changes not only survive perfect testing conditions but also persist after standard video compression, which is much closer to what happens when videos are uploaded online.
- One of the things we’ve noticed is that the attacks keep hopping from one detection model to another, not just the one they are built against.
- Most vendors do not have any information available related to adversarial testing, red-teaming or hardening against this type of attack.
- A real attacker is not “a random AI-generated video. It is a person who is familiar with you and literally studies you. The accuracy number on the homepage can’t describe it.
The Number on Every Homepage Measures the Wrong Adversary
Imagine how the vendor could get to 98% accuracy. They’d run a set of standard deepfakes through their model and measure how many they correctly identified.
Because that only checks one thing: whether the tool can detect AI-produced media that wasn’t designed to be picked up.
But it does not imply anything about what might happen if the attacker decides to get past a tool after carefully studying it and finding ways to trick it. In security, this is the entire reason you are paying for the tool.
Never Worry About AI Fraud Again. TruthScan Can Help You:
- Detect AI generated images, text, voice, and video.
- Avoid major AI driven fraud.
- Protect your most sensitive enterprise assets.
In the research world, there is a term for that difference between “accurate against ordinary fakes” and “accurate against someone trying to beat you”. It’s called adversarial robustness, and it’s been called that name for six years.
What “Adversarial Robustness” Means
Adversarial robustness is a detector’s ability to make correct decisions despite someone making very slight and invisible changes meant to fool the detector.
Let’s understand it with a simple example.
Let’s say the Deepfake detector is like a security guard trying to do it’s job to verify IDs.
A test would ask: “Will this guard be able to see if someone is using an illegible fake ID?”
Well, most detectors do a pretty good job at detecting it.
But real fraudsters are getting more skilled, so now the question should be:
“Can this guard spot a fake ID even if someone specifically studies the guard’s weaknesses and decides to design the fake ID to fool him?”
That’s what adversarial robustness means.
A repeatable phenomenon was observed. If you have a deepfake that a detector has identified as fake, you can make a tweak to the file that a human won’t see, but the same detector will confidently say that it’s real.
And multiple research teams have shown this problem across:
- Images
- Videos
- Audio
And they have repeated the results over several years.
| The majority of these studies used open-source academic detectors that researchers are able to explore and adjust.These attacks have not been shown in public settings against any particular commercial products designed for deepfake detection.That’s not to say that commercial tools are off the hook.It just means we don’t know, as companies typically don’t publish their resistance to these attacks. |
What the Published Research Actually Shows
The UC San Diego and Facebook AI teams conducted a 2021 study on deepfake detectors that performed well, identifying more than 95 out of every 100 deepfake videos.
They used those same videos and used a technique to slightly modify them to fool the detectors. The accuracy of the detector dropped to a few percent. The performance was significantly affected, which is alarming.
Read more about deepfake Statistics: AI Fraud Data and Trends in a separate blog post.
Another companion journal paper noted that the problem had received little attention in prior deepfake detection research and that the attack would still be successful even when the attacker couldn’t view the model’s internals.
It not only affects videos. Let’s talk about a 2025 peer-reviewed study presented at the IEEE/CVF Winter Conference that demonstrated a decrease in tool’s accuracy from 98% all the way down to 26% when the attacker had complete access to the model and made a few changes to the fake audios to confuse the tool.
To be more realistic, the next question is: Can an attack designed for one detector also fool another detector?
Well, the same research did find the answer to that.
The attack designed to fool detector A was used to test detectors B and C, resulting in a drop in accuracy from 91% to 46% and from 94% to 67% on two public datasets, respectively.
By now, we’ve proven the point here, but you should be aware of these studies and public facts before you buy deepfake detection software.
First is the 2023 paper on 3D adversarial face manipulation, which described an attack with a 96.8% success rate even when the attacker had no access to the tool’s source code or internal design.
But the next is the AADD (Adversarial Attacks on Deepfake Detection) 2026 challenge, which focuses on the robustness of deepfake detection systems in real-world situations like:
- Compression
- Resizing
- Re-encoding
- Uploading
- Resharing and downloading
Every year, the researchers compete in different challenges to improve deepfake detection tools. But this issue has not yet been resolved. The same was the reason behind the NTIRE 2026 competition, and here’s what’s been concluded:
| Many detectors perform very well on clean and high-quality files, but their scores drop significantly when the files are being edited, changed, modified, or compressed. |
So you need a reliable tool that can detect fake images, videos, or audio with the same accuracy as the detector checks real files.
Why This Matters More Than the Accuracy Percentage
Typically, a detector’s claimed accuracy is based on a standard, unaltered deepfake test. Those tests are conducted to see how the software works when no one is trying to bypass it.
For low-risk jobs, those numbers could be sufficient for using deepfake detection.
However, when you are trying to prevent fraud, identity verification fraud or disinformation campaigns, fraudsters can simply change their content to escape the radar. You can read more real-world examples of these frauds.
When it does, you should be aware that a high accuracy percentage is only part of the picture. So you should know how the detector will behave if it’s being actively bypassed.
Learn how the most sophisticated AI detection platforms are stopping million-dollar fraud attempts and transforming the economics of cybersecurity.
Four Questions You Should Ask a Vendor
You should be asking more penetrating questions than the other buyers who come on the calls. Great vendors would be able to address these concerns even without the proof.
Here are four questions that you can ask, and you don’t need to run an adversarial attack first.
- Have you tested your detector against adversarially modified fakes, not just clean adversarially unmodified generated media?
- Have you tested whether any attacks that target other vendors will work against you?
- Does your accuracy number remain the same after the usual compression (which occurs on upload)
- Will you provide results from a setting in which the tester did not have access to your model’s inner workings, or just a clean-sample accuracy score?
Most vendors won’t have to have a well-crafted response to these questions, but what they should have is helpful information. A vendor who has considered this problem will be able to provide helpful insights directly.
Choose the right deepfake detection software
Below is the guide you can follow when choosing the right deepfake software for yourself:
- Choose why you need the tool, whether it’s for image, video, audio, document, or all of them.
- Don’t just rely on manufacturer claims; ask for independent test results.
- A good detector should be able to detect fake content while minimizing the false detection of real files.
- Check if the software has been tested with content from social media, messaging applications, compressed videos, and edited files. (Lab-testing alone isn’t the only answer)
- Ensure that it integrates seamlessly with your current processes, whether you’re using APIs, dashboards, or automation.
- Determine whether the vendor complies with your organisation’s privacy, security, and regulatory requirements.
- Select something that is affordable for you now and can accommodate more as your workload increases.
- Reputable vendors should be ready to discuss testing methods, limitations, testing frequency, and test improvements, rather than claiming a 100% detection rate.
- Test the software with your own content before making a final decision. This will help you to understand what it looks like in your real environment.
Adversarial robustness is a line of questioning, and there are no published benchmarks you can use to compare vendors today, but if you ask about it, a good vendor should be able to address this concern for you.
Common Pitfalls to Avoid When Choosing Deepfake Detection Software
- Never treat the ‘homepage accuracy number’ as a base truth.
- Remember that the tool’s accuracy is measured under conditions that could be completely different from yours. So always ask the vendor about your exact situation to see if the tool is the right match.
- Don’t assume that if a vendor has no public conflict, they are unlikely to be suspected of one.
- The absence of a reported failure doesn’t imply that there’s no resistance to failure.
Frequently Asked Questions
What is adversarial robustness in deepfake detection?
It is the ability of a detector to detect content that has been deliberately altered to deceive it. Most published accuracy claims concern whether the tools can detect ordinary deepfakes and that is why a high accuracy score does not necessarily imply robustness.
Can adversarial attacks on deepfake detectors really survive after being uploaded to social media?
Yes. The research published specifically tested whether such modifications can withstand regular video compression used for automatic upload and found that they can. And that’s the real concern.
Do any commercial deepfake detection vendors publicly address adversarial robustness?
Not if the top famous vendors are not talking about this publicly, that doesn’t mean that their tools are weak. It is because people who are buying the stock are not readily informed of either side, but that is why it is worth asking on a call.
Does a high accuracy score mean a detector is secure against a motivated attacker?
Not on its own. A high score indicates that the detector works well with AI generated content. It does not tell you how it performs against an opponent trying to beat it. Better to ask vendor’s take on a real call.
Is this the same as saying deepfake detection tools do not work?
No. It simply indicates that the accuracy number that you’re told is a more limited question than most buyers realize. Despite all that, the tools still manage to capture the vast majority of the content that they’re designed to capture in the real world.
The gap is what happens when the motivated and technically capable attacker tunes their fake to beat your tool.
How can I evaluate a vendor on this without running my own adversarial research?
Use the four questions above and ask directly. If the vendor has been thoughtful about adversarial robustness, they are likely able to articulate their strategy in non-technical terms, without even a formal benchmark to call on.
Final Thoughts
So it’s fair to say that a majority of deepfake detection purchases happen on a number that was never designed to describe the attacker you are actually concerned with.
You don’t need a new benchmark or a lab of your own to fix it.
It involves asking a better question and being willing to pay attention to the answer that comes from the vendor.
It doesn’t look like we’re the only ones using this point, but we would like to note that there isn’t currently a formal adversarial robustness benchmark for TruthScan online either.
We’d like to be the vendor that leads buyers to the right question, not the one repeating an accuracy number that wasn’t designed to answer it.
If adversarial robustness ever becomes a part of the vendor bidding process, then all the detectors in the world, including ours, will be measured more fairly than they are now, and that’s a market we’re willing to be measured in.
Try TruthScan today.