Experienced UIUC graduate student specializing in Math, Computer Science, and Writing
Availability:
Every day, 10:00am-10:00pm PST
Subjects:
Math
Computer Science
Writing
What To Do When AI Detectors Accuse You Of Plagiarism
Last Updated:
- Studies and surveys show that most AI plagiarism detectors make misleading claims about their accuracy that don't hold up to independent testing
- Most AI plagiarism detectors are relying on statistical measures that are not readily explained to see if they're logical.
- Many believe that education will have to fundamentally change, for the better or worse, to account for the future of generative AI.
There’s a feeling of powerlessness that comes with revolutionary changes. Sometimes a trend moves with such great momentum that it seems unstoppable. It challenges us to redefine both our world and our place in it.
Artificial Intelligence is one such revolutionary change. AI tools are in their infancy right now, yet they’ve already caused crises in areas like customer service, data analysis, and creative content creation. The worst crisis of all may be happening in education as students try to get ChatGPT to do their assignments — an existential threat to the model of education that conventional schools use. I’ve known students who got caught using ChatGPT multiple times, even when facing the threat of expulsion from college.
Many snake oil salesmen have come out of the woodwork to take advantage of people’s fears about these radical changes. AI products with questionable, if any, track records of success are marketing themselves aggressively to people trying to future-proof themselves by adapting to new technology.
Because of the complexity of these models and the lack of good evaluation methods, it can be hard even for experienced AI practitioners to assess the benefits and limitations of a specific product. In education, this has led to AI plagiarism detectors that make grand claims about helping teachers avoid ChatGPT and other LLMs in their classrooms. However, most of these services are terrible for several reasons.
If you’re a teacher or professor considering these tools to enforce academic integrity, be very wary of them for the reasons below.
If you’re a student, this article will help you make a case for yourself if you’re unfairly accused of using AI.
Inaccuracy and Misleading Claims
AI plagiarism detectors have high false positive and false negative rates. This means they commonly label human-written work as AI-generated and AI-generated work as human-written. These errors in the software are even worse for some demographics like non-native speakers, who are more likely to use a formal tone or word choice that resembles an AI.
OpenAI, the company that created and manages ChatGPT, had to take down their own AI detection software after numerous errors were discovered. Yes, even the leading developers of the most advanced AI models are stumped when it comes to detecting its content.
These products might claim to be 99% accurate, but that’s very misleading. There are lots of ways to distort the effectiveness of models:
Using simple, cherry-picked examples to test with that resemble training data closely
Having a high false positive rate, meaning that a lot of human-written work is incorrectly flagged as well
Ignoring subtle forms of plagiarism where students might steal an idea but change the words
Independent testing shows that these detectors are nowhere as good as their marketing claims. Some computer scientists, myself included, believe that this is an impossible problem to solve. There are just no features or rules you can rely on to detect plagiarism using statistical properties.
Lack of Transparency
Normal plagiarism detectors like Turnitin are copy-paste checkers. They compare your work to existing works, which makes it easy for humans to track any influence or stolen ideas between strongly matched texts.
AI detectors rely on hidden statistical algorithms to assess a piece of text on its own terms, which means there’s no clear algorithm or logic for humans to follow. They might throw out some metric like “30% AI-generated” with a lot of confidence, but they really cannot say that any portion was definitely made by an AI. There are no rules being thoughtfully applied by experienced experts, just some unverified probabilities.
In simple terms, because it’s impossible to tell whether any piece of writing is stolen without a point of reference to compare it to, AI detectors are hoping they can rely on impressive-sounding statistics that you won’t question too deeply.
Punishes Good Habits
AI plagiarism detectors are basically just measuring how close to ChatGPT you might sound. The most common statistical measure is called perplexity. This means if you use quotes and evidence similar to the training data used for an AI, you have a higher chance of being flagged.
As a result, you actually get punished for using credible evidence that’s commonly cited because ChatGPT was probably heavily trained on it. The United States Constitution and the Bible both register as AI-generated to many of these detectors, so using their quotes in your original human-written work would raise the risk of being flagged as AI-generated.
While the same risk happens with Turnitin, it’s very easy for a human to look at a comparison between two texts and see that one is just quoting another rather than plagiarizing. Since AI systems don’t have this kind of transparency, they’re more likely to flag papers with reliable evidence or classic excerpts as being plagiarized.
Strangely, this also means you’re less likely to be flagged for plagiarism if your essay contains many grammatical and syntactical errors, since ChatGPT doesn’t often make those mistakes.
What to do if you’re being accused
Save all your process work
This is one of the most convincing ways to show that you did your own work. Keep your planning notes, your outlines, your rough drafts, and even any conversations about your work you’ve had with people around you.
Handwritten material is ideal, but if you work with Word and Google Docs, you might have a history of saved drafts with a date automatically available for you. Showing these intermediate steps to produce a final work is the clearest evidence that you didn’t rely on an AI to spit out a whole completed work.
Tell the Story of Your Writing Process
Remember that you’re asking a human to make a judgment call based on their personal feelings, so you should appeal to that subjectivity as well. Discuss your learning, researching, and writing process as a journey you took over time. Tell them about how your ideas changed over time and why. Tell them about obstacles or mental blocks you had and how you overcame them. Be sincere, of course, but building your case is still about delivering a story someone can believe.
Remind your teacher or professor about the flaws of AI detectors
If you know the exact product or service they use, you might be able to find critical reviews and examples of false positives that show how unreliable the AI detector is. Otherwise, feel free to use any of the resources available to you online — including this very article — to highlight how inaccurate and inexplicable methods for AI detection currently are.
Familiarize yourself with how plagiarism accusations are handled in your institution
If your teacher or professor doesn’t believe you, they might assign a failing grade that you want to appeal or they may escalate the situation to a higher authority themselves. Either way, you should look at how your specific school or college handles academic integrity violations and consider reaching out to others who have been falsely accused to see how they handled it.
In fact, your best chance of dealing with the issue once it has escalated to this point is to combine your efforts with other students to show there’s a systemic problem that unfairly blames individuals.
Avoid the Problem in the First Place
The more unique your individual voice and the more original your content, the less likely AI detectors are to catch it. Remember, these detectors are usually checking to see how similar your word choice is to ChatGPT’s writing style. If you can really personalize your connection with the material and your work, you’re more likely to produce something original that ChatGPT wouldn’t and AI detectors won’t flag.
As tutors, many of us offer essay review services that encourage independent critical thinking so that you can articulate your unique perspective. Feel free to check us out here!
