As an antidote against generative artificial intelligence, there is the artificial intelligence detector, which can tell you if a content It was created by artificial intelligence to a greater or lesser extent.
From the boom ai, tools such as gpzero, writsonic, copyleaks or sapping were the Teacher support And other professionals to avoid being deceived.
Obviously because Use artificial intelligence to write test Or written works is a temptation in which many students or lawyers have fallen, something that is considered directly a trap.
Los Artificial intelligence detectors They are a necessity and they are used as much as the generative IA itself, which makes us ask: How reliable are they really?
Many professional researchers have studied this question, Testing the skills of artificial intelligence detectorsSo we have a concrete answer.
If you are a teacher or trust your IA detectorWe are sorry to disappoint you, because this type of tool is not as precise as it should and, of course, is full of errors.
The short answer is that no, An artificial intelligence detector does not work. It is not reliable to support any decision using the result of his analysis, since there is the possibility that it provides false positives in the detector.
Now, to understand the reason for this, it is necessary to know How IA detectors work At the moment. We will explain it to you as simple as possible.
The key is to understand how the detector works to
Since the generative IA has an entire method to be exactly the type of intelligence it is, the IA detector Follow a similar scheme.
While they count Ars TechnicaAll detectors go for the same recipe, with the occasional change to differentiate: they are Trained with millions of examples of writingAnd with this, they can identify certain patterns to determine if the text was created by an AI or by a human being.
Based on this, there are Techniques used by the detector ai To do your job. The two main ones are:
- Analysis of perplexity: Provides a measure of how much a fragment of text differs from the content used by the AI to be trained.
- The “without bags” analysis: which evaluates the variability in the dynamics of the written text, including the length of the sentences, the paragraph structures, etc.
Analysis of perplexity
Perplexity comes from the moment in which artificial intelligence writes a text. Of course, this will use the content provided To learn to generate similar things.
This means that if a text is Leaves the usual patterns to his understandingIt will have a higher perplexity rate and will indicate the text has actually been written by a human being.
This is based on the chaotic that a person can write, following his own style, added to the trend of generate phrases that are usually not too commonEspecially when it is in a creative process.
Las predictable and frequent phrases In human speech or writing they will always be read as content generated by artificial intelligence, yet we know They can also make peopleAbove all because they are formal texts in which technical language is required.
To show a button: in the example of Ars Technica it is evident The IA confused the writer of the Constitution of the United States With a robot and even the Bible.
Analysis without bags
The “Buresass”, on the other hand, focuses on more technical aspects of the text to establish whether or not it was generated by the AI. Your approach is to aim How much the writing style is variable which analyzes.
Normally, people’s writing is very volatile and spontaneous: we can have long and short phrases in the same paragraph. More adjectives in the first prayer and less in the second.
While More variable is a textMore “explosive” is considered by the artificial intelligence detector and therefore it will seem more human.
However, we also know Our writing is not only in this way. Returning to the theme of technical or formal style, a person can write with a coherent style and perfectly predictable.
Science was not yet: many studies reveal the lack of artificial intelligence detectors
As well as the Generative It was the study center of many curious academics, they also have many artificial intelligence detectors Scientific research They try to determine whether they are effective or not.
For example, a study published in the BMC In 2023, he decided The effectiveness of AI detectors is variablebeing a little more precise than others. There has been a general insence in determining whether a content has been generated by people or not, giving many false positives.
Another study of the Cornell University In 2023 Errors in the classification of artificial intelligence detectorsLike other types of vulnerability at the code level that could confuse these programs and make them indicate a text as done by humans when they were generated by artificial intelligence.
The panorama is even more worrying when it comes content made in languages other than English. This is what indicates This study of 2023, in which the investigators felt the Openii detector against texts made by English writers and other nationalities. In the end, they concluded that with non -native writers there was a greater margin of error.
In short…
There is still a lot of work in advance for artificial intelligence detectors. Currently, it is clear that They are not reliable. And although they try to get along with the progress of artificial intelligence, they continue to present a margin of error that costs the note or reputation to a student/academic for having thought badly, when their work was actually legitimate.
And, if we think a little out of the box, in the end, The content used by an artificial intelligence to be trained was made by humans. It is impossible that there are no confused traits with what creates an artificial generative intelligence that a person can generate, especially in the areas where the technician prevails.
What can you do not to rely on the detector ai
Before a world that progresses and will not be the same after the IA, the Evaluators techniques To verify that their students have really learned what they teach, they must be mandatory.
Himself MitAnd instead, it recommends several Strategies for teachers In order to avoid accidents in which they damage their students or benefit them during cheat.
What many think is Classical methods As exams written in real time, oral tests or homework written in the classroom, but the institute suggests that these are stressful methods and that limit the real student’s ability.
Instead, they suggest:
- Help understand better how to do a job according to one’s skills.
- Give more flexibility to studentsso that they include their interests and their style of doing things in any task or work.
- Deepen the understanding e Invite your reflection To ensure understanding.
- Use Cases or examples of real life This motivates the student to pass the test with his own skills to measure himself realistically.
Also the MIT It discourages the practices that include satanizing the use of the AIBecause it is better to focus on what is really: a tool that must be used responsible and the one with your help, it is possible to improve the understanding and learning of the class.
