GPTZero, a content detection tool for artificial intelligence (AI), was devised by Edward Tian, a student at Princeton University. Tian’s inspiration stemmed from the rise in instances of “AI plagiarism,” where AI tools generate content resembling existing information without appropriate citation. This tool was specifically tailored for educators and university lecturers to address this growing concern.

GPTZero, developed by Princeton University student Edward Tian, serves as an artificial intelligence content detection tool aimed at educators. Tian’s motivation stemmed from the surge in “AI plagiarism,” where AI-generated content lacks proper attribution. This tool addresses concerns surrounding AI content creation, including its potential misuse by students, particularly in exams.

Tian positions GPT Zero as a solution to this issue by analyzing texts based on perplexity and burstiness. Perplexity gauges a text’s randomness to a model, while burstiness assesses its uniformity over time compared to human-written text.

Currently in beta mode, the app intends to release a paper showcasing its accuracy stats derived from student journalism pieces.

As AI content creation becomes more prevalent, tools like GPT Zero are essential for regulating AI usage and upholding the integrity of student work. Exploring its features further can reveal whether it’s a viable solution to AI plagiarism.

‍Features

• Perplexity score: As mentioned earlier, perplexity is a randomness metric. It examines how good a language model (like ChatGPT) is at predicting the next words. Higher scores indicate unpredictability, implying that a text is human-made. Currently, GPT Zero bases perplexity on the GPT-2 model.
• Burstiness score: This metric looks at the distribution of sentences and determines whether there is uniformity or pattern. It’s based on the idea that humans tend to mix short and long sentences when writing.
• Highlighted AI-written portions: GPTZero’s latest update includes phrase detection. The app highlights which phrases are most likely written by an AI within a text. Additionally, GPT Zero will show you which sentence has the highest perplexity score.
• API integration: The creator made GPT Zero’s API readily available for anyone who wants to integrate it into their systems. You can also request customization support from the app’s engineers via email.

Pros

• Free to use similar to our Free AI Detector
• Can detect GPT-2 texts
• No sign-up needed

Cons

• Not as accurate according to this AI detection accuracy study
• Lacks features (Plagiarism Checker, Readability Checker, Fact Checker)

Testing the accuracy of GPTZero

To assess the accuracy of GPTZero, we conducted a test using seven text samples generated by Jasper.ai, a widely-used AI writing tool. In comparison, we’ll display the results from GPTZero alongside those from Originality.ai. However, it’s essential to first clarify how GPTZero presents its results.

Unlike many other AI content detection tools, GPTZero doesn’t utilize a probability format. Instead, it offers a general assertion regarding whether the text is likely generated by AI. Additionally, the app displays the perplexity score and burstiness score.

In contrast, Originality.ai presents its results in percentages, indicating the originality of the text and the portion likely produced by AI. Unlike GPTZero, Originality.ai is trained on multiple language models, including GPT-3, GPT-3.5, GPT-4, and ChatGPT.

For the sake of comparison, we’ll present GPTZero’s perplexity score and statements. Here are the findings.

Right off the bat, it’s apparent that GPTZero’s results are somewhat perplexing. While the concept of perplexity is clearly defined by the creator, interpreting the numbers provided can be challenging. We primarily rely on the statement that indicates the higher the perplexity, the more likely the text is human-generated.

From our experimentation, we observed perplexity scores reaching as high as 30,000. Alongside the general statements provided by GPTZero, this serves as the basis for our analysis.

GPTZero did not achieve 100% detection accuracy for any of the samples. The app would display the statement “Your text is likely to be written entirely by AI” if it did. In contrast, Originality.ai achieved 100% detection on samples #1 and #2, and detected over 90% on three additional samples.

However, there are positive aspects to GPTZero’s performance. It does highlight phrases that are likely generated by AI. Here’s an example:

In summary, while GPTZero can detect instances of AI-generated content, there’s room for improvement in how scores are presented. This refinement is crucial for educators to fully understand the significance of AI plagiarism within a text. In terms of accuracy and clarity, Originality.ai emerges as the superior choice in this evaluation.

GPTZero’s Potential

Given that the tool is currently in beta mode, it’s reasonable to refrain from emphasizing its drawbacks at this stage. Instead, let’s focus on the potential trajectory of GPTZero. Firstly, its current availability as a free tool renders it highly accessible to educators.

Moreover, the API is open for those who request access, allowing other developers to leverage and integrate it into their own systems.

From the outset, Edward Tian emphasized that the tool was developed with academics in mind, distinguishing it from many AI content detection tools primarily tailored for blog posts and marketing copy.

As the team continues to test GPTZero on more student work data, its accuracy is likely to improve. When combined with online plagiarism checkers, it holds promise for enhancing the integrity of student essays and theses.

However, it’s prudent for educators to use GPTZero judiciously during its developmental phase. As the app cautions users on its page, AI detection is just “one of the many pieces of a holistic assessment of student work.” Additionally, educators can utilize plagiarism detection tools that compare documents against vast repositories of public content online.

Conclusion

GPTZero currently offers a free and respectable option for identifying AI plagiarism. Nevertheless, it has ground to cover to match up to competitors such as Originality.ai AI Checker, which not only identifies AI-generated content but also detects plagiarism.

It’s crucial to acknowledge that GPTZero is still in beta mode. As the Princeton team dedicates efforts to further developing the app, we anticipate enhancements in both accuracy and features. In the interim, GPTZero’s metrics serve as a solid initial resource for teachers seeking to identify AI plagiarism.

See more news at here