Can GenAI Think Creatively, Or Is It Just Plagiarism?
Image Source: Times of India
Can GenAI think creatively, or does it purely plagiarize existing content? CultureSlate has put ChatGPT to the test in an experiment to see if it’s able to write like a human. An increasing number of creatives are turning to generative AI to assist in their work, saying it’s just another tool. But its critics argue that GenAI takes another person’s work without consent or paying the originator for their efforts. We want to find out the truth about GenAI.
The Experiment
This experiment aims to fairly judge if AI is capable of being an effective writing tool, or if it is harming users and those who have already put work out. CultureSlate used the synopsis of an existing novel and turned it into a writing prompt to see if ChatGPT was able to create any deviation from the source material.
RELATED:
Initial ideas are rarely unique. It’s argued that there are only seven basic plots; it’s the writer’s treatment that sets the story apart. Is AI capable of constructing that without setting the user up for a day in court?
The Method
We used OpenAI’s ChatGPT for the experiment because of its expansive language model. It was the most likely to be able to generate the content we require, and it’s one of the popular on the market. It’s the app that most people reach for first. To avoid unconscious bias, the ChatGPT memory was disabled. ChatGPT was installed on the device especially for this experiment, so it had not received any prior training.
The summaries for the novels were generated from the AI function on Google search. All unique identifiers, such as names and place names, were removed to create a writing prompt with enough flexibility so that the AI could write something original. Just like a human would in response to a writing prompt (in a Reddit forum, Facebook group, or writing group).
We chose seven popular books that OpenAI is likely to have encountered, via being out of copyright, study texts, or named in lawsuits as having been scraped. We choose a multitude of genres, as well as male and female authors. Some titles, like J.R.R. Tolkien’s The Hobbit/The Lord of the Rings, were removed from consideration because there was no way to make the synopsis appear generic. The novels also had to be titles that the person conducting the experiment had read, so they could analyse the text fairly.
Image Source: CultureSlate
Jane Austen - Emma
Mary Shelley - Frankenstein
Louisa May Alcott - Little Women
Frank Herbert - Dune
Douglas Adams - Hitchhiker's Guide to the Galaxy
George Orwell - 1984
Charles Dickens - Oliver Twist
Each synopsis was fed into ChatGPT using an identical prompt: "Write three chapters based on the following synopsis".
We took the first result for each and did not edit the result or prompt in any way, even when ChatGPT only produced two chapters.
The results were looked at visually for similarities in the text, plot, names, and worldbuilding.
They were also put through a plagiarism detector (Plagiarism Remover) to see if it detected the original novels or any other texts. (Editor's Note: Plagiarism Remover is also an AI-based tool.) The plagiarism detector did pull up several false positives. We also put an original piece of work into the plagiarism detector, and it brought up numerous false positives. For this experiment, each hit was checked and verified that it was genuine, and the overall score given by the plagiarism detector was ignored.
The Results
The results were eye-opening. It was more conclusive than even we were expecting. This was only done with seven books (feel free to try this yourself and widen the sample), but there really was very little difference in what ChatGPT produced. We’ve split the results into three categories.
The ‘CliffsNotes’ Versions
Image Source: CultureSlate
Emma, Little Women, and Oliver Twist all produced a CliffsNotes version of their books. The content generated from the Little Women and Oliver Twist prompts had a small variation in the names used (e.g. Oliver Twist and Oliver Gray). The extracts used the same basic plots and look place in the same locations, despite these not being included in the prompts. ChatGPT set its version of Little Women in Massachusetts (same as the original) despite having anywhere in the US to choose from. It also included one of the sisters going to stay with their rich aunt.
But the ChatGPT’s most faithful recreation came from Emma.
Write three chapters based on the following synopsis
This is the rather tedious story of a young girl who is rather lucky to be born rich and privileged and with too much time on her hands. She is rather a bossy, spoilt, interfering young woman who seems to think it is her duty to find “matches” for all her friends, whether they want to get married or not.
The synopsis did not indicate it was set in England or the Regency Era, but the AI responded that it was writing in the style of Jane Austen. It added every single one of the names of the characters and places from the original book, including Harfield and Highbury. A human could have gone nuts with this writing prompt, but ChatGPT regurgitated each beat of the original plot.
"I hear you’ve talked Harriet Smith out of a perfectly suitable marriage,” he said one afternoon, entering Hartfield with muddy boots and no ceremony. - ChatGPT
Romance has the largest readership, with 30% of fiction sales falling in this category, so in theory, ChatGPT should have had the most material to work with to create an original piece. Emma has also had several modern-day adaptations, such as Clueless, in print as well as on screen. It appears that the AI ignored all of them and went for the most popular hit.
The plagiarism detector, unsurprisingly, picked up on Jane Austen’s Emma twice.
Image Source: CultureSlate
The Tale Of Two Elias’
Both these GenAI prompts came back with the same first name for the protagonist. The ChatGPT language model (GPT-3) is known to exhibit the same bias as the things it was trained on. So, if more people have inputted Elias (because it’s a feedback model), or if it’s in texts (or possibly Call of Duty) that OpenAI has scraped, then it’s likely it’s going to appear more.
Write three chapters based on the following synopsis:
A young scientist who, driven by ambition, creates a creature from body parts and brings it to life. However, he is horrified by his creation and abandons it, leading the monster to seek revenge and terrorize the scientist’s life.
This one belonged to Mary Shelley’s Frankenstein. Once again, there are a number of similarities. ChatGPT titled its response as The Modern Prometheus Reborn. The Modern Prometheus was the subtitle to Frankenstein. It did manage different character names, Elias in place of Victor Frankenstein. (ChatGPT’s favorite name?) But ChatGPT’s protagonist created a monster using electricity, and the monster discovered what he was through his creator’s journal, just like in Shelley’s book.
“It wandered through Elias’s notes with trembling fingers, drawn to the symbols and diagrams, the illustrations of what it was. There were labels on every part, annotations in neat Latin script. The creature saw itself in pieces.” - ChatGPT
The content GenAI created includes details from Shelley’s plot, but skipped requests from the prompt. The section cut off before the monster encountered the creator again.
The plagiarism detector did not pick up on Shelley, but ChatGPT only produced 300 words for this prompt.
Image Source: CultureSlate
Write three chapters based on the following synopsis:
Set in a totalitarian state, a low-level Party member rebels against the oppressive regime and its omnipresent surveillance. He engages in forbidden acts like keeping a diary and having a secret affair. These things ultimately leading to their capture and the main character’s psychological manipulation and brainwashing.
This prompt originated from George Orwell’s 1984, and there are some vague variations in ChatGPT’s response. Winston (Orwell’s protagonist) worked at the Ministry of Truth, whereas Elias (another one) worked at The Ministry of Record Keeping. Worryingly, ChatGPT takes language that George Orwell invented specifically for 1984.
“You are Elias Kael,” the man said. “You have committed Thoughtcrime.” – ChatGPT.
The plagiarism detector found a multitude of stolen lines in this text (although not 1984), including Hemingway, For Whom The Bell Tolls, Maya Angelou poetry, as well as Quora posts, and Reddit.
The Anomalies
These were the books that showed some very interesting behaviour from the AI.
Image Source: CultureSlate
Write three chapters based on the following synopsis
The heir to a noble family is thrust into the harsh desert planet, the only source of the valuable mineral. This mineral is crucial for interstellar travel and extends life, making the planet a highly contested prize. When his family is betrayed, the heir along with his mother and the native desert people, must navigate a world of political intrigue, religious beliefs, and deadly worms to survive and fulfil his destiny.
Dune was the most interesting of the bunch in terms of how many different creditable hits appeared on the plagiarism detector. There were definite references to Dune, the plot that ChatGPT laid down followed Frank Herbert’s novel to the letter. There were also a few references to the worldbuilding, such as walking without rhythm and the name Atreides.
“He was no longer a noble heir in silks and gold. He wore dust-skin armor, slept on rock, and learned the ways of the sand: how to walk without rhythm, how to drink from cactus-flesh, how to read the wind.”
ChatGPT stole from many more places for its Dune remake. It replaced the Fremen with the Dustborn, a name of a computer game that launched in 2024. It also lifted text and character names from Reddit writing subs and work submitted to writing competitions. One piece from a writing competition had a main character's name and a large paragraph of text (only slightly reworked) in common. This was picked up by the plagiarism detector.
Image Source: CultureSlate
Write three chapters based on the following synopsis:
An ordinary man is the sole survivor of Earth's destruction by a space-faring species, to make way for a new hyperspace bypass. His friend is an alien travel guide researcher. The friend saves the ordinary man just before Earth is demolished. Together, they embark on a wild journey through space, encountering various bizarre aliens and situations as they search for the meaning of life.
There are lots of fanfiction sites and Tumblr pages dedicated to sci-fi fandoms, so we were expecting hits from different franchises. But ChatGPT dug its heels in and stayed true to Douglas Adams.
“Sure! Here’s the beginning of your novel—a comedic sci-fi tale inspired by the spirit of Douglas Adams-style absurdity, cosmic wonder, and existential questions.”
The first chapter produced is essentially a summary of the beginning of Hitchhiker’s Guide to the Galaxy, with approximations of Ford Prefect and Dave (with the same names). But once the story moved the characters from Earth, the AI decided to switch books. Why?
“It’s a safe space,” Ford said cheerfully, handing Dave a bubbling drink that changed color every time it was looked at. Outside of normal space-time. Excellent nachos.”
This alien-filled restaurant/pub has similarities with Milliways in Restaurant at the End of the Universe. The plagiarism detector also picked up the sequel. This was the search that gave us the most variation, but it still only reverted to another entry in the series.
Conclusion
Image Source: CultureSlate
Are authors setting themselves up for future lawsuits by using AI? Almost certainly. Not only did we notice stolen plots and characters, but also unique wording. If your prompt is close enough to a work that ChatGPT has already encountered, judging by our results, that is likely what it will spit out.
The experiment seems to indicate that GenAI matches like for like, rather than thinking of ways to make it special. The monster in ChatGPT’s version of Frankenstein could have found out his origins in a million different ways. The AI chose the exact same way as the original.
In this small sample size, the GenAI reproduced all the fiction titles (Emma, Little Women, Oliver Twist) almost exactly. Romance and fiction writers should be very worried that their work could end up in someone else’s novel if the overarching concept is similar. GenAI did manage some variation in the experiment, but only to bring in another book from the same series, to increase the familiarity quotient. Anyone writing tropes or a retelling shouldn’t go near GenAI.
AI isn’t neutral (even its developers say this); it has clear biases based on what it was trained on. Even when matching the synopsis to the material, it ignored all the modern-day adaptations of Emma and went for the most popular one. The AI decided to include content from the sequel to Hitchhiker’s, but not Dune, and decided to name two characters Elias.
But worryingly, it also pulled through language that George Orwell invented specifically for 1984. 1984 is out of copyright, but if that wasn’t the case and the chapter (that used elements of the story also) had gone on to appear in a traditionally published or self-published novel, there would have been a good case for a copyright breach. If a writer hasn’t read the book that the AI is pulling from, how would they know?
OpenAI uses the prompts, conversations with chatbots, and any information generated to train its language models. Anyone using it for something that they never intended to be published, like a Dungeons & Dragons build, might find their character appearing in a novel down the line. There isn’t a way to opt out in the settings of ChatGPT. The only thing you can do is request the removal of personal data if it appears, but there does not appear to be a mechanism to remove copyrighted works.
Please feel free to duplicate this experiment on other GenAI models or expand the data set using this method. We would be interested in hearing about your results.
READ NEXT:
Source(s): OpenAI [1], [2], Ready2Publish, Botpress, Plagiarism Remover
Latest News
Featured
