Google researchers studied how creative professional writers viewed AI as a writing tool.
For an experiment, the Google research team asked 13 professional writers from different areas of creative writing to write their stories using the AI writing program Wordcraft. They wrote texts of 1000 to 1500 words without any content instructions.
Wordcraft is a text editor with built-in AI writing assistance based on Google LaMDA’s dialog-optimized AI model. Wordcraft is specially designed for writing creative and fictional texts. The tool is not yet publicly available.
The same goes for LaMDA, which is considered to be one of the most powerful language models currently available. It received particular attention in the summer of 2022, when former Google engineer Blake Lemoine claimed the language system was sensitive. He fell in love with LaMDA’s ability to dialogue, which is based on what the interlocutor says and implies.
The following video shows a demo of Wordcraft to give you an idea. The demo is based on an older version of LaMDA.
AI is not yet replacing professional writers, say professional writers
The writing test lasted eight weeks. The team interviewed the authors in two 45-minute sessions, one at the start of the test to verify expectations and one afterward to ask questions about the experience.
Additionally, all participants kept an AI diary in which they recorded their experiences based on predetermined guiding questions. The researchers evaluated these diaries in detail along with the interviews.
The findings of the research team:
- None of the 13 professionals see their careers threatened by AI writing tools.
- They describe the greatest potential of technology as making creative writing easier, faster, and more fun for professionals and amateur writers alike.
- The language tool is also useful for brainstorming ideas and details in stories.
- AI language tool developers need to focus on the parts of writing that are the least fun and involve the target audience in the development.
Lack of consistency and boring suggestions
Study participants described LaMDA lack of narrative coherence as a hindrance to text generation – a problem that affects all major current language models. In currently known architectures, their attention is limited to short paragraphs, which makes them unsuitable for long, coherent texts with a clear dramaturgical sequence and recurring protagonists.
Participants also criticized boring suggestions without character and clear writing, as well as shortcomings in Wordcraft’s user interface. Researchers see criticism of boredom as a dilemma because large AI language models like LaMDA are deliberately limited in their generative abilities to avoid crossing ethical or moral boundaries.
“A system that always errs on the side of transgression avoidance prevents itself from achieving human-level creativity, which is often grounded in a rejection of tropes and norms,” the researchers write.
AI copyright issue also affects writing assistants
Three study participants also expressed concern about the source of wordcraft text suggestions unknown. If Wordcraft took text snippets as is from other novels, for example, it could damage the author’s image or even have legal consequences.
One author described Wordcraft suggestions from the worlds of Doctor Who and Avatar. The AI software also tried to include well-known characters like the Wizard of Oz and Megatron (Transformers) into the plot.
A possible solution to this problem could be for Wordcraft to use the Internet to check if a text is copyrightable. However, the researchers admit that the language models could mimic the work of other authors in more subtle ways than direct copying. Similar copyright discussions exist for images and for code.
One of the major successes of our study was to reveal the heterogeneous set of writers’ needs and desires for an AI assistant. Just as different writers need different roles for their beta readers and co-writers, we found that participants’ opinion of the appropriate role for AI varied widely.
For a detailed evaluation, see the article “Creative Writing with an AI-Powered Writing Assistant”.