The Problem with AI writing - And why it won't get better

The Premise

Artificial Intelligence (AI) is a technology that is becoming more and more prevalent in our daily lives. The ability that AI possesses makes the use of it very appealing, especially in daily use, when it is used to automate tasks, answer a quick question, or help write an email to one of your colleagues. Underneath this seemingly harmless use-case comes a harsh consequence: AI language is everywhere, making it hard to find an original thought, sentence or piece of writing. I bet you’ve seen numerous websites or received a lot of emails by now where you just know “This was definitely written by AI”. I bet that you also don’t enjoy reading it when you suspect it’s AI. Why is that? How do you know that it was written by AI? The grammar is fine and the facts are often right, meaning something else is missing. With a bigger focus on content creation and marketing within Meado, we've been thinking a lot of about AI writing and why it sucks!

Let’s first take a step back, since AI is a very broad term. In this article, the use of the word ”AI” refers to Generative Artificial Intelligence (GenAI). More specifically, this refers to the use of Large Language Models (LLMs) open to the public, such as ChatGPT from OpenAI or Claude from Anthropic.

These AI agents have revolutionised the way we write and communicate with each other, both in the workplace and even in research writing. This is widely accepted to be a good thing, as it even speeds up the process of writing itself, making us more efficient with our time.

Where the problem comes in

What is good writing? Writing is the art of conveying information through words while following the conventions of a language, including its grammar and punctuation. AI has been trained on an enormous amount of published writing, which in effect causes it to be extremely good at following the grammar and punctuation thereof. But this doesn’t tell us what good writing is. Good writing uses language to communicate an idea, experience, story, or emotion effectively to a reader. The word “effectively” in the previous sentence is doing a lot of heavy lifting, as the word “effective” has to be aimed at some sort of goal. A good book might be very effective at making you feel very strong emotions, but ineffective at providing us with an understanding of nature. In the same way, a good scientific article should extract the knowledge from an experiment and present it with precision. Every word should earn its place, and every sentence should carry usable information. The goal of a scientific article isn’t to make you feel sad or excited, but merely to present the information (although some articles do show bias and excitement, this is merely proof of the passion the author has in their field).

A model writes for a generic reader with no particular purpose, so it ends up effective at nothing in particular. This is where the problem lies. AI has been trained to focus more on writing “correct” and “efficient”, rather than elegant, clean and with purpose. This makes it very easy to spot AI when reading a piece of media, as it often uses sales-y language like
“It’s not X, it’s Y” or
“Not just X, but Y”.

These nuggets get introduced to the models in post-training, where models are tuned toward answers that the human evaluating it prefers. Post training causes these nuggets, while training itself causes the emptiness. Noticing these nuggets is also very important in today’s society. The use of the infamous em dash (“—”) has made good writers wary of their own punctuation, as they often get accused of using AI instead of writing the piece themselves. This has led some authors to stop using this punctuation character altogether, as doing so has the risk of letting people believe AI was used in the writing process, which could significantly damage both credibility and reputation.

Removing these language nuggets is not necessarily difficult, as companies could add instructions for the AI agent to sound more human-like, proofread and change these nuggets or even prompt more specifically.

Why it won’t get better

Suppose all of these language nuggets are removed tomorrow. Would this solve our problem of AI writing being so obvious to spot? No. The writing would still feel empty, as the problem with AI writing is not necessarily the way it is being trained. It is being trained on virtually every piece of media, ranging from poems to cliché movie dialogue. It is being trained on real data, meaning its responses (at least, in theory) should sound real. But this is the part we have to understand. Media, as it stands today, almost exclusively consists of final versions. Rough drafts of masterpieces aren’t posted to the public in most cases, and can’t be used to train the AI. Some drafts do exist publicly: Wikipedia edit histories, Git commits, forum posts, fan fiction. But even the “drafts” are usually work that someone decided was valuable enough to post. Posting on Wikipedia means it has gone through a rough draft, and only requires slight edits here and there on average.

The fact that training doesn’t have enough draft sources is a very significant issue, as the drafts are where a piece of writing finds its heart, direction, and where the wheat gets separated from the chaff. AI never learnt how to write creatively, as it was deprived of the process of creativity itself. The funny thing is, most humans learn to write by reading finished books too. However, we are trained from very early on how to write well using brainstorm sessions, first drafts, second drafts and final version.

Even though both humans and AI “train” by reading finished books, the difference is that nobody hands a child a finished novel and says, “Now write one”. The skill of writing is curated and developed as a child grows up. People write for different reasons, different audiences. Most importantly, we get the opportunity to write something bad first. Unfortunately that is the part that AI skipped. So when you read an email that is perfectly correct but feel empty, what you’re noticing is the absence of a decision. Nobody chose this specific word over that one, nobody had to cut a paragraph they loved because it didn’t fit. Let AI fix your commas if you want. But write the first, incoherent, messy version yourself, because that is where you find out what you actually think. That is the problem with AI writing — and why it won’t get better.