How to make an LLM write like someone else

A work in progress attempt to escape the LLM slop basin

LLMs write poorly

By default, all available LLMs write in an off-putting “slop” style. They overuse cliches, prioritize (bad) style over substance, repeat themselves, and use characteristic LLM-giveaway phrases like “it’s not <X>—it’s <Y>”.

LLMs have seen millions of pages of high-quality writing in training. So why aren’t they able to produce similarly high-quality writing? AI models are better than most CS graduates at Codeforces and better than most math graduates at IMO problems. But when it comes to creative writing, they sound like an overenthusiastic middle-schooler trying to tick all the boxes on the essay scoring sheet.

Perhaps addressing this failure to write well is a matter of “unhobbling”. We do a lot of post-training (supervised learning on curated examples, and increasingly RL) to elicit SOTA models’ math and coding abilities. But we don’t spend similar amounts of effort and compute on creative writing. Understandably so—it’s unlikely to be the same kind of cash cow as software engineering.

However, it’s possible that the amount of post-training needed to elicit high-quality writing is much smaller than needed for solving math problems. High-quality writing is much more abundant on the internet than high-quality solutions to IMO problems. But of course there’s also a large amount of terrible writing and noise. So maybe it’s just a matter of successfully eliciting this “good writing” distribution (and not a matter of teaching the model anything new about the distribution).

Chat LLMs already undergo instruction-following and persona training, to make them function as chatbots with a certain personality. This training is probably what’s responsible for their characteristic writing style—we have elicited the style of “helpful AI assistant”. But if it wouldn’t take much compute to improve the model’s writing ability, it should be (at least partially) doable via in-context conditioning, with the right prompt.

So I decided to investigate this: can we make LLMs write in a different, improved, style just by using a long prompt?

The idea

I came up with the following pipeline to generate a few-shot prompt:

1. Select 5+ pieces in the target style and convert them to markdown, removing any artifacts like images and links that an LLM could not have outputted.

2. Feed these to an LLM and ask it to generate a description of the texts’ style (this will be used in the system prompt.)

3. Separately, feed every text one by one into the LLM and ask it to generate a prompt that could have been used to write that text. (I say something like “Generate a short prompt that could be used by another author to write a similar piece. The prompt should be concise and capture the essence of the original text.”)

4. Construct a human-assistant conversation history with the generated prompts as the “user” messages and the curated texts as the “assistant” responses.

5. Use the conversation history, alongside a system prompt that incorporates the style description from step (2), as the prefix to your final prompt that asks the model to write whatever you would like it to write.

This is implemented in a single short Python file here.

The main principle behind the approach is trying to get the model to learn (in-context) that the assistant responds in the characteristic style of whatever texts you selected (similar to how with many-shot jailbreaks, the model learns that the assistant agrees to harmful requests). Because it’s much easier to create prompts from existing texts than new example texts from prompts, we can trivially construct a long conversation history demonstrating the kind of writing we would like as long as we have access to writing samples. The system prompt part is an extra step so that we also tap into the model’s instruction-following capability by specifying explicitly what kind of writing we want.

In the system prompt, I also augment the extracted style instructions with a constant postfix of:

It is crucial that the Assistant follows the style described above and avoids stereotypical, generic, or cliche language. Things to avoid:
- Cliches like 'it's not X, it's Y'
- Repeating the same concepts in different words
- Generic phrases
- Prioritizing style over substance

Results

As a simple starting point, I’ve been testing this prompting approach with Scott Alexander’s “Every Bay Area House Party” series. I think it emulates the style of Scott’s posts pretty well. Here is an example. Here is another example. Using the same Bay Area House Party writing samples, I prompted the model to write a similar satirical piece about a fictional AI company all hands meeting. This is the result. Though personally I found these pieces humorous, and think they successfully reflect aspects of Scott’s style, the humor is indeed simplistic and there are still too many cliches.

I think there’s still some refinement of the scaffolding prompts to do, and I’m testing the approach with more authors and styles.

Do you have favorite LLM prompting approaches to elicit better and more diverse writing styles? Any writers I should try to emulate next?