Once Upon an Algorithm

The Commonwealth Club hosted a panel discussion between Nina Beguš (a Berkeley academic), Ted Chiang (the sci-fi author whose work became the film Arrival), and James Yu (an author and cofounder of Sudowrite: an AI writing assistant). The topic: how do LLMs affect the art of writing?

Nina literally wrote the book on this but it only comes out in the fall of next year! But she interviewed a wide array of authors and it seems like there’s quite a mixed bag of responses. This is reflected in the choice of Ted and James.

Ted takes the anti-AI camp: I didn’t realize that he actually coined the phrase “blurry jpeg of the web” (New Yorker). He argues that to create art one’s tools need degrees of freedom to be able to navigate the enormous possibility space. LLMs instead dilute the author’s intentions and project onto the necessarily more limited manifold captured by the model. Indeed art likely exists as the outliers from this manifold.

James is on the pro-AI camp. He makes the point that Ted’s argument mostly refers to whole story generation. This is a bit of a red herring. Actual usage is much broader and more bespoke: asking for review, tone analysis, or consistency checking. He gave a great demo of his sudowrite tool which did involve much more back and forth. He used the phrase “scraps of prose” to describe this process as human and AI iterate on small little sections.

I tend to lean more towards the Ted side of things. I don’t find AI text generation to be particularly useful. It elides fine distinctions and just sounds horrible. I could see some of James’ point though: using AIs for review or exploration could be useful so long as you exercise your own judgment in acting on the results.

That said it’s not clear there is going to be a choice in the matter. If you need a sequel as soon as you deliver the first book then you’re going to gaze longingly at AI tools. I’ve joked about this for years with my father that Clive Cussler independently created an LLMs to churn out his books for him but this could now literally become true¹. This is sort of a PED situation for the arts: even if you have objections to using AI you’re going to be at a output disadvantage relative to writers who are less fussy.

And much like PEDs I think we’re likely to be worse for it in more subtle ways. Ted put me on to an excellent distinction by Anna Rogers on “writing = nuisance” vs “writing = thinking”. Normalizing the use of AI in writing deprives us of the practice of thinking through our thoughts and expressing them clearly. This is essentially the entire point of me writing these words into the void so clearly I agree with this take. I think this holds true for coding as well: if you don’t practice writing out code yourself I suspect that your ability to think algorithmically will atrophy and your ability to evaluate the LLM’s output will correspondingly suffer.

I think there is a final point which none of the panel addresses: what should the audience make of AI writing? In particular what does it ]communicate to the reader? to me it doesn’t seem to communicate care and respect; slop is what you feed to pigs.

This shows up in professional contexts: it’s easy to joke about LLMs writing annual reviews but I am fairly confident it’s already happening. Nothing screams “management is not a priority” like giving someone a LLM review of their work. Part of the point of the exercise is to force the manager to think through the work and reflect on what it all means.

I’m fighting a colleague who continually uses AI to write a corporate blog post. It’s mostly right about the subject matter, but the style is so grating because I can tell it’s not being given the care and attention it would otherwise have received. It’s wearing the mask of authoritativeness without necessarily earning it.

I don’t know if I’d enjoy AI written fiction. Certainly there’s a lot of slop out there, but my favorite writers do take their time with it (judging from the delay between books). There’s something special about the style of Neal Stephenson, Matt Ruff, or Michael Lewis that I would hate to lose in the process.