How ChatGPT can change the conversation about summarizing and revising in scholarly writing

To begin, a caveat: I know almost nothing about Large Language Models (LLMs) in general or ChatGPT in particular. I will also admit that my immediate reaction is to want to hide my head in the sand, to fall back on the old humanist values of originality and that ineffable concept that often goes by the name of “soul.” Finally, I will not hide my pleasure at the fact that that GPT (Generative Pre-trained Transformer) pronounced in French sounds like “J’ai pété” (“I farted”). It won’t stop the spread of this kind of technology, but at least it gives me a measure of puerile amusement.

As someone whose primary job is to work with graduate students on their scholarly writing, the accessibility and power of ChatGPT (and other such bots) inevitably raises concern, but I agree that it is an opportunity to rethink assignments and how we assess knowledge, learning and critical thinking.

My discussion here is specifically about one aspect of scholarly writing that is both massively important and chronically undervalued: revision. I don’t mean editing, which is a relatively mechanical task, involving correcting small errors in spelling, word choice, punctuation and syntax. I actually think a bot could be very helpful as a copy-editor. I mean revision, the larger, structural and stylistic transformations that more or less gradually turn a messy draft into a coherent, cohesive piece of writing.

Writers often dread revision. I suspect that this is in part because many writers, and particularly graduate students, still think of writing in terms of product without recognizing the importance of process. The need to revise can feel like a problem or even a failure if what we have in mind is the perfect final product. In my workshops I try to change this view by framing revision as a necessary good, rather than a necessary evil (let alone a mark of failure to get it right the first time–an impossible goal with any writing that achieves something new).

Why do drafts of a dissertation chapter or journal article almost always need at least one or two rounds of significant revision? Because drafts are a form of pre-writing, of outlining. Don’t think of them as an attempt at the final product. Think of drafts as an outline in prose, a sketch of the chapter/article to be. This is why revision is necessary: revision takes the inevitably messy/shitty draft and re-shapes it into something closer to the final product, though it may need revisions of its own.

Here’s where ChatGPT comes back in. Until yesterday, I had thought of this technology as something that could write for you. What I had seen of ChatGPT’s outputs made me think the panic about it was far overblown. From what I could tell, the bot could write convincingly like a human, but not like a human who knows how to write. The best outputs I had seen were C+/B- level at a first-year university level, to be generous. For now, then, I didn’t see ChatGPT as a problem for graduate writers, who as a whole aim for something rather better than a B- at a Biology 101 level.

But yesterday I attended a discussion on ChatGPT and its implications for graduate education, and I was made aware of some of the other functions that actually complicate my response. I mean specifically its ability to summarize texts and to revise drafts. This made me more nervous, not because I felt that it threatens my job but because I can definitely see the appeal of these functions for many graduate researchers. I can imagine that a graduate student would want to use GPT to summarize an article they’re reading, or to summarize their own article into an abstract. I can also imagine a harried grad student wanting automated help in transforming a messy first draft into something a bit more shapely. As you can probably guess from what I said about revision’s role above, this is worrying.

First, summarizing. Summarizing is a basic skill for researchers who engage with huge amounts of readings, and it’s a difficult one. Summarizing is often also an unpleasant task–I know this from having to write abstracts summarizing my own articles, and really finding the work unexciting and frustrating. How tempting to get a bot to do it for you! But here’s the thing: summarizing is not a neutral task. Like translation, summary is an interpretation. When you summarize your own article–or someone else’s–you’re not just extracting its basic points: you’re choosing what counts as its basic points, for your purposes. As a purely linguistic tool, ChatGPT can’t make those decisions for you. In fact, YOU might not really know what those basic points are UNTIL YOU DO THE SUMMARIZING. In other words, revision is not just interpretation: it’s an act of meaning creation. ChatGPT can mimic creativity, but I don’t believe it can do it for real.

(For more on this line of thinking, you might be interested in this pre-print: Mahowald, Kyle, et al. “Dissociating language and thought in large language models: a cognitive perspective.” arXiv preprint arXiv:2301.06627 (2023).)

Now, revision. Just as I can imagine a grad student wanting automated help in summarizing something, I can easily picture the temptation to use ChatGPT to revise a draft. The reasons are similar: revision is a lot of work, and without a method it can feel not only slow but sometimes counterproductive. (There are plenty of very good methods of revision, including reverse outlining, abstracting, and sentence outlining–on which more soon.) But the tempting automation of revision concerns me even more than the automation of summary, though for similar reasons. Revision is an act of refining and organizing thought. It is the act of recognizing which half-formed ideas are worth foregrounding and which are worth eliminating, as well as the act of turning a mess into an argumentative progression that is often very inchoate in early drafts.

Think of revision as thinking. Or, think of revision as a kind of whetstone for your ideas, a sparring match in which you, the Reviser, parry with another self, the Drafter, pushing that former self to explain their ideas, to unpack implications, to order information in accordance with the significance of your contribution to the field, as a real reader would need that information to be ordered. ChatGPT can’t do that for you because it cannot know what that significance is; what’s more, YOU often don’t know what that significance is–or at least you don’t know it as well and clearly as you could–until you put yourself and your draft through the act of revision.

There’s a scene in The Matrix when Neo is plugged into a program and emerges with an amazed expression, “I know kung fu.” Wouldn’t that be nice? But would it work? Can you know something without rehearsal, without putting your body and/or mind through the moves, through trial and error?

Think of revision not as a form of correction, but rather as a step in an education. A messy first draft (a redundant expression because first drafts are always messy)… A messy first draft is not a failed performance: it is a rehearsal early in a sequence of rehearsals, in which revision plays a crucial role. ChatGPT can give you notes, I guess, but it can’t do the revision for you. It can perform a mimicry of revision, and that might be helpful. But it doesn’t do the work. I mean, it doesn’t think, or have ideas.

All the above is a specific version of a larger (and older) argument about the role of writing in scholarship. In some fields more than others, writing is often seen as a bothersome extra task, something separate from the research. The argument that your research doesn’t matter until it’s written/published doesn’t go far enough: that still implies that writing is an add-on. But writing is more than an add-on: writing is not just “writing up.” Even in experimental or quantitative fields, it includes activities we might not think of as writing, such as talking through the study with collaborators, supervisors, conference attendees; producing outlines; generating hypotheses and alternative hypotheses; outlining; experimental design; note-taking; and of course drafting, revising and revising again.

I understanding that in some fields, writing is more instrumental than it is in mine, where the research happens largely through the writing process. But even instrumental writing isn’t just an output. There is a feedback process, a recursivity that the acts of drafting and revising perform that are crucial not only to clarifying your ideas for readers, but also for generating those ideas for yourself. ChatGPT might seem like a convenient shortcut, but skipping the feedback process is not like cutting diagonally across and intersection to avoid crossing twice: it’s like going from A to C when C cannot truly exist without B.

“We write and revise our earliest drafts to discover and express what we mean,” as Joseph Williams puts it in Style: Ten Lessons in Clarity and Grace. If a bot is revising for us (let alone writing for us), we are not merely skipping a burdensome task on the way from draft to manuscript: we are forfeiting the creation of knowledge and clarity that emerge from the messy task of revising.

No doubt I’ll have more to say about this in the coming months and years. In fact, I have no trouble admitting that this is an unrevised first draft (except for the belated addition of the Williams quotation above), undoubtedly still largely unformed as an argument. It’s fine for a post on a blog no one reads, but I would certainly revise it (and show it to other readers for comment) before submitting it to another platform.

For now, I simply wanted to write about this. ChatGPT is an opportunity here: it makes it even more apparent that we need to rethink how we present writing and revising to our students, especially our graduate students. Many graduate students are never explicitly told anything about writing and especially not about revising (except that it needs to be done), and I think the massive task of supporting our graduate students has always included the need to clarify the role of revision in the process. ChatGPT’s ability to perform an imitation of revision makes that old problem new again.

I wonder if there’d be interest in a workshop “What ChatGPT teaches us about the importance of revision”?

Work cited

Joseph M. Williams. Style: Ten Lessons in Clarity and Grace, 6th ed. Longman, 2000.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s