The Reader Who Isn’t There

What a machine reader taught me about trust, and about being understood.

A student writes an essay, rewrites one sentence three times, and presses submit. Somewhere on the other side, someone reads it. Today, that someone can be a machine.

I research how large language models assess student essays: how closely their scores agree with those of human examiners, and what makes them more or less reliable. I came to the ABOAGORA Workshop Series thinking of this as a technical problem. Over the two workshop days, I began to see it as a problem of communication as well. These pages follow that change of mind, and are honest about the questions it left open.

What surprised me, and changed my thinking

The first day, with Emma Frans, asked how we build trust and keep our curiosity when fake news and AI-generated content fill our feeds. I had not expected that question to sit so close to my own work. One idea stayed with me: facts alone rarely change minds, and how we say something matters as much as what we say.

In my research, I treat trust as something to measure. I compare a model’s scores with human examiners’ scores and calculate kappa, a statistic that shows how far two readers agree beyond what chance alone would produce. It is a useful check on overconfidence, because language models can sound just as convincing when they are wrong as when they are right. But the workshop made me see that agreement is only half of trust. The other half is how a judgement reaches the person it is about. A score can be statistically sound and still be accepted without question, or received badly, depending on how it is explained. I had been counting agreement and neglecting communication.

My experiments point the same way. With the same model and the same essay, the score can change depending on how I ask: giving only the rubric (zero-shot), adding worked examples (few-shot), or having the model reason through each criterion (chain-of-thought). How we ask shapes what we get, for machines as well as for people.

Models also disagree with each other. Two systems can read the same essay and give it different scores, just as two human examiners sometimes do. It is tempting to treat that disagreement as noise to be averaged away. I think it is information. In my own work, I combine several models and give more weight to those that have agreed more closely with human examiners on a particular part of the writing, such as vocabulary or coherence. That is a technical choice, but it carries a lesson about communication too: when we explain a result, we should say how much evidence stands behind it, and where the readers differed.

The moment that has stayed with me

Before the first day, we were each asked to introduce ourselves and our work in a few sentences. I found it harder than I expected. My natural way of describing my research is a technical sentence: “I evaluate large language models for scoring student writing using prompting strategies and multi-model ensembles, measured by kappa agreement with human raters.” It is accurate, and it invites almost no one in.

In a room of people from very different fields, an accurate sentence is not the same as a sentence that can be heard. A better introduction, I realized, is a question: What happens when a machine becomes the reader of your writing? Everyone has written something and wondered how it would be understood. That question does not ask the listener to know anything about kappa or prompts. It asks them only to remember a time when they were read, and perhaps misread.

What I learned from fields unlike my own

The group came from many fields and career stages, and on the second day we turned to the journalists who tell science and art to wider audiences. Their talks, on what makes research or art worth a story and how to make science irresistible, changed how I think about my work. My instinct is to begin with the method. A story, I am learning, begins with a question the listener already carries. Behind terms like “zero-shot” and “kappa” are very familiar worries: Can I trust this judgement? Will I be understood? What happens when two readers disagree?

This is a different discipline from the one I was trained in. In a paper, the careful qualification is a virtue. In a story, the qualification has to be earned by first giving the reader a reason to care. I do not think the two are opposed, but I had rarely had to decide which comes first. For a researcher who wants her work to matter beyond a small circle, that order is a real choice, and I now notice that I make it every time I describe what I do.

What art and science could bring to each other

The wider theme, Mercury, the winged messenger, is about how information, language and meaning travel, and what happens to them along the way.

An essay makes a similar journey. An idea becomes a sentence; the sentence becomes an essay; the essay becomes a score; and the score becomes a judgement about a person. A machine in the middle of that chain does not only carry the message. It interprets it.

Each step in that chain is a place where meaning can be kept or lost, and each is also a place where someone makes a choice: which rubric to use, which examples to show, which words to put in the feedback. Those choices are easy to forget once a number appears on the screen.

What could the arts add? My kappa can tell me how closely a machine’s scores match a person’s. It cannot tell me whether the machine understood what the writer meant, or noticed a sentence that was unusual because it was original. Translators, poets, and editors spend careers learning to notice exactly that. Before essays, I worked on spelling correction for Urdu, a language with far fewer digital resources than English, and learned that language technologies do not serve all languages and writing styles equally well. I would like to work with people who read for meaning and not only for error, and ask what they see in a machine’s reading that my numbers miss. I do not yet know what that collaboration would look like, but I now think my research needs it.

I can imagine small, practical beginnings. A translator and I could read the same set of essays and compare what each of us notices with what a model reports. An editor might show me which of a model’s explanations read as helpful and which merely read as polished. A writer could tell me what it feels like to receive a score without a reason. None of this would replace measurement. It would help me decide what to measure, and for whom the measurement matters.

Questions I am left with

These questions are open, and I would rather leave them open than answer them too quickly.

  • When a model explains its score, how do we know the explanation reflects what it actually did, rather than a story that sounds right?
  • Who explains a score to the student, and in what words?
  • What would it mean to design assessment with writers in the room, and not only examiners and engineers?
  • What should a student be told about the machine that read their essay?
  • How do I describe my work so that someone outside my field wants to ask a second question?

The messenger has changed. The message is still human. Behind every essay is a person trying to say something, and our job is to make sure that person is still being heard.

Thank you to Emma Frans, and to Liisa Lalu, Silja Laine and Anna-Stina Hägglund for bringing us together, and to my fellow participants for the conversations.

Author

Romila Aziz is a postdoctoral researcher at the Turku Research Institute for Learning Analytics (TRILA), University of Turku. She studies how reliably large language models assess learner writing, how their judgements compare with those of human experts, and where they fall short. Her wider interests include multilingual language technology and AI for education. She holds a PhD in Computer Science with a specialization in artificial intelligence and has taught several university courses, including machine learning, natural language processing and data mining.

Aboagora
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.