After I published The Bottle Keeps Changing. The Work of Literacy Does Not., a reader pushed back on what happens to sourcing when an AI system produces the answer.
They described large language models (LLMs) as “illiteracy machines,” arguing that these systems obscure the origins, credit, and confirmation that evaluating information depends on. I really love that choice of words and it’s sticking with me. Those three things, where an answer came from, whose work made it possible, and whether it holds up, are exactly what gets harder to recover when a system generates the answer for us.
The comment brought me back to a question I had left underdeveloped, and had explored in Verified, Not True. There I called provenance a new literacy. What I want to sharpen here is how familiar questions about sources and credit change when the conditions for answering them shift. What happens when we know which questions to ask, but the system does not make available what we need to answer them?
AI did not create this problem.
For years, reading online has increasingly separated us from the sources behind what we encounter. Search results summarize pages before we visit them. Social platforms detach claims from their original contexts. Screenshots, reposts, aggregators, and recommendation feeds move information farther from the people and evidence that produced it.
We can know that provenance matters and still find ourselves working backward through an information environment that makes provenance difficult to recover.
LLMs push that problem considerably further. Instead of encountering a source and deciding what to make of it, we increasingly encounter a generated answer assembled from an enormous body of material whose relationship to that particular response may be difficult, even impossible, to reconstruct.
That changes the conditions under which literacy happens.
An example from my own work
My earlier post argued that we should carry what we know about literacy into encounters with new technologies. We have spent decades studying how people assess information, negotiate authority, make meaning, participate, and exercise power. That work gives us somewhere to begin with AI.
But bringing those questions with us does not guarantee that we can pursue them in familiar ways. The questions may persist while the conditions for answering them change.
I ran into this recently while trying to answer what sounded like a simple question. How much energy does it take to ask an AI a question? This came up over the summer as I was interviewed several times by news outlets trying to make sense of the data center debates. It also came up in workshops I was presenting to individuals.
In my research and teaching, I’m spending more time thinking about local AI and helping people, especially educators and students, think about running their own models. Locally and offline. This unpacks a huge engineering problem that covers the amount of power the machine consumes, writing and building in a way that AI can understand, and figuring out how to handle the outputs…hallucinations and all.
One of the questions I addressed head on was how much the small local models on Raspberry Pis used. We also tried to figure out how much power the local models used while running on powerful computers in the lab. We then tried to figure out how much electricity was being used when we sent a chat prompt off to an online service (e.g., ChatGPT, Claude, Gemini).
At first I thought I was looking for a number and a comparison. If I use my laptop locally, it’ll cost me A dollars, use B amount of electricity, grab C amount of my data, and give me D level of quality. If I send a chat to an online service, it’ll cost me W dollars, use X amount of electricity, involve Y amount of my data, and give me Z level of quality. What data would move to the service, what would persist, and what might be reused?
Actually finding that number was almost impossible. On my desk, I can see how much CPU and GPU I’m using. I can hear the fans spin up and the laptop get warm. I can use a power meter to monitor electricity.
When I use a hosted model like ChatGPT or NotebookLM, my computer barely notices. The work moved elsewhere. Somewhere in a data center there are accelerators doing the inference, host computers and memory supporting them, machines sitting ready for the next request, networking equipment moving data, and infrastructure cooling and operating the data center.
But trying to get that information about cost (electricity, money, tokens, data) was confusing to pin to a simple answer. And I wasn’t sure the different models (yes, I tried this query across multiple tools) were being fully honest with me. More precisely, I couldn’t tell whether they had access to the information needed to answer.
This simple question led to a ton of others. Does the model actually know enough about the system it is running on to tell me what this interaction costs? Does it know which hardware handled my request, how much electricity was used, what data moved where, or what the company counts in its own estimates? And even if it gives me a confident answer, how do I know whether it is reporting something it can actually know rather than generating a plausible explanation from what it has learned?
The work of literacy does not disappear when we reach the limits of what a system lets us know. Sometimes the literate act is recognizing that limit and adjusting what we are willing to trust, use, or claim.
That experience changes how I think about the “illiteracy machines” comment. Deciding what to trust, use, and claim requires separating questions we often treat as interchangeable.
Verification, Provenance, Credit
For me, part of AI literacy is recognizing that whenever I use an AI tool to research a topic, write a paper, create a podcast, or think through a problem, there are boundaries to what I can know about the answer it gives me. The challenge is not simply learning to ask better questions, engineer better prompts, use more tokens, or choose a more powerful model. Sometimes I can know exactly what I want to investigate and still reach a point where the information I need is no longer available. Better questions can reveal how much we cannot see.
When evaluation is taught mainly as checking credibility (is this truthful?) and relevance (is this useful?), questions about purpose, circulation, and power can fall out of view. Who created the content, why was it shared, and what systems brought it to me? AI raises these stakes and adds questions about the sourcing of ideas and physical materials, along with privacy, labor, cost, and infrastructure. A crucial first step is to focus on three questions we often collapse into one.
Verification, provenance, and credit are related, but they ask different things. Verification asks whether a claim is accurate and supported by evidence. Provenance asks where the answer came from and what can be traced about how it reached us. Credit asks whose ideas, research, and intellectual labor deserve acknowledgment.
Figure 1. Evidence, history, and acknowledgment require related but distinct investigations.
An AI response can satisfy one of these without the others. The energy question showed me another wrinkle. I might trace a number to a published estimate and check what that estimate includes, yet still have no evidence that it describes my particular request. Where the number came from, what was counted, and whether it applies here are separate questions. A sourced estimate is not necessarily a measurement of the interaction I just had. Elsewhere the gap falls differently. I may identify the scholars whose work established an idea without knowing whether their work shaped this particular response. I may even trace an answer back to documents supplied to the model while knowing very little about the broader body of material that made the synthesis possible.
Seeing the impossible
This is where I keep coming back to the reader’s word impossible. Impossible to determine what?
Some things are recoverable. If a system is working from a defined set of documents, I may be able to inspect those documents and see which passages support its claims. I may be able to follow citations, check quotations, and compare an answer with the evidence it names.
Other things may not be recoverable in the form I want. I may not be able to reconstruct every influence that shaped a generated response, or identify every piece of intellectual labor folded into a fluent paragraph. The system may offer no usable route back to the history of the ideas it is arranging.
A reader can understand all of this perfectly well and still reach a point where there is nowhere further to go. That is not, by itself, a failure of literacy.
The user’s next decision matters. They can seek independent evidence. They can qualify the claim. They can choose another tool. They can decide not to use the answer at all. Recognizing that a claim cannot be adequately traced, and adjusting how much weight we give it, is a literacy practice.
This connects to the boundary work I examine in my research with preservice teachers. Students negotiated what they would allow AI to contribute through correction, restriction, revision, and refusal. In one interaction, a student pushed back on a source-grounded summary because its neutral language flattened the urgency and political stakes of the material. Being able to trace the sources did not settle whether the interpretation was adequate.
The energy question brings me to a related decision. When I cannot establish what a number measures, I have to decide whether I can responsibly use it. The missing information marks a limit on what I can know. Boundary work happens in deciding what authority I will give the answer under those conditions.
Figure 2. Boundary work makes judgment visible through what we qualify, revise, constrain, or refuse. Those choices do not repair a missing record.
That matters, because much of the conversation about AI literacy (and technology in general) still places responsibility on the individual user. Check the sources. Verify the answer. Be skeptical. Use critical thinking. All of it is good advice. None of it guarantees that the information necessary for an investigation will be available. None of it places any responsibility on the developers of the tools, platforms, and models.
At some point, telling people to become better readers can obscure a different question. Do they have the means to enact these practices? Do they have enough data and information to actually investigate what they’re reading and perhaps creating?
This is why literacy has to include examining the conditions under which people are asked to read, judge, and participate. The design of the system matters, and the information it exposes matters. The institutions that decide what counts as sufficient transparency matter. Teaching people to recognize the limits of an answer does not relieve the institutions providing these systems of responsibility for those limits.
Ordinary writing has gaps in provenance, too. My references do not document every conversation, half-remembered idea, abandoned draft, or exchange with a colleague that shaped this post. Citations have never been complete histories of intellectual development. We should not romanticize the older system. Instead, we can acknowledge that it has imperfections, and that not every form of opacity is equivalent.
Generative systems can gather, compress, rearrange, and present enormous bodies of material in a seamless voice. That changes the scale of the provenance problem and makes the boundary between evidence, synthesis, inference, and invention unusually difficult to see. The answer arrives looking finished. The history behind it often does not.
If we keep the old wine in new bottles analogy from the first post, the bottle keeps changing. The bottle is also not neutral. It can change what we are able to inspect, which paths remain visible, and how far a reader can follow them. The methods, constraints, and distribution of responsibility all shift.
The work of literacy does not disappear. But sometimes the most important thing that work reveals is the edge of what the system lets us know.