Co-Constructing AI Boundaries: Agency, Judgment, and Ethical Literacy in AI-Mediated Meaning-Making

What is this study about?

As generative AI becomes part of everyday reading, writing, research, and communication, future teachers face a question that goes beyond whether they should use it: What parts of the work are they willing to hand over to AI, and what parts do they need to keep for themselves?

This study looks at that decision-making process. Rather than asking whether students used AI well or poorly, I was interested in the boundaries they created while working with it.

Those boundaries are not entirely new. We negotiate them whenever we work with other people or use tools to help us think. We divide work according to expertise, rely on others for things we do not know, and use notebooks, calculators, search engines, and other tools to carry part of the cognitive load.

Generative AI changes the equation because it can now take on work that looks much more like interpretation, writing, synthesis, and judgment. That makes an old question newly important: Where do I draw the line?

What did students let the AI do, and what did they insist on doing themselves? Did they let it organize information but keep the interpretation? Did they accept the AI’s explanation of what their data meant, or question it? When it produced something technically correct but too generic, too neutral, or simply not what they intended, did they push back?

At its heart, the study is about how people divide the work between themselves and an AI system. Taking a look at the small moments when they decide, “You can help me with this part, but this part is still mine.”

How did I study it?

The study followed one class of 23 students completing a semester-long literacy ethnography. As part of the project, students were required to use Google’s NotebookLM while they collected information, reflected on what they were finding, and analyzed their data.

NotebookLM was chosen because students could load in their own materials and ask questions about those sources rather than searching across the open web. The goal was not to have AI do the analysis for them. It was meant to act more like a thinking partner. My hope was to have it help students organize what they had collected, notice patterns, test ideas, and question their assumptions.

In a more traditional version of the assignment, students might have talked through their ideas with a classmate. Here, NotebookLM filled some of that role, while the student remained responsible for deciding what the information meant.

Rather than looking only at the finished projects, I looked at the process. I collected and examined students’ prompts, the AI’s responses, the changes students made, and their reflections on what the AI gave them.

I was especially interested in moments of pushback. When did students correct the AI, limit what they wanted it to do, reject a response, or rewrite what it produced?

Those moments made human judgment visible. Sometimes the response was essentially, No, that’s not quite what I mean. Try again. Other times students narrowed the AI’s role, returned to the original sources, or rewrote its output to better reflect their own interpretation and purpose.

What did I find?

Two different patterns stood out in how students worked with AI.

Some students treated the AI more like a junior analyst. They let it organize information, identify patterns, and help them think through their data, but they stayed in charge of what the information meant. When the AI missed the point, flattened an idea, or took the work in the wrong direction, they corrected it, narrowed its role, rejected its response, or rewrote what it produced. I call this pattern the Orchestrator.

Other students handed more of the organizing and interpreting over to the AI. They were more likely to accept the structure or explanation the system offered and build from there, with less back-and-forth or revision. I call this pattern the Outsourcer.

These are not two types of students, and one is not simply “good at AI” while the other is “bad at AI.” They are two different ways of dividing the work between a person and an AI system.

What surprised me was how difficult that difference was to see in the finished projects. The final work could look very similar. The difference became visible only when I looked at the process: what students asked, how they responded to what the AI gave them, when they pushed back, and what they changed before the work was finished.

In other words, the final product did not always show me who was still making the important decisions. Looking at the process did make this a bit easier to make sense of what they were doing. This showed me the Trio: a student’s prompt, the system’s response, and the student’s reflection on it.

Why does this matter?

We often talk about keeping a “human in the loop” with AI as though simply having a person involved is enough. This study suggests that it is not.

Being a human in the loop means continuing to exercise judgment as the work unfolds. Exercising human agency by questioning the AI, correcting it, limiting what it is allowed to do, reframing its responses, and sometimes refusing what it gives you.

That makes human-in-the-loop less of a technical safeguard and more of a literacy practice. The important difference was not whether students used AI, or even how skilled they were with the tool. It was whether they continued to make the important decisions.

For teacher education, that means the goal should not simply be better prompting or stricter AI rules. It should be designing learning environments where judgment, friction, and refusal remain visible parts of the work. This also includes identifying where students learn when to use AI, when to push back, and when some parts of the work should remain human.

That is what I mean by designing loops worth living in.