Stop Starting Over: Two Techniques for AI That Builds on Itself

If you’re like me, you often have long, winding conversations with AI chatbots, with one of us posing ideas and the other pushing back, refining my thoughts along the way. Just like long, meandering conversations with colleagues, these AI chats are often enjoyable and productive. But, whether with colleagues or AI, these conversations suffer from two problems. Sometimes late in the conversation, the quality begins to slip, primarily because of forgetting things that occurred early in the chat. You’ve probably experienced this in face-to-face chats with colleagues where you have that moment when you think, wait, didn’t we already talk about this? It’s also not unusual to start losing the thread of a conversation when it goes on for a long time.

In the AI world, this is called context rot, and it’s something I’ve written about before. Context rot is a phenomenon where AI responses degrade as conversations get longer. The large language models underlying AI chatbots have finite context windows; in other words, they can only remember so much within a conversation. As the conversation gets longer, earlier material gets compressed or effectively forgotten. The result is that your conversation seems to get worse over time. Early on, there are a lot of brilliant insights, and later the output gets mediocre because the AI can no longer hold the full thread in its working memory.

But there’s another memory problem with using AI, this time on the human side. If you’re a heavy user of AI, it’s likely that you’ve forgotten what you discussed in earlier sessions. The pattern often goes something like this: we want to use AI for something. We have the chat, we get what we need, and we move on. That conversation sits in a never-ending list of chats that you may or may not ever revisit. If you need to pick up where you left off, you either have to search or scroll through a long messy thread, or maybe just start over again from scratch, re-explaining what’s already been established.

Both context rot and losing track of chat sessions are real problems that create unnecessary friction when using AI. Fortunately, there are two relatively simple solutions that can address these problems: handoff documents and memos. Handoff documents summarize what’s occurred in a chat for the purpose of, as the name implies, handing off the task to either another chat session or sometimes an entirely different AI tool. Memos are simply ways to remember what you’ve talked about during a chat session, in much the same way a post-meeting memo would establish what was discussed and decided on during a meeting.

Both of these are related to a growing trend in AI context engineering. Context engineering is the emerging term for managing what information an AI has available when it generates a response. It’s a step beyond prompt engineering; where prompt engineering is about crafting a good question, context engineering is about shaping the entire informational environment the AI is working in.

Handoff Documents

A handoff document is exactly what it sounds like: a structured summary of a conversation that you can hand off to a new chat session (or a different AI tool entirely). Think of it like shift notes in a hospital. When a nurse ends their shift, they don’t just say “good luck” to the next person; they leave a detailed account of each patient’s status, what’s been done, what still needs attention, and any concerns. The incoming nurse can pick up seamlessly because the critical context has been preserved in a transferable format.

The same logic applies to AI conversations. When a conversation is getting long (or when you’ve reached a natural stopping point and know you’ll want to continue later), ask the AI to generate a handoff document. You can prompt something like: “Create a handoff document that summarizes our conversation so far. Include the key decisions we’ve made, open questions, the current state of whatever we’re working on, and anything a new session would need to know to continue productively.”

What you get back is a compact, structured summary that captures the intellectual state of the conversation without all the back-and-forth that got you there. When you’re ready to continue, you start a fresh chat, paste in the handoff document, and tell the AI to pick up where you left off. The new session starts with clean context and full awareness of what’s already been established.

This is useful in at least two situations. The first is the obvious one: continuing work across sessions when a conversation gets too long. Instead of fighting context rot, you just sidestep it. Start fresh with a clean context window and a good summary of prior work. The second situation is less obvious but potentially more valuable: moving work between tools. I regularly start research exploration in one tool and continue it in another. ChatGPT’s deep research might surface interesting sources, but Claude might be better for the kind of iterative, dialectical thinking I want to do with those sources. A handoff document makes that transition smooth. Without one, I’d have to re-explain everything from scratch; with one, I just paste it in and the new tool is up to speed.

Although you can get structured with your handoff documents, I’m typically more in favor of a simple path. I simply ask the AI chatbot to create a handoff document. Occasionally, I’ll ask it to be sure to include certain things like decisions made, the current state of the work, open questions and the like. But even just saying “write a handoff document” will likely yield decent results. That being said, I encourage you to scan the handoffs and edit them when necessary. The AI tool will probably create a decent handoff document, but it may leave out key facts or get some things wrong. So it’s important to spend a couple of minutes reviewing and if necessary editing the handoff document.

The handoff document serves another useful purpose. It can help you keep track of what you’ve been doing. If you save your handoff documents (and you should), you end up with a chronological record of a project’s evolution. Where did the research design stand last Tuesday? What had you decided about the framework before you changed direction on Thursday? Your handoff documents become a breadcrumb trail through your own thinking. This is especially valuable for longer projects where you might be working in bursts across days or weeks; it’s surprisingly easy to forget not just what you decided, but why you decided it. Most chatbots make it pretty easy to save the handoff documents in Google Drive, but the handoff documents are typically in markdown format, which means they can be used in virtually any text editor or word processor.

The handoff document solves the problem of continuity across chat sessions or AI tools. AI memos solve a different problem: managing the accumulation of insights across many conversations.

AI Memos

The idea of AI memos comes from a blend of two different research documentation techniques. The first comes from qualitative research, specifically grounded theory, which encourages researchers to document their thoughts in theoretical memos. These tend to be exploratory and serve the purpose of capturing fleeting ideas before they escape the researcher’s mind. The second technique is the lab note. Lab notes serve a slightly different purpose than theoretical memos. They’re more focused on creating a chronological record of what was done, what was observed, and what the researcher made of the interim results. They create an audit trail that allows the researcher or someone else to reconstruct not just what happened but why decisions were made. The AI memo is a blend of these two. The basic idea is to capture important moments as they occur, rather than relying on your memory. After all, that’s literally what memo (memorandum) means - a thing to be remembered

If you’re a heavy AI user, you’ve probably had the experience of knowing that you worked something out in a chat session weeks ago but not being able to find which chat it was in. I haven’t counted but it wouldn’t surprise me if I had hundreds of conversations sitting in my chat histories. Many of them contain genuinely valuable thinking that needs to be captured in a more enduring, easier to find, more organized form than a chat transcript.

This problem was driving me crazy until I remembered the idea of the theoretical memo from grounded theory. So I’ve started routinely asking AI tools to write theoretical memos to capture not just entire conversations but important parts of long conversations. Any time there’s some sort of an insight that I want to make sure I capture, I ask AI to write a theoretical memo. I then save that either in a note-taking app such as Bear or as a Google Doc. (If you’re a Mac user, I highly recommend Bear as a low-friction note-taking app.)

My AI memo workflow operates like this: whenever I think “that’s interesting” or I need to remember that or something similar, I just ask AI to write a theoretical memo. Sometimes I’ll specify exactly what I want in the memo, but often just asking it to write the theoretical memo is sufficient. Here’s a pro tip: always start the document name with “theoretical memo” or “lab note” (or whatever you want to use). Using a consistent naming scheme makes it much easier to find memos later on.

Here’s a prompt you can use to get started:

Create a memo that captures the key insights from this conversation. Focus on conclusions, decisions, and anything I’d want to reference later. Keep it concise. Include “AI Memo” and today’s date at the beginning of the document name.

Build the Habit

The details of exactly how you use and create handoff documents and AI memos aren’t as important as building the habit of creating them. AI is rapidly becoming more central to knowledge work, so it’s important to figure out how to make your AI use cumulative rather than episodic. Early on, many of us tended to use AI interactions as one-off conversations. That’s fine as far as they go, but it’s not enough to make AI truly useful for knowledge work. Treating every AI conversation as a fresh start simply won’t cut it in the future. Humans and AI both have limited, fallible memories. So it’s important to document your collaborative work with AI effectively. This doesn’t have to be complex. Simply getting in the habit of creating handoff documents and AI memos will go a long way. Start creating these documents today, and someday, perhaps far in the future, future you will thank you.

---

Want to continue this conversation? I’d love to hear your thoughts on how you’re using AI. Drop me a line at Craig@AIGoesToCollege.com. Be sure to check out the AI Goes to College podcast, which I co-host with Dr. Robert E. Crossler. It’s available at https://www.aigoestocollege.com/follow.

Looking for practical guidance on AI in higher education? I offer engaging workshops and talks—both remotely and in person—on using AI to enhance learning while preserving academic integrity. Email me to discuss bringing these insights to your institution, or feel free to share my contact information with your professional development team.