AI Supervision: A Meta-Skill for the Next Decade

The Rise of AI Agents

AI agents are coming. I’ve written about this before and I remain convinced that the future of AI isn’t in chatbots (although they will remain important) but in networks of AI agents working together to accomplish complex tasks. It may be a few years away, but this future IS coming and higher education should be ready for it. I’ll admit, I’m out on the ledge (or the edge) here, but I’m confident I’m right.

What are AI agents? Briefly, they are artificial intelligence tools that can:

• Perceive their environment

• Reason, and

• Take independent or collaborative action to achieve specific goals.

I wrote about the importance of AI agents a few months ago. Here’s how I described them in that article:

AI agents are like digital robots that perform tasks for you, such as arranging trips, scheduling appointments, responding to emails, ordering groceries … just about anything you do digitally. I envision vast networks of these agents working together to perform complex tasks on behalf of users. This idea isn’t new; one of my first conference papers was about intelligent agents (in the 1990s!). We’ve had AI agents for some time; think of Alexa, Siri, or Google Assistant. These have their place, but they’re pretty limited in what they can do. The pieces necessary to vastly expand the capabilities are falling into place.

As I noted, the idea of agents has been around a long time. Early intelligent agents were largely experimental, but today all of the pieces required to make AI agents a daily reality are falling into place.

The Changing Nature of Work

Let’s switch gears for a minute. I promise, this is relevant. Over the last few months, I’ve been having numerous conversations about the future of work. Many of these have focused on the changing nature of entry-level jobs. I teach in a business school, so this is a huge deal to me. There’s a real possibility that AI will eliminate the need for many entry-level knowledge work jobs. The sorts of work new professionals do overlaps with AI’s capabilities. For example, some of our graduates start their careers on IT help desks, helping other workers solve their technology problems. Within the near future, AI chatbots will be capable of doing this work (in fact, this is already happening). As AI’s capabilities continue to evolve, more and more jobs will be threatened. This is not an isolated problem, with a little thought, we could come up with similar examples in virtually every field.

We (higher ed professionals) need to start thinking about this NOW. Yes, I realize that education isn’t just about employment, but it’s certainly a major reason why students go to college. If you accept that premise, you can see why we need to be prepared for some big changes.

OK, that’s a long preamble to my main point. I think we need to start teaching AI supervision as a sort of meta-skill along the same lines as critical thinking and communication. That’s a bold statement, I know, and I’m not necessarily arguing that AI supervision is as important as other meta skills, but I am stating emphatically that we need to consider AI supervision seriously. AI supervision goes beyond AI literacy by integrating that literacy with ethical judgment, analytical assessment, and effective management of AI agents enabling individuals to not only understand AI but also to thoughtfully and effectively shape its use and impact.

AI Supervision: The Four Core Skills

What do I mean by AI supervision? That’s a good question, one that I’m still pondering. Here’s what I have so far. AI supervision is made up of four subskills:

• AI Management

• AI Ethics and Governance (Reflective subskill)

• AI Evaluation and Assessment (Analytical subskill)

• AI Communication and Explanation (Interpretive subskill)

AI management is an operational subskill that involves planning, organizing, coordinating, monitoring and controlling teams of AI agents. This parallels human management. It’s putting agents together in a way that’s well coordinated, effective, and efficient. There’s a lot to this, hiring and firing agents, structuring agent teams, monitoring and evaluating agent performance, allocating resources to agents … the list is long. If a manager of humans does it, there’s probably a parallel in AI management.

AI ethics and governance is more of a reflective subskill that involves thoughtfully considering and dealing with AI-related ethical issues and accountability structures. At lower levels, the governance aspect of this will be focused on implementing governance measures more than developing these, but AI supervisors, like human supervisors, will be responsible for making sure policies and practices are followed by the agents. Ethical judgment and decision-making will be a critical aspect of this subskill. This is tricky with humans and with AI agents.

AI evaluation and assessment is an important analytical subskill. This goes beyond the sort of operational evaluation that goes on as part of AI management to systematically consider and analyze deeper issues and broader considerations, such as strategic alignment, broad organizational impacts, systemic risk and long term value. This subskill is directed at ensuring that AI agent teams are not only efficient, but also coherent and ethically sound.

AI communication and explanation allows an AI supervisor to be the bridge between the AI agents and other relevant parties, such as higher levels of management and other AI agent teams. This involves effectively communicating about AI processes and decisions to stakeholders, explaining AI outputs, and facilitating understanding between human and AI agents. It's about bridging the gap between human and artificial intelligence in ways that enhance both.

Some might argue that AI supervision is yet another technical skill and not a true meta-skill such as critical thinking and communication. I disagree. Meta-skills are fundamental capabilities that transcend specific contexts, situations, and tasks, prompting adaptability, insight and continual learning. Meta-skills help you navigate complexity and change. To me, AI supervision fits this bill.

Think about it this way. Just as critical thinking isn’t about memorizing logical fallacies but about developing systematic ways to evaluate information, AI agent supervision isn’t about learning specific AI techniques but about developing frameworks and methods for effectively directing AI agent resources. These skills will remain valuable, adaptable, and relevant even as AI tools and applications evolve.

What Now?

The emergence of AI agent supervision as a meta-skill has significant implications for higher education. Regardless of the discipline, the nature of entry-level jobs WILL change. But this isn’t just about preparing students for future jobs, it’s about ensuring that they can help thoughtfully shape and direct AI’s impact on their professions and society. That’s a lofty goal, without a doubt, but we need to rise to the challenge.

We can start small:

• Have students use multiple AI tools to complete a project or task, then reflect on how they managed and coordinated these tools.

• Include AI ethics in discussions of professional ethics.

• Ask students to document and explain their AI use, which will equip them to better understand and communicate how they use AI.

These modest steps can help lay the groundwork for more comprehensive AI supervision education as AI agents continue to evolve. The goal isn't to become AI experts overnight, but to start preparing for a future where AI supervision will be a fundamental skill.

More than anything else, we need to start thinking and talking about AI supervision as an emerging meta-skill. I’ll say it again. I am confident that AI supervision is going to be critical to many jobs, perhaps sooner rather than later.

Higher ed has a rare opportunity to get ahead of a looming change, for the benefit of our students and society. Let’s get started.

---

Want to continue this conversation? I'd love to hear your thoughts on how you're using AI to develop critical thinking skills in your courses. Drop me a line at Craig@AIGoesToCollege.com. Be sure to check out the AI Goes to College podcast, which I co-host with Dr. Robert E. Crossler. It's available at https://www.aigoestocollege.com/follow. Looking for practical guidance on AI in higher education? I offer engaging workshops and talks—both remotely and in person—on using AI to enhance learning while preserving academic integrity. Email me to discuss bringing these insights to your institution, or feel free to share my contact information with your professional development team.