I recently demo-ed an AI agent at a Women in AI event in SF. I spoke to dozens of busy and driven professionals who said they felt ‘left behind’ on AI. What struck me was that it didn’t matter how much they were using AI. Heavy users said it too.
Meanwhile, I see job postings looking for candidates with “AI proficiency” and no one knows what that means.
That’s not a motivation problem. That’s a definition problem. We’ve never really agreed on what good AI literacy looks like.
That’s the question I’ve been sitting with: what does AI literacy actually look like? Here’s my attempt . I invite you to tear it apart.
AI literacy is not tool mastery
Let’s say you use Excel at work. Nothing much has changed in Excel over the past 2-3 decades. Yes, there may be an occasional optimization or more elegant way to do something but when you put Excel on your resume, a recruiter / hiring manager / peer would have a common understanding of what you can deliver.
Unfortunately, AI doesn’t lend the same luxury because capabilities change every 3 months. In mid-Jan, I proudly posted “Don’t let AI do math” and then Anthropic said “Hold my beer” and dropped updates that did a thorough analysis for a given dataset — something that would have taken me a solid 4 hours if I was completely productive.
So the bar for AI literacy can’t be tool mastery. The goalpost will keep moving because the tools keep moving. Understanding, on the other hand, compounds.
Four levels of AI literacy
In practice, people tend to progress through a few recognizable stages of AI use.
Level 1: Mastering the Conversation
Most people start here. This level involves using ChatGPT/Gemini/Claude or any out of box AI product for drafting emails, summarizing documents, rewriting text, brainstorming ideas and so on.Literacy at this level means knowing how to ask clearly, iterating prompts to get what you want, recognizing when output is generic versus actually good, and understanding that fluency doesn’t mean correctness. (AI can write something that sounds completely right and isn’t. That’s the trap.)
At this level, personal productivity increases but the back and forth with AI can be frustrating and time-consuming. So it’s hard to truly measure gains.
Level 2: Building the Personal System
This level involves taking the best practices from Level 1 and making AI a repeatable personal system rather than a one-off tool. You could also augment AI with more context (eg - if you are using AI for interview prep, you would feed AI your resume and profile to get tailored feedback). Commons tools here include Custom GPTs, prompt libraries, knowledge grounding. The key shift here is from using AI in its factory settings to shaping AI behavior to work for you. You’re designing instructions, not just prompts. The leverage is still personal, but it compounds when what you’ve built can be reused by others.
Level 3: Connecting the Dots
At this level, you go beyond your AI tool of choice and start connecting it to other tools and data sources. Think: AI that monitors LinkedIn for new jobs, proactively refines your resume and cover letter for each one, and sends you the artifacts by email — without you lifting a finger. Common no-code tools are Zapier, n8n, Make, More technical setups include Agentic workflows or Claude Code with MCPs. These are setups where one trigger sets off a chain of actions.
The key difference from Level 2: in Level 2, the data you feed AI is static — your resume, a document, a brief you chose. In Level 3, the data is live and AI has to keep up as conditions change.
When Level 3 works, the unlock is significant. Processes that once required your active attention run on their own, throughput scales, and work that used to take hours can happen in minutes. The bottleneck shifts from execution to judgment: deciding which decisions AI can handle autonomously and which still need a human in the loop.
What’s hard is that reliability and edge cases don’t announce themselves upfront. I’ve built things at this level and the part that takes the most time isn’t the initial build — it’s figuring out all the ways it breaks and deciding which ones you care about.
At an organizational level, the technical challenge is often the easier one. The harder problem is change management: who owns this when it breaks?
Level 4: Strategic Supervision
This is the vision AI companies are selling where the boundaries between roles shift. Example: AI writes journal entries to your ERP (in my field, companies like Campfire and Rillet are building this), handles customer support, proactively runs analysis before anyone asks for it.
This stage goes beyond workflow automation. You’re redesigning work itself i.e. rewriting job descriptions, rethinking org structure, deciding which decisions still need humans and which don’t.
Where value actually concentrates at this level isn’t the AI, it’s what the AI removes: legacy constraints, coordination overhead, decision latency. Teams that waited days for a report get answers in real time. Roles that existed to move information between systems become redundant.
Which is also why friction is highest here. It’s no longer a technology problem. Habits have to change, authority structures shift, governance and accountability have to be built from scratch, and people have to trust a system with decisions that used to belong to them. Technical fluency is necessary but nowhere near sufficient.
Much of the conversation around AI today focuses on levels one and two. Companies rolling out Copilots to everyone and then wondering why productivity numbers aren’t moving are experiencing exactly this: tool access isn’t the same as literacy.
The harder and more interesting questions show up in levels three and four, where AI begins interacting with workflows and decisions. At that stage, the advantage comes less from tool familiarity and more from systems thinking and organizational judgment.
What good AI literacy actually looks like
Across all four levels, it converges on the same things: understanding what AI can and can’t do, knowing when it will help and when it will quietly mislead you, being able to design around its weaknesses, and making better decisions about where to use it and where not to.
Not tool memorization. Not prompt cleverness. Not staying current on every release.
A definition that survives tool churn: the ability to predict when AI will help, predict when it will fail, and integrate it into real work. That’s it.
The reassuring part
Feeling behind is not necessarily evidence of deficiency. In an environment with genuine uncertainty, permanent fluency isn’t possible anyway. Literacy is a more realistic target … and it’s cumulative. It builds through repeated interaction, observing failures, watching systems break, refining mental models. Not through chasing every new announcement.
If this technology follows the trajectory of past general-purpose tools, literacy will increasingly mean less “knowing tools” and more “understanding systems.” That’s slower to build. But it’s what holds.
---
Andrew Ng offers a reasonably good AI Skill Assessment that provides tailored feedback for your AI use-case along with ways to overcome challenges and/or increase scale. Check it out here: https://skillbuilder.deeplearning.ai
P.S. If you’re one of the people who reached out — this is for you. Happy to go deeper on any of this.
This is a great breakdown of the levels of AI literacy! Such a breakdown is important. I am a software engineer and I draw a lot of parallels between what you are describing and the world of software engineering. I find that where the interesting problems lie is not in the lines of code you write (for tasks that are pretty well refined) but rather the more open ended task of understanding and designing systems.