How to Vet Marketers for True AI Fluency (and Why Most Hiring Processes Miss It)

[tldr]

TL;DR: How to vet marketers for real AI fluency

• Before assessing candidates, marketing leaders need enough AI fluency of their own to recognize real capability and set role-appropriate expectations.
• Strong AI-fluent marketers combine marketing expertise, systems thinking, sound judgment, and the ability to build useful workflows.
• A quality vetting process should evaluate depth of tool usage, workflow design, marketing judgment, decision-making, and the ability (and willingness) to keep learning as AI evolves.
• Go beyond interview questions with practical assessments, such as solving a real marketing problem, critiquing an existing AI workflow, and comparing self-assessed fluency with demonstrated ability.
• To assess our marketing talent’s AI fluency, Right Side Up combines AI-powered analysis with human judgment, drawing on onboarding responses, real-world experience, and a live technical screen.
• Through our AI Services, we deploy our most AI-fluent marketers to help teams identify the right opportunities—then build and/or implement the workflows that bring them to life.

[/tldr]

Marketers have been expected to use AI for a while now.

But marketing teams are increasingly being mandated—not just vaguely encouraged—to deliver better, faster outcomes with AI.

By now, most marketers have developed at least some AI skills, usually through thoughtful prompting or basic workflows that automate previously manual work. Yet a meaningful skill gap remains: Only 12% of senior marketers consider themselves fully AI-fluent.

For marketing leaders hiring an FTE, contractor, or agency partner, that creates a real challenge. Investors and executives expect every new hire to be an AI expert, but truly fluent candidates are still the exception, not the rule.

Marketing teams don’t screen for AI fluency effectively

What makes hiring even harder is that many marketers can talk the AI talk but struggle to turn that “knowledge” into real workflows, systems, or results. Yet we’ve found that most brands still rely on ineffective or outdated processes for gauging AI fluency.

To solve this issue, we’ve developed a dedicated AI screening process, which we apply to every marketer that joins our collective or internal team. This process was meticulously built, combining extensive research, conversations with marketing leaders, and input from our own subject-matter experts. 

Now, we want to share some of our findings with you. In this article, we’ll share what we’ve learned, explain how to evaluate AI skills more effectively, and provide a look at the framework we use ourselves.

Before assessing AI fluency, get these fundamentals right

Before you can properly start evaluating marketers for their AI skills, there are a number of prereqs you need to check off. This baseline includes having some AI ability of your own, staying updated on what marketing roles are most affected by AI, and understanding what AI fluency actually entails. 

Marketing leaders need to be good at AI too

It’s extremely hard for you to evaluate candidates on their AI knowledge if you’re a beginner yourself. Yet many brands only ask their junior members to be AI experts, while leaders are only expected to be “in the loop.”

We believe all senior leaders should be AI-proficient, if not AI-native. This means they should be past the stage of treating AI like a mere productivity enhancer and able to at least build and execute basic workflows that produce tangible outcomes. Without this baseline knowledge, it’s near impossible to truly understand someone else’s AI level.

[tldr]

Related article

How do advanced marketers actually use AI? We did our own research and have got some answers. Read our full report to learn what percentage of marketers are actually AI-fluent, what tools these AI-fluent marketers use and where they’re creating the most value.

[/tldr]

Fluency also matters for credibility. A CMO who isn’t using AI to make their own work more efficient can’t tell her marketing managers to do the same—nor can they establish credible metrics for what successful AI adoption looks like. 

Understand what marketing functions are most affected by AI

While every single role in marketing is being changed by AI in some capacity, leaders need to understand what functions are being affected the most, and how. These include:

• Marketing operations: These roles are running complex workflows, managing data pipelines, and stitching together martech. AI is already heavily changing how that work gets done. 
• Marketing analytics: This is another one to pay attention to. Tools and models that used to require a data science team and a six-figure budget can now be built in a fraction of the time at a fraction of the cost. We’ve seen this firsthand with Media Mix Modeling (MMM).
• Growth marketing: The pace of change in performance and creative automation and SEO/GEO means that teams who aren't building AI into their testing and iteration cycles are already falling behind.

Define AI fluency before you try to measure it

Even marketing leaders who are well versed in AI sometimes misinterpret what being “AI-fluent” encompasses. The key mistake: they think AI hard skills (e.g., what a pure AI developer would specialize in) make all the difference while undervaluing marketing subject-matter expertise. 

Our definition of an AI-fluent marketer

An AI-fluent marketer is first and foremost a strong marketer that has also developed the instinct to think in AI systems and ability to design the right workflow for the right use case. However, they can and will make the smart marketing calls when the model doesn’t. 

Put otherwise, they combine marketing expertise, systems thinking, and sound judgment to use AI effectively.

The difference between AI usage and AI fluency

‍

What a strong AI fluency vetting process should include

Let’s take a look at five key elements you need to judge candidates on when speaking to them.

Evaluate depth of tool usage, not breadth

This is an instance where a jack of all trades but master of none is not especially useful. A candidate may have tried dozens of tools, but if they cannot point to a meaningful workflow, outcome, or problem they solved, that breadth does not tell you much.

Look for candidates who can speak to what broke and how they fixed it, not just what they've launched. People who've only dabbled tend to have clean—but generic—success stories.

Keep in mind:

• Depth of real-world usage matters more than the number of tools.
• Look for production workflows.
• Understanding of foundational AI terms: open source vs frontier models, LLMs, harness & evals
• Ask about failures and iteration.

A key question to ask:

• What's the AI tool you know best? Walk me through how you've used it beyond the basics.

Evaluate AI concepts and technical foundations

AI fluency requires understanding the concepts underneath the tools, like the differences between types of model types, workflows, or development approaches. Strong candidates can weigh cost, speed, privacy, control, and performance when making decisions.

They shouldn’t be expected to be an AI engineer, but they do need enough technical knowledge to competently evaluate new platforms and adapt as the ecosystem changes.

A top candidate can:

• Distinguish between frontier and open-source models
• Explain concepts like agents, retrieval, structured outputs, skills, and projects
• Understand what evaluations and test harnesses are for

A key question to ask:

• How would you decide between using a frontier model, an open-source model, or a simpler automated workflow for a marketing use case?

Evaluate systems thinking

Strong candidates think beyond individual prompts or isolated tasks. Using their technical and tool knowledge, they can design entire workflows that connect people, processes, and AI—and produce predictable outcomes. They also understand where those systems can break and how to build in the right checks and safeguards.

A top candidate can:

• Describe complete workflows in great depth
• Identify bottlenecks and failure points
• Understand:
  
  • inputs
  • outputs
  • triggers
  • guardrails
  • QA
  • human review
  • evaluations & iterations

A key question to ask:

• Walk me through a marketing workflow you've designed (or redesigned) with AI. Where does AI fit, where do humans step in, and where are the quality checkpoints?

Evaluate marketing judgment

Marketing judgment has always mattered in hiring.

But, today, it’s not uncommon for the luster of AI ability to overshadow good old domain expertise—and it’s a trap all marketing leaders should be aware of. 

Remember: AI can produce convincing yet terrible marketing, so taste and judgment are arguably more important than ever. Think about how many times you’ve seen AI produce alluring copy that’s not on brand or recommendations that sound great on paper but don’t jive with your strategy.

Keep in mind:

• Candidates should tell you how they QA AI-generated work. The answer should be a specific process, not a generic answer about “reviewing it.”
• Strong marketers catch:
  
  • Weak positioning and poor messaging
  • Misleading analysis or flawed attribution
  • Off-brand content
  • Broken automation logic
  • Recommendations that don’t with channel, team, or business goals

A key question to ask:

• Tell me about a time you disagreed with an AI recommendation. Why did you override it, and what informed your decision?

Evaluate decision-making

In addition to what they built and how they built it, candidates should be able to tell you why they built something. This is the evaluation of how they put it all together—how they apply both marketing and AI skills to address important business problems. 

They should recall several situations where they can tell you:

• What business problem(s) they were trying to solve
• Why they chose a specific tool or workflow
• The tradeoffs behind their choices
• The outcomes they expected, in measurable terms

A key question to ask:

• Your CMO comes to you and says "We need to use AI more." How would you find and prioritize the right first use case—and when would you advise them not to build?

Evaluate change management and team adoption

A valuable and technically impressive workflow unfortunately loses all its value if no one uses it.

This matters even more for senior marketers and consultants. They need to think beyond the build itself and consider how different people will interact with the workflow and what support the team will need to use it consistently.

What to look for:

• Plans for training, documentation, and adoption
• Builds in feedback and continuous improvement
• Balances technical sophistication with usability

A key question to ask:

• Tell me about an AI workflow you introduced to a team. How did you drive adoption, and what did you change based on how people actually used it?

Evaluate learning velocity

The shelf life of any specific AI skill is short. Think about the possibilities and the tech stack that were available six (or even three) months ago compared to today. Huge difference, right? That is why you want someone who keeps learning without waiting to be told.

Look for marketers who:

• Continually experiment and test new workflows
• Refine existing systems without being attached to specific tools
• Regularly share learnings with you and the team

A key question to ask:

• What proactive measures do you take to keep up with all the changes happening in AI, and what’s one thing you’ve learned recently as a result?

Three types of assessments that further reveal AI fluency

You’ve identified the right evaluation criteria and a few of the questions to uncover them. But interviews never tell the whole story. To get a more complete view of their skills, it’s helpful to combine traditional interview questions with these three assessments.

Give them a real marketing problem

Bring up a role-relevant scenario (theoretical or real) and keep it open-ended. This forces the candidate to show their systems thinking and tool knowledge on the spot while also giving you valuable clues about their marketing mastery.

Example:

"You're launching a B2B SaaS product in four weeks with a $100K budget. Walk us through how you'd use AI throughout the launch."

Ask them to critique an existing AI workflow

This is another type of test designed to reveal the candidate’s ability to pair their AI and marketing knowledge to solve a problem. In some ways, it’s more revealing than building a workflow from scratch: it doesn’t just tell you whether they understand the technology, but also if they can spot unnecessary complexity, weak processes, or places where human oversight should be implemented.

Example:

"Here's an AI-assisted content production workflow. What would you improve? What would you remove? Where would you introduce more human review?"

Pair the live questions with a self-assessment

Before the live exercise, ask candidates to rate their fluency across a few dimensions. These can mirror your interview criteria (tool depth, workflow design, systems thinking, marketing judgment, learning velocity).

Then compare those responses with how they perform during the interview and practical exercises. 

This allows you to evaluate calibration: Strong candidates usually have a realistic understanding of both their strengths and limitations, while less experienced ones often overestimate their capabilities. 

What Right Side Up looks for when vetting AI-fluent marketers

Now let’s take a look at our own vetting process, and how we’re able to bring some of the most AI-savvy marketers on board. Just as we recommend hiring teams, we evaluate AI qualities through multiple sources of evidence rather than relying on résumés or interview performances alone. You’ll see some overlap with the core vetting concepts we’ve covered, but also a structured proficiency framework of our own. 

Principle 1: Marketing expertise comes first

The most important thing: We’re looking for senior marketers who are also AI fluent, not AI experts with some knowledge of marketing. 

We firmly believe that AI amplifies expertise, but doesn’t replace it. So while our vetting process now includes an important AI fluency component, it still very much places marketing knowledge at the center. Everyone we work with has those in spades: they understand their channels, audiences, and business problems first.

Principle 2: We evaluate applied capability

It’s easy for a candidate to mention a few certifications and buzzwords and pass off for an AI expert at first glance. That’s why, to fact-check these claims of expertise, we rigorously test for applied capability during our vetting process. We look for:

• Detailed, measurable outcomes they’ve generated
• Workflows they’ve built from start to finish
• Tools they’ve used—not just a list, but their knowledge of those tools (depth > breadth) and how they’ve chained and integrated those across systems

Principle 3: We combine human and AI evaluation

Evaluating AI fluency requires both scale and nuance. Conveniently, that calls for a combination of AI and human judgment. So we do just that for our own process—how meta is that?

We use AI to identify patterns at scale (e.g., profiles, work history, self-assessments, screening data) and human judgment to evaluate the context and reasoning behind those patterns, especially during our live screening.

Inside our AI fluency framework

Our structured framework assesses marketers across three distinct areas, combining self-reported experience, evidence from real work, and applied evaluation from a real human. 

These three complementary inputs help us get all the information we need to build a complete picture of a candidate’s AI abilities. 

• Onboarding survey: Provides an initial view into how candidates think about AI and whether they describe concrete, applied experience.
• Profile data: Validates that AI fluency shows up in real work by evaluating past engagements with RSU clients.
• Technical screening: The final piece of the puzzle. We heavily lean on human judgment to confirm whether candidates can apply AI effectively in real marketing scenarios and distinguish between good, bad, and unnecessary uses of AI.

‍

Our grading rubric

Once the assessment is complete, we combine the results into a proprietary AI fluency score.  That score places each marketer into one of four proficiency tiers: beginner, intermediate, proficient, or native. Here’s what each of those levels entail:

‍

AI fluency is becoming a baseline marketing skill

As marketing continues to evolve in the age of AI, fluency is quickly becoming a baseline skill, not a nice-to-have. 

A common mistake we see from brands is that they trust—or mandate—their junior team members to figure it out while senior leaders only need to “stay informed,” but it’s the wrong approach: Everyone on your marketing team needs to understand how to use these tools, full stop.

The teams that invest in AI-fluent marketers today will be better positioned to adapt as the technology—and the role of marketing itself—continues to evolve. That starts with a hiring process that can distinguish real AI fluency from basic AI usage with some buzzwords thrown in.

[tldr]

We’ve already done the vetting

Not ready to figure out who’s actually AI-fluent on your own? Let us give you a jump-start as you hone your own vetting skills.

Through our AI Services, we can connect you with RSU-vetted experts who combine deep marketing and AI expertise. They can help assess your current processes, identify where AI can create the biggest wins, and prioritize the highest-impact opportunities. 

From there, they can also build and/or implement one of our proven AI playbooks—repeatable, real-world workflows they've already built, tested, and refined—and train your team to make them part of your day-to-day operations.

Contact our team to find the right AI-fluent marketing expert or workflow for your business.

[/tldr]