Why evaluation could become one of the most important skills in AI engineering

There is a strange gap emerging in AI hiring.

Companies are spending heavily on engineers who can build AI systems. They are hiring people to develop agents, integrate models, build infrastructure and take AI products into production.

But there is a much smaller pool of people whose primary responsibility is working out whether those systems are actually any good.

That is becoming a problem.

A recent snapshot of 2,852 live AI roles found just 91 positions classified across AI safety, evaluation, red teaming and governance. That's around 3.2% of the market. Evaluation and red teaming accounted for 34 of those roles. (aihiringboard.com)

At first glance, that makes sense. Most companies are still trying to build their AI products. Evaluation sounds like something to worry about once the product exists.

I'm not sure that's going to hold.

The demo problem

AI has created a particularly difficult problem for engineering teams.

It is remarkably easy to make something impressive.

Give a capable model the right prompt, connect it to a few tools and you can have something that looks extraordinary in a demo.

The difficult part starts afterwards.

Does it work consistently?

Does it work with different users?

What happens when the input is ambiguous?

Does retrieval improve the answer or make it worse?

How often does the agent take the wrong action?

Does a new model actually improve performance?

What happens when the context gets much larger?

How much does it cost?

And perhaps the most difficult question of all: how do you know?

Traditional software gives engineers relatively clear ways of answering that question.

You write a test. The expected output is known. The test passes or fails.

AI systems are different.

An answer can be technically different and still be perfectly acceptable. An answer can look convincing and still be completely wrong.

That makes evaluation much closer to an engineering discipline than a final quality check.

The benchmark isn't your product

One of the traps companies can fall into is relying too heavily on public model benchmarks.

A model performs well on a benchmark, so it must be good for the application.

It doesn't necessarily follow.

A model might perform exceptionally well on a standard reasoning test but struggle with the very specific documents, terminology and workflows inside a particular business.

An agent might perform well in a controlled environment and fail when it has access to real tools.

A retrieval system might return technically relevant documents but still produce a poor customer experience.

The closer AI gets to real business processes, the less useful generic benchmarks become on their own.

Companies need their own evaluation frameworks.

And that means people who understand how to build them.

What an AI evaluation engineer actually does

The role is still being defined, which is part of what makes it interesting.

In some organisations, evaluation sits with ML engineers.

In others, it sits with applied AI teams, research teams or dedicated safety groups.

The work can include designing evaluation datasets, creating automated tests, building human-feedback systems, measuring model performance, testing agents against realistic scenarios and monitoring performance after deployment.

It can also involve something that sounds surprisingly simple but is incredibly difficult:

deciding what good looks like.

For a customer-service agent, that might mean resolution rate, accuracy and escalation behaviour.

For a coding agent, it could involve whether the software actually passes tests, whether the implementation introduces vulnerabilities and how often the engineer needs to intervene.

For an enterprise search system, it might be retrieval quality, factual accuracy and whether users can find the information they need.

There isn't one universal definition of success.

The evaluation system has to reflect the product.

This changes the engineering team

As AI systems become more capable, I think we're going to see evaluation move closer to the centre of engineering.

The traditional development cycle was relatively straightforward.

Build something.

Test it.

Deploy it.

Monitor it.

AI adds another layer because the behaviour itself can change depending on the model, context, prompt, tools, data and interaction.

You can make a change that improves one part of the system while making another part worse.

That creates a need for continuous evaluation.

It also creates a need for engineers who are comfortable working with uncertainty.

The best person for that job isn't necessarily a traditional QA engineer.

It might be an ML engineer who understands experimentation.

It might be a software engineer with strong testing and observability experience.

It might be someone from research who has moved into production.

Or it could be an engineer who has spent enough time building AI systems to understand exactly where they tend to fail.

The job title matters less than the combination of skills.

Why hiring for it is difficult

There is a fairly obvious chicken-and-egg problem.

There aren't many people with "AI Evaluation Engineer" on their CV because it hasn't been a standard role for very long.

That means companies searching exclusively for that title are going to miss a lot of relevant talent.

The better approach is to think about adjacent experience.

Someone who has built ML evaluation pipelines may have exactly the right foundations.

Someone who has worked extensively on automated testing and observability for AI products might be highly relevant.

A research engineer with experience designing model evaluations could transition into the role.

So could an experienced applied AI engineer who has spent considerable time measuring and improving production systems.

This is another example of why AI hiring is becoming less about matching job titles and more about understanding what people have actually done.

The commercial reason to care

There is also a much more practical reason companies should take evaluation seriously.

AI mistakes cost money.

If an internal coding agent produces poor code, engineers spend time fixing it.

If an enterprise search system gives unreliable answers, people stop using it.

If an autonomous agent takes the wrong action, the consequences can be considerably more serious.

And if a company cannot measure whether its AI product is improving, it becomes very difficult to know where to spend the next £1m of engineering investment.

Evaluation creates a feedback loop.

Build.

Measure.

Improve.

Measure again.

That loop is going to become increasingly important as AI moves from experimentation into core business processes.

The hiring signal we would look for

For companies building AI teams today, we think evaluation experience deserves much more attention than it currently receives.

It doesn't necessarily mean hiring a dedicated evaluation engineer tomorrow.

It does mean making sure someone on the team owns the problem.

When interviewing candidates, we'd be interested in people who can explain how they knew something they built was working.

Not simply that it worked in a demo.

How did they measure it?

What did they test?

What failed?

How did they identify the failure?

What changed after that?

Those answers can tell you a lot about how someone thinks as an engineer.

The next generation of AI teams

The first wave of AI hiring was largely about finding people who could build with the technology.

The next phase is going to be about finding people who can make those systems reliable.

That means evaluation, observability, testing, security and infrastructure are likely to become increasingly important parts of the AI engineering stack.

The interesting thing is that most companies don't necessarily need a new department for all of this.

They need the right capability somewhere in the team.

For some businesses, that will sit with an ML engineer.

For others, an applied AI engineer or platform engineer will own it.

And for companies running AI at significant scale, dedicated evaluation teams may eventually become as normal as dedicated infrastructure teams.

The technology is getting better at producing answers.

The engineering challenge is becoming knowing whether those answers deserve to be trusted.

That's a hiring problem we think more AI companies should be thinking about now.