Your AI agent works. But is it ready for production?

Written By: Yehuda King, Head of Artificial Intelligence Solutions & Operational Excellence

Your AI agent works. But is it ready for production?

Why Agent DevOps, governance and visibility matter as agents move from experiments into business operations.

Getting an AI agent to complete a task is an important milestone. It is not the same as making that agent stable or reliable enough for everyday business use.

Once an agent can access information, execute code or interact with business systems, governance becomes an operational requirement.

Who can modify it? What can it access? How are changes tested? Can you investigate an unexpected action? And what happens when an update goes wrong?

This is where Agent DevOps matters: the practices used to develop, test, release and operate agents with appropriate control.

Small changes can have significant consequences

An agent’s behaviour depends on more than its model. Instructions, knowledge, tools, Skills and permissions all shape what it can do.

A seemingly minor instruction change can alter how it interprets a request. Adding an integration or connection can expand its access and reach. Updating a reusable Skill can change how a business task is performed.

These changes deserve a managed lifecycle.

Separate draft, test and production environments allow teams to experiment without immediately affecting live Users. Version history and review processes make changes visible and establish responsibility for approving them.

Testing should cover more than the successful path. It should examine ambiguous requests, failures, inappropriate access attempts and potentially malicious content.

Security starts with boundaries, not promises

An instruction telling an agent to “only access approved systems” is not a substitute for enforcing that restriction.

Effective controls sit around the agent:

• Least-privilege access: provide only the permissions needed for the task

• Credential management: keep passwords or secrets out of conversations and avoid exposing them to running code where unnecessary

• Network restrictions: limit which external destinations the execution environment can reach

• Human approval: requires review for significant changes and actions with material business consequences

  
  

"User-delegated access and shared application credentials also have different implications. An agent acting with an individual’s permissions is not the same as one using a shared identity across Users." - Roelof Schoeman, CTO of Amplify

These controls can limit the impact of mistakes or malicious inputs. They do not eliminate every risk, but they reduce dependence on the model making the right decision every time.

If you cannot investigate it, you cannot govern it

When an agent behaves unexpectedly, knowing what it said is only part of the story.

Teams may also need to establish which version was running, which tools or Skills were used, what external actions were attempted and whether those actions succeeded.

Version history explains what changed. Operational logs help explain what happened. 

Both matter, but they serve different purposes.

Logging itself needs governance. Records can contain sensitive information, so access, retention and redaction should be deliberate. Credentials should not be captured in logs.

The objective is useful evidence, not indiscriminate collection.

Reusing Skills helps, but reusable does not mean risk-free

Skills capture an established workflow so an agent can reuse a defined approach rather than work out how to perform the task from the beginning each time.

This can reduce repeated development work and make capabilities easier to inspect, test and maintain. It may also reduce unnecessary model work, depending on the implementation.

But a reusable Skill can still contain errors or inappropriate permissions. It needs ownership, testing and change control, just like any other executable capability.

Recovery needs more than a rollback button

Restoring an earlier agent version can help recover from a problematic update. It does not necessarily undo actions already taken in external systems.

An email sent, a record changed or a transaction submitted may require separate remediation.

A recovery plan should therefore address both the agent’s configuration and the consequences of its actions.

The bottom line

Agent DevOps is not about slowing innovation. It is about making experimentation safer and successful capabilities easier to operate at scale.

The important question is no longer simply:

“Can this agent do the task?”

It is:

“Can we understand, control and support how it does the task, including when something goes wrong?”

That is the difference between an impressive feature demonstration and a dependable business capability.