Agents that audit a CRM and show their evidence
An agent that reads a CRM's configuration will confidently tell you what is broken. A large share of the time it is wrong, and a second agent reviewing the same configuration will agree with it.
- The goal
- Audit and operate live client CRMs at a depth that would normally need a team, without ever letting a model make an irreversible change on its own judgement.
- What it was costing
- An agent that reads configuration will confidently tell you what is broken, and a substantial share of the time it is wrong. Acting on those findings means changing a system a business runs on.
Built and operated by Pietro Sassanelli. Published with permission. No client portal, client name or portal identifier appears here or in any published material.
- Tool modules
- 51 MCP modules over the CRM API
- Specialist agents
- 20 agents, each with a narrow remit
- Documented skills
- 42 skills encoding operating rules
- Gate before any write
- Dry run, approval, rollback
The finding that shaped everything else
The system exists to operate real CRM portals — reading them, auditing them, and making changes that a business depends on. The single most important thing learned building it is this: configuration describes what a portal is capable of doing, and only the action log records what it actually did.
Those diverge constantly. A workflow can be enabled, correctly configured by inspection, and have enrolled nobody for two years. A routing rule can branch four ways and rotate into two pools. A permission set can be the real cause of a symptom that every configuration screen attributes to teams.
When config-derived findings were checked against behavioural evidence, a substantial share turned out to be refuted or overstated. So every finding the system produces carries a label — verified, hypothesis, or question — and a config-only finding is never allowed to carry a bare severity. It is a hypothesis until behaviour confirms it.
Why a second opinion from the same source is worthless
The intuitive fix for an unreliable analysis is to have another agent review it. That does not work here, and understanding why is the design.
A reviewer reading the same configuration reaches the same conclusion, because the configuration genuinely does say what the first agent said it says. Agreement between two readers of one source is not corroboration; it is the same claim, twice, with more confidence attached.
So the critic in this system is given the findings and the portal identifier, and explicitly not the original agent's reasoning. It re-derives each claim from the portal's own state and action log, and returns an accept or a revise naming the specific claims that must change. Independence has to be structural, not just a second prompt.
Separating the agent that acts from the agent that checks
The same principle runs through execution. The agent that applies a change is not permitted to decide whether it worked.
Verification is a separate agent that receives a factual packet — the approved plan, the expected end state, the protected assets, the portal — and never the executor's account of what it did. It reads the portal directly. Anything short of a pass becomes a new plan requiring new approval, rather than an in-flight repair, because a repair improvised mid-execution is exactly how a small mistake becomes a large one.
There is a gate in front of every write. A change is planned, dry-run, approved by a person, snapshotted, then executed — and the executing agent holds precisely one mutating tool, which refuses anything not named in the approved plan. Guardrails that live in the prompt are suggestions. Guardrails that live in the tool are constraints.
The boring parts that make it usable
Credentials are redacted before anything reaches the model, and because the learning store is shared across engagements, that redaction is a cross-tenant boundary rather than a nicety.
The portal identifier is passed explicitly on every single call. That began as defensive practice and stayed because a near-miss demonstrated why: an operation aimed at the wrong portal is not recoverable by apologising.
Rules discovered during real work are captured as candidates rather than promoted straight into the operating instructions. A candidate rule is not loaded by default and is not published until an evaluation exists that would catch its regression. A knowledge base that grows without a test for each entry becomes a pile of confident folklore.
What it is honestly for
This is not a system that runs a CRM unattended, and it is not sold as one. It is a system that lets one person audit and operate portals at a depth and speed that would otherwise take a team, while keeping a human at every point where something irreversible happens.
The measure of it is not how much it automates. It is whether the findings hold up when somebody checks them, and whether the changes are the ones that were approved.
What changed
- Every finding carries a label — verified, hypothesis or question — so a configuration-derived claim can never reach a client wearing a severity it has not earned.
- Verification is structurally independent: the critic receives the findings and the portal, never the original agent's reasoning, so it re-derives claims instead of agreeing with them.
- The agent that applies a change is not permitted to judge whether it worked, and anything short of a pass becomes a new plan requiring new approval rather than an improvised repair.
- Every write passes a plan, dry-run, human approval and snapshot, enforced by the executing agent holding exactly one mutating tool that refuses anything not in the approved plan — a constraint in the tool, not a suggestion in a prompt.
- Credentials are redacted before reaching the model, and the portal identifier is passed explicitly on every call after a near-miss showed why.
- The work behind several of the client engagements published on this site was carried out through this system.
Where it stands
In continuous production use across live client portals. Described here by capability only — no portal, client or engagement is identified, by design and by agreement.
Questions people ask about this
Can AI agents be trusted to audit a CRM?
To produce findings, yes. To assign severity to them without checking, no. Configuration describes what a portal is capable of doing; only the action log records what it actually did, and those diverge constantly. A workflow can be enabled, correct by inspection, and have enrolled nobody for years. Treat any configuration-only finding as a hypothesis until behaviour confirms it.
Does having a second AI agent review the first one help?
Not if it reads the same source. A reviewer given the same configuration reaches the same conclusion, because the configuration genuinely does say what the first agent said. That is not corroboration, it is the same claim twice with more confidence attached. Independence has to be structural: give the critic the findings and the system, never the original reasoning, and make it re-derive each claim.
How do you stop an AI agent making an unwanted change to a live system?
Put the constraint in the tool rather than the prompt. A plan, a dry run, human approval and a snapshot before anything is written, with the executing agent holding exactly one mutating tool that refuses any operation not named in the approved plan. Guardrails written into instructions are suggestions; guardrails in the tool are constraints.
Should the agent that makes a change also verify it?
No. Verification is a separate agent that receives the plan, the expected end state and the system itself — never the executor's account of what it did — and reads the live state directly. Anything short of a pass becomes a new plan needing new approval rather than an improvised repair, because a repair invented mid-execution is how a small mistake becomes a large one.
What is MCP used for here?
As the tool layer between the model and the CRM API — a set of modules exposing specific, bounded operations rather than a general-purpose API client. That boundary is what makes it possible to hold the mutating surface down to a single gated tool.
This is one engagement. The seven judgements that recur across all of them are what the work actually has in common.
Agent systems that operate real business systems need evidence, independent verification and a gate in front of every write. The interesting engineering is in the constraints, not the prompts.
Operational AI Strategy