Scope and test plan
System version, permitted actions, environment, scenarios, expected behavior, exclusions, and decision criteria.
AI AGENT EVALUATION / SCOPED PILOTS
Evaluate agents before they take consequential actions.
Holikov Systems tests tool-using workflows against defined failure scenarios and documents what happened, what can be repeated, and what remains unknown.
Current public evidence is an internal synthetic lab, not a client or production result.
EXECUTED / AGENT ACTION LAB V0.115 SEP 2026
What the local test showed
Across 72 author-designed dangerous attempts per mode, observed state changes were 52 in the basic mode, 28 with limited guards, and 0 with the Action Contract. The same contract implemented as ordinary deterministic code also had 0.
Local services and synthetic scenarios. This does not measure real-world failure rates or establish a unique product advantage.
Read the method, results and limits01THE OFFER
A bounded evaluation pilot for a specific decision.
Start with the agent, its tools, and the decision you need to make. The pilot scope is agreed before any testing begins.
We turn a defined concern into testable scenarios, capture the agent’s behavior, and report findings with their limits.
A pilot is scoped individually. Access, duration, price, and disclosure are agreed in writing; this page does not imply a fixed package or certification.
02DELIVERABLES
A result your team can review and act on.
The proposed pilot output is a traceable evaluation record. Its exact contents follow the agreed scope.
Observed traces
Inputs, tool calls, resulting state, repeat runs, and a clear link from each finding to its supporting artifact.
Findings and regression set
What failed or held, severity rationale where applicable, uncertainty, and tests suitable for a later retest.
03PUBLIC EVIDENCE
One executed laboratory study, shown with its limits.
The Agent Action study uses 93 author-designed synthetic scenarios and local services. It is a demonstration of a test method, not validation on a customer system.
AGENT ACTION / V0.1INTERNAL LAB
Four policy modes compared
Dangerous action escapes: basic 52/72; limited guards 28/72; Action Contract 0/72; equally complete deterministic policy 0/72.
The scenarios are dependent and selected by the author. The separate Gemini batch produced no dangerous proposals, so it did not establish a safety improvement.
Inspect the case reportSTART WITH A CLEAR QUESTION
Describe the system and the decision.
Send a short, non-confidential summary. We can then agree whether a bounded pilot is suitable.