Giving my agents practical web tools
I use Hermès for tasks that involve finding, reading and interacting with web pages. Some sites make automated access difficult; once a page is accessible, the agent still has to identify the right controls in a potentially dense interface. I extended redf0x1’s camofox-mcp, the adapter connecting an agent to the Camoufox browser.
The browser engine and fingerprinting work come from Camoufox and camofox-browser. My contribution is in the adapter: interpreting instructions, returning structured information, preserving task context and diagnosing navigation problems.
Separating observation, decisions and actions
An instruction such as “read the product name and price” goes through a page snapshot, a specialised prompt and a language model. Extraction returns structured data and identifies missing information. Observation proposes relevant elements without acting; the action tool resolves a target and performs one interaction.
I retained a deterministic execution path for plans whose element references are already known. The agent can therefore choose between interpreting an intention and executing an explicit plan. Prompts require one action, references grounded in the page and no invented input text. They live separately from the TypeScript handlers so the instructions can be revised without changing tool implementation.
Understanding the result of a click
A completed call does not necessarily mean that a checkbox changed or a form advanced. I organised task context and recovery instructions around the observed outcome: a state change, a new dialog, an alert or no progress.
Snapshots can focus on a region and retain only the elements relevant to the task. Tab context keeps the objective and recent actions. Recovery instructions bound retries and require a changed parameter or strategy. This gives the agent useful material for its next decision when the interface no longer matches the original plan, including cases where a new dialog interrupts an otherwise valid sequence.
Maintaining the service and its contracts
I added model routing with fallbacks, validation and JSON repair. Tool profiles expose either a compact or broader set of capabilities through MCP, the protocol connecting the agent to the service.
Health checks test the adapter and browser server separately: a running process can answer requests while navigation remains unavailable. I also fixed cookie restoration so one rejected entry does not prevent the rest of a session from loading. These changes remain in a fork maintained alongside upstream updates. The repository and architecture documentation describe the tools, prompts and boundaries between the components.