Comment by gojkoa

1. I have a custom-built minimal flow of 5 "commands", that take an idea to implementation through capturing intent, impact analysis, iterative planning, execution orchestration and then review/demo. each of these effectively builds up a single shared plan file. The intent command takes one or two lines of free text and turns it into a structured intent document (goals/anti-goals/constraints...); impact-analysis takes the intent document and maps it to the codebase, looking for functional gaps and things that need to change. iterative-planning takes the result of the impact analysis and splits into tasks/phases that are independently verifiable and deployable, and builds a task list... so they all build on each other, and update the same plan file that sits in git and I review it as it goes through the pipeline

2. we have a minimal CONTRIBUTING.md that explains the shape of the workspace and the key rules how to work, that's applicable to humans and agents. CLAUDE.md loads it from @CONTRIBUTING.md

3. the guardrails of what the agent is allowed or not allowed to do mostly sit in deterministic tools, such as custom linting rules, custom style checks, and they are all invoked from eslint via the custom language plugin. this has grown to thousands of rules, linting programming language code but also html, scss, yaml, liquid, markdown.... Eslint runs as a post-tool use hook on edit and write, so each file an agent writes gets immediate feedback and fixes. when we catch the agent doing something it should have not done with the code or docs, we get it to write another custom rule with unit tests for the rule. With each rejection Claude also gets a helpful message what to do instead.

4. there's a "regulator" script that helps us avoid decision fatigue for approvals. it runs as a pre-tool use hook for Bash commands, does deep parsing of whatever sausage Claude wants to run, goes into loops, function definitions etc, then goes through our rule set and approves/rejects or forces an ask. With each rejection Claude also gets a helpful message what to do instead. (e.g. don't run npx, use eslint directly from the path). Each time claude asks, we run the command through a debug script to understand why it's asking, and add another rule.

5. the latest addition is an orchestrator script that takes our plan file format and turns into Claude Dynamic Workflow descriptions deterministically, and parses the progress of the workflows so suggest what's slow and what can be moved out of LLM processing to deterministic tools. This significantly reduced the token spend (now running at about ~20% of the token spend for workflows before) and time (from average 5-6 hours per workflow to about 20-30 minutes). It also removed the 10-15 minute wait that we had while claude was LLM constructing workflows from our plans.

6. there's a demo command that flies through the user interface based on the plan to demonstrate what users see with the new version, recording it to webm using playwright, and I can play it at a higher speed to quickly get an overview what an agent did.

7. we tend to look for ways to get faster feedback on things that agents repeatedly do badly, such as UX or UI changes. We have a set of static HTML pages with the visual design language, showing styling for key elements and components, and a set of static demo pages showing key application pages with realistic data in lots of different states. as part of the impact analysis, agents will update demo pages or add new ones so we can review/complain. there's a script that audits demo pages for WCAG and other styling issues. another example are end-to-end api tests, which evolved massively to prove api contracts but also allow agents to get their own feedback and troubleshoot quickly. generally, divide and conquer for feedback.

This tends to work generally well. I feel productive. I still review most code when it completes via git diff, but it's mostly clean because the linting rules are forcing it to write code the way I want it to be written. We use a method based on the attribute-component-capability matrix to figure out what needs manual exploratory testing and how much, and do that in addition to automated tests when needed.

happy to provide any more info if you're interested.