One thing I hope to see included is a precursor step when constructing specs where Claude is used to intelligently inquire about gaps to fill that would disambiguate the implementation. If you told an engineer to do something with a set of requirements and outcomes, they'd naturally also have follow-up questions to ensure alignment before executing.
joshstrange · 46m ago
Yes, kind of like open AI’s deep research tool. I often find that a number of mistakes are made because no clarification questions or ask or even considered.
CompoundEyes · 3h ago
For anyone that’s applied one of these at what level of autonomy are you using it? And in what setting? Greenfield?
I see one mention brownfield development. Has anyone with experience using these frameworks fired up Claude Code on enterprise software and had confident results? I have unchecked access to Claude Code at work and based on personal agentic coding I’m sure they do aid it. I have decent but not consistent results with my own “system” in our code base. At least until the front end UI components are involved even with Playwright. But I’m curious — how much litter is left behind? How is your coworker tolerance? How large are your pull requests? What is your inference cost? How do these manage parallel?
The README documentation for many have a mix of fevered infomercial, system specific jargon, emoji splatter and someone’s dad’s very specific toolbox organization approach only he understands. Some feel like they’re setting the stage to sell something…trademarked!? Won’t Anthropic and others just incorporate the best of the bunch into their CLI tools in time?
Outside of work I’ve regularly used a reasoning model to produce a ten page spec, wired my project with strictest lint, type check, formatter, hooks, instruct it to check off as it goes and do red green TDD. I can tell gpt-5 in Cursor to “go”, occasionally nudge to stay on task and “ok next” then I’ll end up with what I wanted in time plus gold plating. The last one was a CLI tool for my agent to invoke and track their own work. Anyone with the same tools can just roll their own.
dsiegel2275 · 1h ago
I'm only three weeks into using Claude Code but I'm now seeing impressive results using a structured, "role" or "persona" based approach in a large (500K+ SLOC) Elixir / Phoenix codebase. I'm using the $200 Max plan - so my inference costs are fixed.
For certain, the results are better when I use it to build new features into our platform - as opposed to making complicated refactors or other deep changes to existing parts of the system. But even in the latter case, if we have good technical documentation capturing the design and how parts of the system work (which we don't in many places), Claude Code can make good progress.
At first I was seeing a fair amount of what I would consider "bad code" - implementation and code that either didn't follow accepted coding style and patterns or that simply wasn't structured for reusability, maintainability. But after strengthening the CLAUDE.md file and adding an "elixir-code-reviewer" subagent which the "developer" persona had to use - the quality of code improved significantly.
I didn't realize just how bad LLMs are with unpopular languages until now:
"Elixir lists do not support index based access via the access syntax"
"Never use else if or elseif in Elixir, always use cond or case for multiple conditionals."
iandanforth · 56m ago
Thanks for sharing that extensive documentation!
stocksinsmocks · 58m ago
I got a strong sense of LLM style in the blog. Interesting information, but that I’m learning about AI from AI is amusing.
stingraycharles · 50m ago
Yeah, unfortunately this is very often the case with articles about AI.
In my own experience, this type of stuff is just wishful thinking right now: for anything non-trivial, you still need to monitor Claude Code closely and interrupt when you discover it goes on the wrong train of thought.
Additionally, for security reasons, you don’t want it to give it too many permissions, and/or actually see which commands it’s executing.
The “frameworks” OP talks about are still far away. Right now the best way to think about it is an intern which is usually wrong but can cramp out code at lightning speed.
Szpadel · 14m ago
looks like either author didn't checked well repos that link or it's indeed redacted deep research output.
Examples:
superClaude is not mcp server at all
metaGPT looks like is not compatable with Claude code at all
matt3D · 5h ago
Pretty surprised BMAD-method wasn't mentioned.
For my money it's by far the best Claude Code compliment.
3uler · 4h ago
The BMAD system seems similar to the AgentOS mentioned in the post.
This way of context engineering has definitely been the way to go for me, although I’ve just implemented it myself… using Claude to help generate commands and agents and tweaking them to my liking, lately been using json as well as markdown to share context between steps.
CGamesPlay · 4h ago
What is this? Just a system prompt? What makes it so good for you?
An AI tool finding issues in a set of YAML and Markdown files generated by an AI tool, and humans puzzled by all of it.
> We should really have some code reviewer...
Gemini to the rescue!
matt3D · 2h ago
It manifests as a sort of extension for Claude Code.
When I'm in the terminal I can call on Agents who can create standardised documents so there is a memory of the product management side of things that extends beyond the context window of Claude.
It guides you through the specification process so that you have extremely tight tasks for Claude to churn through, with any context, documentation and acceptance criteria.
Perhaps there are others similar, but I have found it completely transformative.
3uler · 4h ago
It’s basically a set of commands and agents and a way to structure context.
MarcelOlsz · 5h ago
Same with taskmaster, also not there.
3uler · 4h ago
I never found taskmaster that useful, something about how it forced you to work didn’t click with me…
troupo · 4h ago
> a set of rules, roles, and workflows that make its output predictable and valuable.
Let me stop you right there. Are you seriously talking about predictable when talking about a non-deterministic black box over which you have no control?
andsoitis · 4h ago
> Are you seriously talking about predictable when talking about a non-deterministic
Predictability and determinism are related but different concepts.
A system can be predictable in a probabilistic sense, rather than an exact, deterministic one. This means that while you may not be able to predict the precise outcome of a single event, you can accurately forecast the overall behavior of the system and the likelihood of different outcomes.
Similarly, a system can be deterministic yet unpredictable due to practical limitations like sensitivity to initial conditions (chaos theory), lack of information, or the inability to compute predictions in time.
sublinear · 3h ago
The topic of chaos is underrated when people talk about deterministic systems, but I think it's at least (usually?/always?) a tractable problem to draw up a fractal or something and find the non-chaotic regions of a solution space. You have nice variables to work with when you draw up a model of the problem.
Maybe someone can elaborate better, but it seems there is no such luck trying to map probability onto problems the way "AI" is being used today. It's not just a matter of feeding it more data, but finding what data you haven't fed it or in some cases knowing you can't feed it some data because we have no known way to represent what is obvious to humans.
troupo · 3h ago
From the discussion in the link: "Predictability means that you can figure out what will happen next based on what happened previously."
Having used nearly all of the methods in the original article, I can predict that the output of the model is nearly indistinguishable from a coin toss for many, many, many rather obvious reasons.
raincole · 3h ago
Yes, and there is absolutely nothing wrong with that. Living creatures are mostly black boxes. It doesn't mean we don't aim for making medicine with predictable effects (and side effects).
sublinear · 3h ago
I don't think that's accurate.
Don't most people working in medicine usually have biology and chemistry degrees? Are you saying those sciences are dark arts?
raincole · 3h ago
Having biology degrees doesn't make you understand every detail of human body. There are many, many drugs that are known to work (by double blind testing) but we don't know exactly how.
The details of how penicillin kills bacteria were discovered in 2000s. Only about half a century of after its commercial production. And I'm quite sure we'll still see some more missing puzzle pieces in the future.
sublinear · 3h ago
Yes, but I think we want to know how they work? Not knowing "exactly how" but having a good ballpark idea is not equivalent to letting AI throw stuff at the wall to see what sticks.
troupo · 3h ago
Medicine that can either kill you, cure you, or have no effect at any given time for the same disease is quite unlikely to even pass certification.
Do you know why?
raincole · 3h ago
That is exactly my point.
dist-epoch · 3h ago
Non-deterministic does not mean not-predictable.
Quantum mechanics is non-deterministic, yet you can predict the motion of objects with exquisite precision.
All these "non-deterministic boxes" will give the same answer to the question "What is the capital of France"
sublinear · 3h ago
Yes, but the "exquisite precision" comes from the deterministic parts of physics.
baq · 2h ago
Nah. The only thing we can establish precisely at the lowest levels is probability. We can and do engineer systems to maximize probabilities of desired outcomes and minimize probabilities of undesirable ones.
Frankly I don’t understand how software engineers (not coders mind you) can have issues with non deterministic tools while browsing the web on a network which can stop working anytime for any reason.
AbuAssar · 3h ago
did anyone try any of these so called frameworks? do they deliver or just riding the hype-wagon?
stuartjohnson12 · 1h ago
Having occasionally looked into these, the overwhelming issue is that pretty much all of them are built with themselves and the output is what you'd expect - vast quantities of untested functionality, no documentation beyond a wall of Claude-isms about how next generation and feature rich it is, and no thought to applicability beyond the narrow set of projects that interest the author.
I see one mention brownfield development. Has anyone with experience using these frameworks fired up Claude Code on enterprise software and had confident results? I have unchecked access to Claude Code at work and based on personal agentic coding I’m sure they do aid it. I have decent but not consistent results with my own “system” in our code base. At least until the front end UI components are involved even with Playwright. But I’m curious — how much litter is left behind? How is your coworker tolerance? How large are your pull requests? What is your inference cost? How do these manage parallel?
The README documentation for many have a mix of fevered infomercial, system specific jargon, emoji splatter and someone’s dad’s very specific toolbox organization approach only he understands. Some feel like they’re setting the stage to sell something…trademarked!? Won’t Anthropic and others just incorporate the best of the bunch into their CLI tools in time?
Outside of work I’ve regularly used a reasoning model to produce a ten page spec, wired my project with strictest lint, type check, formatter, hooks, instruct it to check off as it goes and do red green TDD. I can tell gpt-5 in Cursor to “go”, occasionally nudge to stay on task and “ok next” then I’ll end up with what I wanted in time plus gold plating. The last one was a CLI tool for my agent to invoke and track their own work. Anyone with the same tools can just roll their own.
For certain, the results are better when I use it to build new features into our platform - as opposed to making complicated refactors or other deep changes to existing parts of the system. But even in the latter case, if we have good technical documentation capturing the design and how parts of the system work (which we don't in many places), Claude Code can make good progress.
At first I was seeing a fair amount of what I would consider "bad code" - implementation and code that either didn't follow accepted coding style and patterns or that simply wasn't structured for reusability, maintainability. But after strengthening the CLAUDE.md file and adding an "elixir-code-reviewer" subagent which the "developer" persona had to use - the quality of code improved significantly.
Our platform is open source, you can see our current Claude commands and subagents here: https://github.com/Simon-Initiative/oli-torus/tree/master/.c...
"Elixir lists do not support index based access via the access syntax"
"Never use else if or elseif in Elixir, always use cond or case for multiple conditionals."
In my own experience, this type of stuff is just wishful thinking right now: for anything non-trivial, you still need to monitor Claude Code closely and interrupt when you discover it goes on the wrong train of thought.
Additionally, for security reasons, you don’t want it to give it too many permissions, and/or actually see which commands it’s executing.
The “frameworks” OP talks about are still far away. Right now the best way to think about it is an intern which is usually wrong but can cramp out code at lightning speed.
Examples: superClaude is not mcp server at all metaGPT looks like is not compatable with Claude code at all
For my money it's by far the best Claude Code compliment.
This way of context engineering has definitely been the way to go for me, although I’ve just implemented it myself… using Claude to help generate commands and agents and tweaking them to my liking, lately been using json as well as markdown to share context between steps.
https://github.com/bmad-code-org/BMAD-METHOD
An AI tool finding issues in a set of YAML and Markdown files generated by an AI tool, and humans puzzled by all of it.
> We should really have some code reviewer...
Gemini to the rescue!
When I'm in the terminal I can call on Agents who can create standardised documents so there is a memory of the product management side of things that extends beyond the context window of Claude.
It guides you through the specification process so that you have extremely tight tasks for Claude to churn through, with any context, documentation and acceptance criteria.
Perhaps there are others similar, but I have found it completely transformative.
Let me stop you right there. Are you seriously talking about predictable when talking about a non-deterministic black box over which you have no control?
Predictability and determinism are related but different concepts.
A system can be predictable in a probabilistic sense, rather than an exact, deterministic one. This means that while you may not be able to predict the precise outcome of a single event, you can accurately forecast the overall behavior of the system and the likelihood of different outcomes.
https://philosophy.stackexchange.com/questions/96145/determi...
Similarly, a system can be deterministic yet unpredictable due to practical limitations like sensitivity to initial conditions (chaos theory), lack of information, or the inability to compute predictions in time.
Maybe someone can elaborate better, but it seems there is no such luck trying to map probability onto problems the way "AI" is being used today. It's not just a matter of feeding it more data, but finding what data you haven't fed it or in some cases knowing you can't feed it some data because we have no known way to represent what is obvious to humans.
Having used nearly all of the methods in the original article, I can predict that the output of the model is nearly indistinguishable from a coin toss for many, many, many rather obvious reasons.
Don't most people working in medicine usually have biology and chemistry degrees? Are you saying those sciences are dark arts?
The details of how penicillin kills bacteria were discovered in 2000s. Only about half a century of after its commercial production. And I'm quite sure we'll still see some more missing puzzle pieces in the future.
Do you know why?
Quantum mechanics is non-deterministic, yet you can predict the motion of objects with exquisite precision.
All these "non-deterministic boxes" will give the same answer to the question "What is the capital of France"
Frankly I don’t understand how software engineers (not coders mind you) can have issues with non deterministic tools while browsing the web on a network which can stop working anytime for any reason.