Using Anthropic Claude models for coding:

Opus is strong on analysis. Industry benchmarks show it’s strong at coding, but I find it ambitious and rangy. It drifts beyond its mission when struggling, and will charge down supposition paths without grounding.

Sonnet is good at tasking out work and quite good at following through on well-defined tasks, but can be short-sighted in terms of pattern and method reuse.

The current version of Haiku falls short. It shows its age with anti-patterns like aggressively duplicating code and hardcoding values, particularly when spins on a task.

So, for now, I live in Sonnet. I use Opus mostly for analysis on big problems. I’ve stopped trying to use Haiku for now. Would love to hear from someone who is getting decent results in complex code with one of the smaller models.

My writeup on my plan-do-check-act AI coding practice is live on InfoQ.

It works well with plan mode. In fact, it anticipated it.

And retrospection is alive and well!

I provide links to prompts and code metrics github actions.

You can’t tack production requirements onto rapidly produced code. The age of AI hasn’t changed this fact.

Agentic code agents can rapidly prototype applications in hours. This is amazing. These prototypes can quickly help people test the viability and value of a feature or implementation approach.

Production code is a complex mix of such details that require scalability, safety, security, deployability, test coverage, quality evaluation and stakeholder sign off, and optimization for performance, troubleshooting and maintainability. Production code takes orders of magnitude longer to create than prototypes.

The disposable prototypes agents generate usually can’t even be incorporated into such systems without a rewrite. Production software requires a more human centric and expert led approach from the start.

Finally, the speed of delivering software to users is not directly driven by the speed at which the code itself is produced. Code authoring is the middle of a value stream and is not the sole bottleneck in building effective software. AI can also speed up investigation and learning. It can facilitate communication and documentation. In the hands of qualified, experienced teams, AI tools can speed up actual delivery of quality production software by 10-30%.

This gain is worth shooting for but it is an iterative, evolutionary practice improvement. Not something you get out of the box as part of a software license.