The Shape of Code

It’s 2.5 years since my 2024 post summarizing what I knew about software effort estimation. For ongoing projects the role of estimation is as a planning tool, breaking down a large chunk of work into what are estimated to be small, well-defined, chunks. With two-thirds of estimates being within a factor of four (over or under) of the actuals, accuracy was not a primary goal.

The impact of coding agents capable of implementing these small tasks for peanuts is on developer staffing, not on the need for planning and breaking down work into connected small chunks.

Coding agents follow the specification they are given. Large, complicated programs will have large complicated specifications, which are likely to require a lot of human effort to create. For instance, the specification for the recent LLM generated C compiler

How long will it take to create a specification and how many iterations will be needed to create a good enough program? Estimation has not gone away, it has just moved upstream of the implementation. While specifications have existed millennia longer than software development, we are at the start of learning to use them to direct coding agents (the term vibe coding was created in February 2025). While logs of developer coding agent sessions are starting to be collected, it will take some years for patterns of practice to stabilize.

The process followed to estimate Function Points appears to be ideally placed to be extended to cover specification creation for coding agents.

Peanuts in bulk can be expensive. The novelty of much lower costs, when using coding agents, will soon wear off. Agent costs will become the new normal, which means there will be pressure to reduce them, and management will want estimates because the requirement to fill in boxes has not gone away.

Is agent token consumption purely a driven by the LLM(s) used and the selected configuration values? A recent study investigated the impact of the wording of the specification on the tokens spent by a selection of agents. The results showed that a specification containing bare user stories increased token spending by 30%. We will have to wait to find out whether the implications of this result (i.e., investing in more detailed user stories reduces token spending) replicates.

It would be very useful to have an estimate of likely token usage for a given specification, before any code is generated, or perhaps a real-time cost estimate during generation. One study found that current frontier models failed to accurately predict their own token usage (correlation of 0.39), humans predictions were equally poor.

Many projects don’t have a separate line item for testing costs, which often get bundled with implementation costs (the SEA Data Warehouse dataset separates out design/implementation/testing effort). Large projects may have people who work on integration and/or release testing.

When coding agents are used, testing is clearly a distinct piece of work separate from implementation. How effective is the specification used to generate the code at generating good enough tests? Current research is an extension of existing testing practices, rather than anything coding agent specific. A lot of existing research has focused on unit testing, and of course models have been fine-tuned for unit testing.

Implementation cost reductions have brought N-version programming into the realm of consideration for some projects.

Will the way developers work change to be like tradesmen who are hired to install a new fitted kitchen or bathroom? With developers installing new application functionality specified by the client. Perhaps some will be more like builders hired to make structural changes to a house, while others will do small jobs, like plumbers, electricians and carpet fitters.

As always, some researchers fail to move on when the world changes and continue to publish papers that use LLMs to estimate human task implementation time.