But it is a mistake for managers to conclude that software development now comes at near zero cost, or that it does not require practice and additional skills. (View Highlight)
One of the main misconceptions of AI-assisted coding is what some call the demo trap: the difference between perceived vs actual amount of remaining work.
Indeed, a few prompts may be sufficient to create a landing page, a simple game, or a small personal app. The result is often impressive because a) this would have been impossible 3 years ago, and b) these examples typically involve a limited scope and well-defined requirements. (View Highlight)
However, these cases are poor proxies for the complexity of business apps or for entreprise software.
In these cases, as the code base grows, each additional prompt delivers diminishing productivity returns. Beyond a certain code size and number of components involved (databases, APIs, cloud services), AI models start losing track of the context and details, and it is precisely the human developer whose job is to remain specific and knowledgeable about what is being built and provide the context to guide the model successfully. (View Highlight)
Still, one can argue that AI keeps all this effort in the natural-language layer. But this is only the case when tasks are well-defined and the generated solution works as expected. (View Highlight)
According to our own research, as well as our experience teaching software prototyping courses, when the AI fails users are often forced back into the technical layer: reading error messages, inspecting code, understanding dependencies, or running terminal commands. (View Highlight)
Therefore, technical skills may no longer be required to start building software, but their absence represents a liability when projects move beyond the happy path. (View Highlight)
One that has existed even before the rise of LLMs is the alignment problem. This concept refers to the capability of an AI system to comply with the human goals (and values) it was designed for. Since AI systems are trained on data rather than explicitly programmed with every desired behavior, the objectives learned by a model may differ from the intentions of its designers, especially when operating in novel situations or under ambiguous instructions. (View Highlight)
With AI-assisted coding, the alignment problem manifests as ensuring that generated code reflects organizational objectives, architectural constraints, quality standards, or possible user behaviors. (View Highlight)
Complying with alignment requires measures such as providing detailed specifications and business context, incorporating automated testing, or maintaining human oversight for critical decisions, all which represent a substantial effort. (View Highlight)
And even in a future scenario where coding agents might solve many of these issues automatically, organizations would still face what AI pioneer Andrew Ng calls the “product management bottleneck”. (View Highlight)
Specifically, if one can theoretically implement any idea incurring a low cost, the new bottleneck in software development becomes deciding which ideas are worth investing in, shifting the question from “Can we build it” to “Why should we build it”? (View Highlight)
Even in this scenario, those with abilities such as business domain knowledge, rigor for details, and sensibility for design will generally outperform those who think they can disrupt the world with a few prompts. (View Highlight)
Prototyping a landing page can be done in an afternoon. Building a delightful product users love, complying with security or data governance standards cannot. (View Highlight)
Managers should welcome that software creation becomes accessible to a much larger share of the workforce, but ultimately, they also need to be prepared to understand the cost, and to face new organizational challenges. Who is responsible for the inventory and maintenance of these AI-generated applications? How should security, compliance, and architectural standards be enforced? How can companies avoid creating hundreds of disconnected tools? This is a question that some organizations have started exploring in specific contexts such as GPTs, but one that becomes significantly more complex when extended to software in general. (View Highlight)
