Is the 'AI Developer' a Job Threat or Just a Marketing Ploy?

In recent weeks, news has spread about so-called 'AI developers' – artificial intelligence tools designed to autonomously create software. Some startups, such as Magic.dev, are attracting significant investments with the goal of creating a 'superhuman software engineer' that should surpass human capabilities. Recently, the startup Cognition Labs garnered attention with its product 'Devin,' presented as the 'first AI software engineer,' and simultaneously announced significant funding.

Cognition Labs claims that Devin achieved top results in the SWE-Bench benchmark, which assesses AI's ability to solve real-world problems from GitHub in open-source projects. According to them, Devin successfully passed technical interviews and even completed real jobs on the Upwork platform. The company showcases videos where the agent autonomously detects errors, performs debugging using diagnostic outputs, fixes issues, and commits changes.

While these claims have generated sensational headlines in the media, it's appropriate to question the extent of their exaggeration. Cognition AI itself admits that Devin, without assistance, solved only about one out of seven problems on GitHub in tests. While impressive, this is still far from the performance of an experienced human engineer, and even a junior developer would likely manage more. Interestingly, Cognition AI's website, the creator of the 'first AI developer,' contains no elements that were generated by the AI itself, which would confirm its readiness for production deployment.

From current demonstrations and descriptions, these tools appear to be more like advanced agents capable of switching between various tools and automating certain tasks. However, at this stage, they seem more like prototypes in an intensive development phase, with significant limitations and a need for human oversight and input. Although they raise concerns about job stability, their current form suggests they are more assistants than full-fledged replacements.

We might ask whether these bold pronouncements are underpinned by a marketing strategy, necessary for survival in a saturated market. The space for AI-assisted coding tools is already quite crowded today, with numerous established players and solutions. The costs of developing and operating large language models are considerable, making it difficult for new startups to compete on price with existing giants who already have established infrastructure and a customer base.

In such a competitive environment, it becomes almost a necessity for startups with AI development tools to make extravagant claims about fully replacing software engineers. Anything less might not be enough to convince developers to abandon their existing and proven assistants and entrust their code to a new, unknown tool. This strategy clearly works in terms of media attention, providing a valuable gain for these companies in the form of interest and waiting lists.

The claim that 'we won't need software engineers' has been repeated in the history of technology since the 1960s. One example is the COBOL programming language from 1959, designed to be used by people without programming experience to reduce programming costs. While COBOL became very popular for a short time, it ultimately did not end the demand for developers; instead, it created demand for COBOL developers, whose role remains indispensable in some critical systems to this day.

Similarly, modern efficient frameworks or 'no-code' tools, while partially successful, have not entirely eliminated the need for human developers. Large Language Models (LLMs) represent a significant leap forward, but they have inherent limitations, particularly a tendency to 'hallucinate,' i.e., generate factually incorrect information. While it's possible to add verification loops in coding to test code and eliminate errors, the performance of LLMs in unknown technologies and their long-term reliability remain questionable. They are essentially highly sophisticated 'probabilistic machines,' not thinking entities.

It is undeniable that software tools for developers will differ from today's in five years and will be more powerful. However, the vision of 'AI developers' doing most of the work seems more like marketing messages necessary for startups in the industry to attract attention and investors. A more probable future involves improved versions of existing AI-assisted coding tools that help developers without fully replacing them.