Large language models (LLMs) have a grand potential to improve software. I’d even say it has a greater potential to improve all creative arts as well. But I don’t believe we’ll be using it to those ends. Alas, I do not see it making a fundamental shift in what we produce, but rather continuing down a path we’ve been on for a long time already.
I’m going to distil the two directions of this technology, which is a similar two paths any technology can take. LLM can be used as:
- An assistive tool that helps programmers create better software: be more stable, use fewer resources, be faster, and have increased usability.
- A way to increase output and/or reduce workforce but follow the current trend in product quality.
Obviously that’s a simplified view, but it’ll help to make my comparative viewpoint in this article. The reality is that we’ll have a mix of both.
Will we lean more towards increased output and a downward spiral in quality? If we go this route, it’ll be our decision. And given current trends, it is my prediction that the vast majority of software will venture down this pathway.
Though often referred to as AI, I want to use the term LLM specifically, as my viewpoint is about this particular technology. I can envision many AI-like technologies which would make me think differently.
The Positive Assistant
I wrote an article a few years back about how an AI could help me as a programmer. I said it could help me with research, management, refactoring and debugging. Clearly all toward an assistive tool. To avoid fading away as a luddite, I’ve increasingly incorporated LLMs into my workflow. Let me look at how LLM’s have fulfilled my hopes so far.
I’ve found research to be the most impactful feature. I use ChatGPT to quickly find new syntax or refresh my memory of syntax in a language. It helps me find features of libraries, mostly recently for Unity and ChartJS. At a higher level, I’ve found it’s good at collating the various approaches games have used for specific features, like saves or plugins.
Now granted, it spits out a lot of junk info. It has a bad understanding of product versions and gladly mixes two different APIs together. It does not understand the systems it produces and frequently makes mistakes. Nonetheless, I know what I’m looking for, and combing through this mess is often faster than bare internet searches.
Can it refactor? We use Claude at work, but as of yet I have only seen Claude do repetitive boilerplate tasks. I don’t know if this is a limit of my skills or a limit of the technology itself. If an LLM can suggest structural edits to English text, it feels like a code tool should be able to do this for code. This is high on my list of values so I’ll keep trying and hope that the tool improves here.
Debugging is showing promise. As I said in my article, I’m not expecting miracles but hopeful that it finds dumb mistakes. And it’s living up to a lot of that. I find that ChatGPT is doing a good job of understanding fragments of code and finding mistakes. Granted it reports a lot of nonsense as well. I’ve also seen Claude PR reviews, and sometimes it has picked up on difficult bugs. Though again, like ChatGPT, Claude also emits a lot of nonsense.
For the management aspect, I have yet to see something of value here. Tools like Jira have mangled AI into their system and at best it produces poor task lists. I haven’t even seen the inkling of a tool that can cut across my email, chats, code, and issues and track the status of my project. But I still hope, since this theoretically should be something it can do well.
Sure, there are a lot of limitations here, but there’s promise. If we try to make these features better they will get better.
The Negative Viewpoint
I’d liked to focus my negative view primarily on automated reviews and production of new code. They are the key dangers that will explain why a myopic focus on increased volume will come with a constant erosion of code quality.
Code Review
At work we are experimenting with including LLM reviews in our PR pipeline. It makes sense. We spend a lot of time writing best practices and educating junior programmers. Why would we not try to encode the information for an automated system? I’m hopeful this can reduce some common back-and-forth, but we need to be attentive to two serious short-comings.
- Hallucinations: An LLM review can spit out nonsense about how the code is wrong. Even new programmers can usually identify this and discard it. It does however waste time and pepper reviews with distracting junk.
- Opinionated: An LLM can emit clever and plausible sounding concerns yet be completely wrong. It can also recommend defective alternatives. How much this trips up a programmer depends on their experience, but it will definitely lead code astray.
Those problems are of the LLM itself, but they can easily lead to complacency. As an additive review tool, LLM can be beneficial. But if we completely replace manual review, the LLMs will train themselves into a downward trajectory as their opinions and hallucinations become an increasing part of the codebase. This negative reenforcement loop can quickly swell to dominate online discussions and publically available code.
Code Generation
When I see code generated by Claude, I see a lack of concern for maintenance and completeness. It is better at emitting redundant code than abstracting and simplifying. It does not appear to understand locality of code and happily emits duct tape patches wherever they might work.
Perhaps this is manageable, but there’s a significant danger about how these end up being implemented in process. If the LLMs become gatekeepers, one could find that junior programmers will just follow its advice to bad code and senior programmers will have to increasingly justify their quality code.
This is not a foregone conclusion, but I have yet to see an answer to this type of problem.
I’ve also seen a where I spend my interacting with an LLM. Instead of thinking about the code, I’m having a conversion trying to convince them to emit something sensible. I’m not spending my time reading code and building knowledge about the code-base. I’m not spending my time reading documentation or articles building auxiliary knowledge. My ability to use the LLM effectively comes primarily from this knowledge I’ve been exposed to over my career.
There’s also a circular logic problem. How can the same tool fulfil reviewing code and generating code? This can heavily bias one’s approach. I am disincentivized from finding good solutions in favour of Claude suggesting approaches that it itself will accept on review.
But will it work…?
An issue of provenance
Before addressing what’ll happen, I’d like to take a short, but important, detour here.
A big challenge for LLM-based coding is the issue of knowledge provenance. The LLM has to learn about coding from somewhere. Fortunately, or unfortunately, as the world needed more software there was an explosion in the number of programmers and sites offering discussions, theory, answers and code.
In time however, these sites ended up feeding a narrow set of business desires. As every company needed its own app for everything, software companies became the minority presence in the programming world. The programming websites then became focused on providing easy answers for the shovelware that every company needed to have. The goal is the fastest copy-and-paste path to realizing this. Industry was not concerned about stability, security or user experience.
Take a look at the websites and apps that you use? Do they crash and have weird visual bugs? Does the signup fail or is the UI extremely confusing? When was the last data leak? Have there been new features, or just new ads and tracking? This is all part of the code and knowledge publically available.
I could talk at length here, but suffice to say, the vast majority of programming knowledge is aligned with those business goals and that quality. And it’s clear that LLMs have been trained from these sites. This doesn’t mean I can’t get helpful code out of them, but it’s an uphill battle and their responses are a clear focus on the “ship it” mantra.
But will it work…?
We come to the crux: will this approach work? And this is where the issue of provenance and my initial visions for LLMs conflict.
I firmly believe that the success of LLMs depends on keeping the backbones of our systems as well-crafted tools. This isn’t where the majority of coding is being done though. Most coding is on periphery business-specific tools and bloat. And if this gets easier to write, then more of it will be written. It’s easy to see how it can quickly dwarf the core systems in terms of total code.
This doesn’t preclude an LLM from being used as an assistant to produce higher quality code. My fear is that if the focus is on shovelware, the LLMs will reenforce to that goal. It’s possible that the feedback loop could undercut the LLM’s value as a quality tool.
We still have options now though. We’re at a point where an LLM can be used for both ends. Somehow, despite the knowledge slop available, they are capable of improving work process.
So if you’re a programmer, or any part of a company making software, make a decision about how hard you want to fight for product quality. Then ensure your LLM use aligns with that vision.
