Does Software Engineering Education/Training Need To Change For LLMs?
Or has this all happened before?
I recently wrote a post on my code review of Cursor. In the end, I found the AI code editor incredibly valuable. The one big concern that was developing as I became more familiar with it was the future of the software engineering profession. I'm not concerned about LLMs taking jobs, it is very far away from being able to do that. I am concerned with how software engineers are going to be trained from now on.
The caveat of AI code editors is that they are useful *if* you can quickly catch the problems they create. The AI will create all sorts of issues. The ones that prevent the software from running at all are usually simple and easily fixed. The issues that affect the ability to maintain that codebase in production are another matter. The ability to detect those issues is gained from years of hard work and experience. That's the kind of experience you would expect from a senior engineer.
That creates a conundrum. The ability of an engineer already outpaces the growth in salary. A junior engineer will be several times more productive after their first year at a job. The raise they get after that first year is not going to be several times their salary. Most companies with an HR department balk at the idea of a raise more than 10%. When you compound that action over several years, you end up with a senior engineer that is significantly more valuable than a junior engineer at a cost that makes that senior engineer a bargain. That is why it's always been hard for someone to break into the field (I had to submit hundreds of resumes when I started). Now we're adding a tool that makes senior engineers more productive which increases the gap. If a junior engineer uses the tool, they will find it harder to develop the skills that would make them a senior engineer. If the junior engineer does not use the tool, they fall even farther behind in productivity in the short term.
This thought worried me for a few days until I realized it was based on some bad assumptions.
The first is that the ability to write good production code is gained from writing code that goes into production. I have never seen an engineer learn that way. Learning what works and doesn't work in production comes from debugging actual issues. Those issues can come from code written by anyone. You don't actually need to have done the initial write of that code to get firsthand experience with it. Debugging often involves writing a ton of code just to see what happens when conditions change. A person debugging an issue in production will quickly gain the same familiarity with that code as the person who wrote it initially. Because of this, it doesn't really matter whether that code was originally written by a human or an LLM. The learning comes from fixing it.
That leads to the second bad assumption: that the bad code LLMs generate are the worst imaginable. They are not. I've had to fix problems with code that was written by a human, but was decisively worse than anything I have generated with an LLM. If bad code wasn't deployed to production, we would have a lot fewer senior engineers. This was the case before LLMs and will not change because of LLMs.
The quality of code is also not black and white. A lot of it depends on the context. I've had to fix issues by switching the use of a message queue to using API calls just in time. I've also had to fix issues by doing the exact opposite, replacing API calls with a message queue. One can often have the best intentions when initially writing code, but the situation often changes when code goes live. Sometimes it isn't even a matter of one solution being better than another. Every solution imaginable will have some trade-off and it becomes a matter of what you and your team can live with. Senior engineers continue to grow by analyzing these decisions based on their experience and making the choice. Junior engineers lack the baseline experience to perform that analysis and will often spend a ton of time agonizing over a choice they can't make. Having an LLM pick one will speed things up by making that decision for them. In fact, the sooner that happens and the code gets released, the sooner that junior engineer can see what went wrong and learn from it. In this situation, the LLM speeds up learning.
The last bad assumption I made is that all junior engineers will use LLMs in the same way. This has not happened in the past with other tools. When MongoDB was released, a large number of engineer used it because they thought they could now ignore thinking of the schema ahead of time and the database would magically handle it. There were plenty of really smart junior engineers though who were uncomfortable with the idea of magic and dug deeper. They may have still used MongoDB or another NoSQL database, but they thought through what data they needed and how it should be stored. They became senior engineers much quicker than the other group.
I once interviewed a bootcamp graduate who talked about how she refused to use jquery in her capstone project even though her instructor told her to. Her reasoning was that she wanted to know what it was like to write everything with vanilla javascript so that she could better understand the value jquery provided. Even before MongoDB (and with lots of technology that came after), there are engineers who just want something easy and then there are engineers who put in the work to learn.
LLMs are a useful new tool, but there have been useful tools popping up for decades. Every time, there are engineers who take the easy way out and engineers who work to gain an understanding of how things work. LLMs aren't going to change that.