The Problem With Developer Silos
What sounds like a good org structure hinders good product decisions
I've long believed that consumer software applications are of significantly higher quality than most enterprise software applications. Consumer applications are easier to use, are built quicker, and have fewer bugs (or fewer misunderstandings that are interpreted as bugs). One could argue that the reason is because the use cases for enterprise applications are more complex. Having spent some time in e-commerce, I don't think Amazon's use cases are very simple.
I think the reason is because when it comes to consumer apps, developers are often the users of those apps. If you work on Amazon, you probably also buy from Amazon. If you work on Facebook, you likely also use Facebook. If you work on a video game, you probably play video games.
Contrast that with enterprise applications. How many developers on CRMs actually use a CRM outside of work? How many developers who build accounting applications actually do any accounting themselves? I spent some time working on software for K-12 students and I can assure you that no one on the team was a K-12 student or really remembered what it was like being one.
Almost everyone working on a consumer application has an intuition about that domain. By being users themselves, they know what makes sense and what clearly doesn't make sense. That matters because no requirement spec is perfect. There are always hundreds of edge cases that are not taken into account. Example: how often have you seen a loading state defined in the requirements? How often did you just make it up as you went along? How often do you use software that freezes or white screens while loading something?
Fun fact: there is a very popular accounting application used by my business partner that white screens for about 20 minutes while it loads the data he needs. I'm going to let you guess what application that is.
A developer who understands the product deeply can make quickly make decisions on how to handle those edge cases. It is dangerous to assume that all users are like yourself, but some things are clearly non-sensical if you use the product.
My team was once assigned to build a comments system. The requirements showed some nested comments and how to handle them. The requirements did NOT specify a max nesting level. One of the developers on my team automatically assumed that the nesting level was infinite and spent a good amount of time working with comments that were nested 100 replies deep. Handling a low level of nesting is easy. You just add a parent id column on your comments table and call it a day. Handling a nesting level of 100 is a lot more complicated because now you have to worry about how to get a query that's actually performant.
Given who our users were and what our application was used for, there was no need for more than 2-3 levels of nested comments. This would have been obvious to anyone who had to seriously use our application for their tasks. It was less obvious to a developer who was never going to use the product themself. The result was a lot of wasted work and a delayed development timeline.
You may be thinking "well why didn't they just ask?" That's a possibility for sure. Consider what that takes though. You could send a Slack message, but the person you send the message too could be a very busy individual. It may be hours (or days) before you get a response. What do you do then? Keep plugging away with your bad assumptions? Procrastinate while waiting for a response? Context switch to doing something else? In any of those cases, there's still plenty of inefficiency and delayed development timelines. The same is true if you replace the Slack message with a meeting or an email. Different mediums, same problem.
The application quality suffers too. I've looked at countless accounting applications that use a message queue to generate an amortization table. The use of a message queue is obvious based on the UI and the long delay to get the table. The developers probably did this because an amortization table looks big. It looks like it'll take a lot of processing power. Handling an amortization table with thousands of rows could take a lot of processing which would justify having workers behind a message queue.
Consider the use case for an amortization table though. These are built for loans. A 30 year loan is going to have 360 months. That's the upper limit of most use cases. An extreme use case would be 99 years at 1188 months. Anything past that is extremely unlikely and you can easily just say your application doesn't support it without losing any customers.
360 rows of data with 10-30 decimal fields is not a lot of data. I have a sloppy algorithm that runs at O(n^3) and it still generates the table faster than you can type the inputs. Our application provides this data instantly while other applications can force users to wait a minute or more. Understanding the use case behind this data results in a better experience for users than trying to handle unrealistic edge cases that only a programmer would think of.
How can we mitigate these issues? A common way to structure a development organization is to have a separate product and UX team that reports to a CPO/VP of Product while having all your engineers and QA in a separate team that reports to the CTO/VP of Eng. The primary communication channels between these teams are documents, JIRA tickets, and meetings. The product team does the work to understand the users and translates that to a requirements document for the engineering team to implement.
Having two teams where each only understands half of the story does not make everything whole. In the case of amortization tables, an engineer likely said "we need to do this for the application to scale" because they didn't understand how users would use the application and planned for unlikely edge cases. The product manager felt like they had no choice but to agree because they don't understand the technical details.
The alternative is to stop putting these teams into their respective silos. Developers should do the work to understand users. This may mean spending more time talking to users. It could also mean spending time actually being a user. I take time to do some accounting work for this very reason. Product managers should have a place in technical design meetings as well. Good product managers understand system design even if they can't write code (or know what system design is). The ones that don't tend to write shoddy requirements anyway.
Does this cause responsibilities to overlap? Sure. Does it potentially make your org chart look a little messier? Sure. Will your teams be resistant to this workflow? I've encountered plenty that will and plenty that won't. Is all of this going to let you build better software in less time? Absolutely.