Business Analyst - one of the most harmful roles in software engineering

Introduction

When I say Business Analyst (BA), I mean a person whose primary purpose in the company is to translate business needs and ideas into something that a software development team can execute on (requirements). These requirements are in my experience usually represented as a story, but you might be familiar with similar terms like feature or initiative. In the context of this post however - a story, feature or initiative are all the same thing, we call them requirements.

This post is about explaining why I find that particular role more harmful than helpful for an organization. Since all roles exist for a reason, usually to solve a problem, I find it helpful to first zoom out a bit and take it from the start, how a company starts, because then we can see why roles are needed as the company grows.

A similar role is Requirement Handler. So for simplicity in this post I will refer to both Requirement Handler and Business Analyst as just "BA".

Where a company naturally starts

All companies start with just one or a few people that founds the company. You have someone acting as the CEO, and one or more persons that help in other ways. Since we're talking about software development here, it can be the case that the CEO is also the only software developer there is. Or maybe there's a couple of people, and the CEO is not developing but we do have some other software developers. Either way, the person taking all the decisions is usually the CEO. So in this sense, the CEO is clearly the "business representative" if we were to see it from a regular software development team. We don't necessarily need more roles to start developing, so let's see how communication flows in such a small company.

The distance is one arm-length between the CEO and the developers writing the code. If the company just consists of 1-5 people, likely they all talk to each other and discuss ideas and design before starting to write code. Already here we can see an absolutely crucial attribute of this company, which is incredibly important to companies operating in a fast-pacing world. The distance between business and developers is 0. If the CEO and the developer are the same person, the distance is 0. If they are different, but still talk directly to each other, the distance is still 0, meaning no additional step for the developer to talk to the core business person of the company.

The process of taking a business decision, translating it into working functionality is likely a matter of a few verbal sentences between the CEO and developers before they write the first line of code.

Even though we don't necessarily create any stories and put them into our issue tracking system, if we would label the process this is typically called requirement handling in software development. It's the process of translating business needs into something developers can understand and execute. As is typically the case, neither the business nor developers fully understand everything about the business. It's a learning process. But at least they talk directly to each other in this small organization. When a problem shows up, they can solve it relatively quickly. The reason for this is because the feedback cycle is very short (0 in this case).

Requirement handling when the company grows

All of a sudden, the company is a bit bigger. 1-5 developers and a CEO turned into 50 people over the years.

As the organization grows, business seem to not find themselves having much to do with IT. People start to get busy and business realise they must free up time from developers and have someone that understands business a bit better than a regular developer, so they come up with something called a Business Analyst.

Business have a lot of ideas, and they need someone that understands both the business and the IT. Someone that has the time to join all the business meetings but also can join the software development teams and see how things are going for them. Business thinks they need someone that can translate their ideas into something that the development teams understand and can execute on. Hint: now we're heading in the wrong direction.

The business analyst's responsibility

The BA is usually part of the development team, and instead of letting developers talk directly to the business people it is the BA's job to do that. Usually, the BA does not have much mandate to make decisions, effectively making them a proxy between business and developers. One additional person that must have the capability of having an absolutely great technical understanding (in order to keep business very well informed about technical problems and decisions that arise) but also having the capability of understanding the business in great detail. Some BA's do have a technical background, even as a software developer, but as you know, being a developer 5 years ago is not the same thing as being a developer today. The technology (frameworks, tools, architectural patterns) changes too quickly. So if you're not working as a developer today, then you're not a developer and not well suited to give much technical advice.

The BA is available at all times for the development team, will answer all kinds of questions, and whenever the BA is unaware of the answer, the BA acts like a proxy and forwards the question to business and then eventually brings the answer back to the development team. At a first glance this might seem to be very effective: business can talk to someone that understands them, and developers have more time to write code, since the BA now will do quite a lot upfront analysis phase and prepare a well-written story and the developers can really just execute. This is good, right? On a theoretical level it can sound good, and (unfortunately) on a practical level it works too in the short run. But it comes with a cost: the feedback cycle between the business and developers gets longer. The BA is another layer of "communication", which is unhelpful in most cases, as we will see. We have now gone from 0 to 1.

An analogy: How to efficiently distort information

Have you ever been at a party where you play a game where you sit around a table, and a person starts to whisper a word to the person to the left, so no one else can hear, and then that person continues. When the last person is reached, that one tells openly what he/she thinks the original word was. Very often the final word (although whispered carefully without any intention whatsoever of distorting it on purpose) is usually not anywhere near the original word. Alcohol included or not, the final word tends to be different from the original word.

This is a simple analogy of what naturally happens when you increase the number of people between the idea (the original word in the game) and the outcome (the final announced word in the game). Idea here refers to requirement, and outcome refers to the built product.

Try asking a couple of more friends to join the game and compare the idea to the outcome now. Heck, it got even worse!

Try playing the game with just you and one friend, and then compare the idea to the outcome. Heck, the final word was probably the same as the original!

But wait a second, in an organization we don't whisper to each other, we write things down. And yes, you're correct. If everyone in the game now is allowed to write the word down, you essentially could just pass that piece of paper between everyone. In this scenario, clearly the outcome is identical to the idea.

But the game is very simple and has finite rules, software development is complex, rapidly changing and the real world is a very dynamic environment.

There's another important characteristic of the game: you are not allowed to change the idea (initial word) during the game.

What happens in software development? What do we do when we have a CEO, a product manager, a business analyst and a developer? When the business analyst is in a meeting with the developer and simultaneously in another room the CEO changes her mind and wants a slightly different version of the idea? The time spent between the BA and the developer could be seen as waste even though we haven't even started to build something yet.

As we can see, the more people (layers) we add between the idea and outcome adds confusion. It's true both for the game and software development.

Turning developers into programming machines

It's interesting to see how much less responsibility developers usually take when a business analyst is the person that "knows the business" and translates real business requirements into stories for the development team.

The problem I've experienced multiple times is that when you have a BA in place that is the single person you know how to reach and ask questions, the more detailed questions you ask, the fewer questions the BA usually is able to answer. This is very bad. But it's also understandable. A BA is not the business, but instead just represents the business, and cannot possibly know all the details.

Whenever you as a developer ask a question and don't get an answer soon enough you eventually just have one option really: keep coding with the information we have. A devastating approach for the design of the systems in the long run of course.

Also, when developers are told to execute on stories that are well written it puts them in a very passive mode. It's a huge difference when a developer must crunch the knowledge himself and when the information is served on a plate. The former case forces the developer to crunch knowledge to understand the business in order to create a system that supports that business. In the latter case the developer does not need to understand the business, but can instead just execute on the story. I've seen this plenty of times, where developers really do not care or bother to learn the business, they just add the button on the page that the business asks for. This usually has less to do with the attitude of the developer, and more to do with the organisational structure. More (unnecessary) roles between the idea and the outcome tend to cause a sense of lost "why". We don't really always know why we build what we build, but we do it because someone tells us to. This might sound fine, but it is a problem because if we as developers don't know the "why", we cannot possibly know the importance of what we're building and it will be harder for us to know how to prioritize technical decisions.

Short feedback cycle is crucial

So essentially, the feedback cycle is too long between business and the developers. And a short feedback cycle is crucial in modern development teams, because the shorter the cycle (one real business person talks to one real developer, i.e. a 1-1 relationship) the better the information we as developers have when taking our decisions of how to design the system. Another important aspect that requires a shorter feedback cycle is the design of non-happy cases, such as error handling. Business (and the BA's) typically have thought through some common error cases, such as when the customer does not have enough funds on their credit card, but they cannot possibly think through other transient network errors, because they don't know how the software works, only the developers do. And these days, when UX is important to most companies, this matters. We must have a short feedback cycle and business must understand the consequences of when things go wrong as well. This "understanding" is very tedious when we have gone from 0 to 1.

Keeping a short feedback cycle is probably one of, if not the most important aspect to keep an organization flexible and be able to adapt to new requirements at a reasonable pace.

Inspiration disappears

At some point in time, developers will stop being enthusiastic about the business, and turn into a more passive mode to just crunch the stories they are told to deliver on. When all you get is already-written stories and the feedback cycle is too long, you'll stop asking important questions because you know that it is just too cumbersome for the BA to bother proxying the question forward to the real business people and then get an answer back. We need these answers in our refinement meetings and not two days or 1 week later. This puts developers in a passive mode that just executes what's written in the stories. Bad developers? Not so much, but rather basic human frustration. Once or twice you could wait to get the information, but when this becomes the norm, the cost of waiting is too expensive, so we fall back to "just doing what we're told". The long feedback cycle here is the problem.

The ability to adapt quickly or not

One of the problems when designing a system that meets customer needs is the ability to constantly explore, or be open for change. No matter if you have a very modern like data-driven culture, or if you have an "idea" based organization where business people guess what customers might want, and we build that, it is still a very attractive attribute for a team to be able to adapt quickly. With "quick" in this case I mean to change a decision within days and not weeks. Could be a small decision or a big decision. It could even be the colour of a button in the frontend, or where the chat module will be located in the mobile view of the web site your team is building. These (from a customer perspective) very small features can take weeks for slow organisations because of how the information flows and how decisions are taken.

As soon as the developer realizes a potential problem, and wants to suggest a change - no matter the reason - information must flow quickly in the organization. If the work is already in progress, i.e. the developer is actually working on the story and writing the software, any change must be aligned with business people very quickly. When you have a proxy person in between (the BA) you must first convince the BA that whatever reason for change you have is good. The BA must feel that it is worth doing it now, and then send your request to the appropriate business person for approval.

Approval in this case doesn't mean it must be very formal. But usually, the business person at least must give some kind of approval or "ok" before changing the story, and this can take time.

A team with a real business person as part of the team can take this decision very quickly. That's a very flexible team that can adapt to new information quickly. And that's a very healthy attribute for most software development organizations today.

When BA's makes decisions that are naturally the responsibility of a software developer

Now we're heading into even worse territory.

I've been in some discussions where a BA tends to get into more and more technical details. Here are a few of them:

1. When to make an API call
2. Structure of JSON
3. When an event is supposed to be published (in an event-driven architecture)

The list can be long. This is harmful, and it is the responsibility of the BA to actively not get into these discussions. A BA must in this case be very conscious about the different responsibilities and understand that developers are the ones that have the competence to make these technical decisions, not someone else. Even if you as a BA have prior experience as a developer you are not a developer today - you are a BA - so pass that responsibility to the developers.

Software development is a team effort

Software development is not a one-man show, and it is absolutely crucial that the team building the software consists of good developers but also very good domain experts (the business people). It is the collaboration between the two - say a financing expert - and a developer that gives the developer good conditions to design a good system according to business needs.

Because of this collaboration, it is an absolute necessity that the developer:

1. Can talk directly to the financing expert
2. Will talk directly to the financing expert

As soon as someone else comes in the middle and acts like a proxy, information instantly starts to distort.

Software development is about solving a problem

The primary responsibility of a developer is not to write code, but to solve a specific business problem, sometimes through writing a piece of code.

For any problem solving, it is important that we know exactly what problem we are solving, and why we are solving it right now. In my experience, it is not uncommon that a BA does not have the entire picture of the problem. The stories they provide to the team, or the information, is essentially their own understanding of the business problem. Once you start asking questions such as "why do we build this?", you'll see that even the BA might not have a full understanding of the business problem. This is very bad, because it means we're spending hundreds of hours building systems that hopefully solve the right problem, at the right time. A developer rarely has the entire picture either, but if the developer can talk directly to business then the feedback cycle is short enough that it's worth asking all the questions, creating a bigger understanding of the problem to solve.

But our BA is really very good

It does not matter if the BA is good or not, because what matters the most is the feedback cycle between development and business ideas. For a developer to be able to do a good job writing code it's necessary for that person to get answers immediately to all kinds of questions asked by the developer. Going back to our example of how a company starts, there's really only ever two persons that must understand the business problem, and that's the business person and the developer. Why? Because it's the developers understanding of the business problem that gets released into production, not the BA's.

Summary

There will naturally be hard problems to solve in any business. We can't do much about that. What we can do is to let our roles in software development evolve as to what we need. To have a person that neither understands business perfectly nor understands software perfectly, and let that person be the glory key between business and developers can cause more harm to a company than it helps. If nothing else, it will increase the feedback cycle as we have seen which is a problem.

Let developers do their job, they can very well understand business, and have lots of questions to be asked.