One has to take a peek into the inner workings of large IT consulting companies to see how much investment is made into "estimation accuracy", yet for all that effort and attention, less than a third of the estimates are ultimately accurate. For the other two thirds we often hear the stereotypical consulting excuse of “the customer changed the scope”, and the more recent version of this excuse “the customer did not provide a mature product owner”.

  

The problems that software engineering is primarily being used to address today are “wicked problems”, which are defined as limitless […] Yet the IT consulting industry continues to use archaic estimation approaches.

 

The problem is not with the focus on estimation, or even estimation accuracy per se, it is how we approach estimation philosophically. The science of project estimation was introduced into the world of software engineering in the 1960s, when computers were being used for the first time to automate well-defined and well-regulated business and industrial processes. Half a century later, information technology is being used to enable innovation, whether in the form of born-in-the-cloud startups solving the hardest problems facing humanity or legacy organizations seeking digital transformation to stay relevant in the face of radical change. The problems that software engineering is primarily being used to address today are "wicked problems", which are defined as limitless and cannot be described exhaustively. Yet, the IT consulting industry continues to use archaic estimation approaches intended at one time to computerize paper ledgers and mechanical assembly lines, even though it is well established that such an estimation process will mathematically tend to infinity.  

We do not know for sure why the IT consulting industry chooses to do this, often as a "best practice". However, our decades of experience in the IT consulting industry have given us plenty of opportunities to observe the incentives that normalize this behavior. The established business model of this industry is based on monetizing human effort, so there is about as much incentive to limit human effort (consulting hours) as there is for an automotive company to sell fewer vehicles or for a beverage company to sell fewer drinks.  

A Better Approach, Realign Incentives  

The founders of Caravel Labs were horrified to see the effects of this practice across the industry, and creating an alternative, was one of the motivations for them to build Caravel Labs. For starters, we decided to reduce the fundamental divergence in the interests of the customer and consulting company, by tying a significant portion of our consulting fees directly to the usage of the software produced. This creates a real partnership between our consulting teams and our customers to build the most effective and efficacious solution in the most efficient way possible. It creates the incentive for us to truly value, and therefore monitor usage patterns through telemetry, so that we can actively participate in product management decisions that increase user adoption.  

Continuous Light-Weight Estimation 

By refocusing on business outcomes, our new approach to software estimation is summarized as: 

1. Understand the target as a desired business outcome to be achieved by a certain date 

2. Identify the smallest engineering team that can deliver with all the necessary tools, automation, and empowerment to deliver incrementally and continuously 

3. Embrace change and innovation: re-evaluate features needed to meet desired business outcome, and re-estimate continuously to stay on target 

Each of these parts have enough content to carry their own articles. But let’s take a quick look at each. 

Understand the target as a desired business outcome to be achieved by a certain date 

The value of software is not in the number or complexity of features it has, but in its ability to eliminate tedious and error-prone manual processes for the users that takes away valuable time away from the creative work that they are truly meant to perform in order to achieve the intended business outcome on time. While the number of features and their complexity are important, the context in they provide value to their users is critical. After all, features don’t achieve business outcome, the users do. Therefore, we start with understanding the desired business outcome and the target date for this outcome, and then use this understanding to identify the minimum set of elegant features that can be robustly implemented to aid the users in achieving the outcome within the target timeframe.  

Since a lot of time can be spent in discussing and analyzing business outcomes, we limit the time we spend in these activities. Our scoping exercises are conducted in a single 1-4 hour “ideation café” session, followed by a few hours that we use internally to process our learnings. Of course, merely limiting an exercise is no guarantee of completion to an adequate level of fidelity to make a commitment. Therefore, these exercises involve vigorous use of concepts and frameworks that are proven to simplify otherwise complicated scenarios and quickly focus on areas that would benefit most from automation.  

• Human-centered design: That humans performing the work know best what their problems are and what tools would be most helpful. Therefore, the best tools come from learning from and with the users.  

• Design thinking: That the only features of a system that matter are what are valuable to the overall outcome, convenient enough to be usable for the users and feasible to implement within the targeted timeframe.  

• Value stream mapping: That every organization delivers value to its constituents through a series of activities, some of which add value along the way, while others are necessary but do not otherwise add value. Therefore, identifying activities that do not add value and eliminating them through automation frees up humans for value-added work and helps achieve the outcome sooner.  

Identify the smallest engineering team that can deliver with all the necessary tools, automation, and empowerment to deliver incrementally and continuously 

Small, empowered, and self-organizing teams are key to our approach. This helps us keep communication channels simple to normalize close collaboration and negotiation between our customer and the engineering team. In any software development project, regardless of implementation approach, each developer is entrusted with hundreds of design micro-decisions they make every day. This is one of the reasons we believe that placing the right incentives and empowering the developers by giving them a voice and the right tools is essential to hitting the target. This team is supported by the right tools and automation, which are built into our base service, for example we won’t be adding charges for implementing a continuous delivery pipeline or test automation. This is also the same team responsible not just building code but delivering working software to the hands of users to achieve your desired business outcome.  

Embrace change and innovation: re-evaluate features needed to meet desired business outcome, and re-estimate continuously to stay on target 

The engineering team works on a set rhythm that involves constant re-evaluation of the team’s progress towards meeting their target. Every week, our customers can visualize the team’s projection of their chances of meeting the target given the current set of features included in the work backlog. This information is produced by our engineers and communicated by them, no filters, we provide the facts. This gives our teams the opportunity to think, innovate and present our customer with options to help the team stay on target no matter the challenges that normally present themselves in software projects such as changing priorities and requirements. 

We don’t waste time debating estimates based on micro-decisions

Our engineers use lightweight software estimation methods, such as relative estimation to ensure that the act of estimating is fast and efficient, so that it can be done often, therefore avoiding analysis paralysis and stale estimates. Relative estimation allows us to quickly estimate a less familiar piece of work, based on a better understood piece of work by simply comparing the two. We use an arbitrary number to compare the two work items to account for other factors in addition to effort such as complexity and risk. 

We also embrace the idea of the cone of uncertainty and acknowledge that the earlier we are in the process, the less we know about the problem space and the margin of error in our estimate will be larger. We don’t waste time debating estimates based on micro-decisions because we know that 1) it’s too early to know the right answer, 2) we are confident in our ability to quickly re-estimate and often. Instead, we focus on broader architecture decisions that provide the right level of guidance to the development team to make better and quicker micro decisions down the line. We also recognize that the cone of uncertainty is more like a cloud of uncertainty that must be continuously shaped into a cone through continuous re-estimation and ruthless prioritization, staying on track to find the most elegant solution to achieve the desired business outcomes by the target date.  

We combine these relative estimates with our team velocity data to produce forecasts that detail how many development sprints we will need to achieve a desired business outcome. From there, the process we outlined in this article takes over to keep you informed and engaged in the team’s progress towards reaching the target. Let’s remember that the empowered, self-organizing team is properly aligned with your desired business outcome, so they will work with you to find the most elegant and right-sized solution to achieve it.

We hope that this article has answered many of your questions regarding how Caravel Labs approaches software estimation, we have an excellent track record using this process. If you are curious about the estimate for your next big idea, please contact us at info@caravellabs.com.