It is not always easy or pleasant to talk about estimates. On the one hand, it’s a fun activity of trying to look far enough into the future. On the other hand, high-level estimates are often taken as a commitment. It gets shared with the bigger org, the sales organization (read customers), and ultimately with the company leadership.
High-level estimates in software engineering are preliminary projections of the time, effort, and resources required to complete a software project. These estimates provide a broad overview during the initial planning phase and help stakeholders understand the general scope and feasibility of the project before delving into detailed planning, commitments to customers, and execution. High-level estimates also guide decision-making and resource allocation, setting the foundation for the project’s direction and expectations.
So, while subject to refinement as more information becomes available, high-level estimates set the early expectations for project timelines and resource budgeting.
However, high-level estimates (HLEs) are often a dreaded topic for engineering leaders due to several reasons:
- Uncertainty: High-level estimates are inherently uncertain because they’re based on limited information and assumptions. This uncertainty can lead to skepticism among stakeholders who fear inaccurate predictions.
- Scope changes: Projects can undergo scope changes, especially while HLEs morph into more detailed roadmaps while improving project requirements. This process can render initial estimates irrelevant and be frustrating for both development teams and stakeholders who were counting on those estimates.
- Pressure and commitment: High-level estimates are typically prepared as a projection, approximating the time it will take to deliver. Yet, often, HLEs are used by stakeholders to base firm customer commitments.
- Blame game: Inaccurate estimates might result in blame being placed on the development team for delays or cost overruns, even if the calculations were provided with the best available information at the time.
Abbreviations used in this post
- HLE ﹣ high-level estimate
- TPO ﹣ Technical product owner
- COF ﹣ cost of failure
- EM ﹣ Engineering manager
My situation
A while ago, I was hired as an Engineering manager (EM) at a scaleup company to set up a new development team in a new location. Here’s what the starting point looked like:
- New location (and country): no physical office available yet
- New team: just me and the TPO
- New product: we only had high-level requirements at that time
Among other things, together with the TPO, we needed to provide a more detailed roadmap based on the engineering estimates for 2-3 quarters. The estimates would also inform headcount and hiring profiles.
It was one of those situations when decisions need to be made based on limited data, and, frankly, I find enjoyment in the challenge this presents. It’s the intrigue of navigating through uncertainty, the nudge to think critically, leverage available insights, and creatively problem-solve to make the best possible decisions amidst the complexities of limited data. Time, as a crucial factor, was fortunately on my side in the above scenario.
Detailing high-level estimates
Soon enough, I had a clear understanding that a guesstimate based only on the gut feeling of the engineer-in-me won’t do. It may still work in a scenario when, for example, the time is limited to hours. Still, the result is likely a low-confidence / low-reliability input for the business.
To work out a reliable high-level estimate, I looked at the following inputs:
- Clarity of requirements and their level of detail
- My own domain knowledge (in this case, the airline industry)
- The technical prowess of the team
- Required domain knowledge
- Need to onboard into existing codebase and legacy systems
- Tech stack differences and technical knowledge gap
- Company rhythm (perceived)
- Cost of failure (project delay)
Below, we’ll go deeper into each of the five inputs above.
Clarify of requirements
This point is self-explanatory. The clearer the requirements, the better we all understand what to do.
When requirements are ambiguous or lacking, it becomes challenging to determine the full extent of the work involved. This absence of clarity can lead to underestimation or overestimation of effort, affecting the overall estimate and team happiness.
Domain knowledge
Domain knowledge plays a crucial role in informing high-level estimates in software engineering. Domain knowledge is your bridge to understanding project requirements [better].
In my case, I had some airline industry knowledge at that moment already, thanks to beginning the onboarding a little before my first official working day. When we started, I was fortunate to have a professional and experienced technical product owner (TPO) on my team. In hindsight, having a skilled TPO who understands what we aim for and why made it much easier to define the “how”.
In summary, domain knowledge empowers you to provide high-level estimates grounded in a holistic understanding of the project’s technical intricacies and potential challenges.
Technical prowess of the team
Here’s what I mean by the technical prowess of the team: it’s the collective skill, expertise, and experience of the team members when it comes to understanding, designing, implementing, and maintaining a software system (or an application).
It includes but is, of course, not limited to the following:
- Work estimation: A team with higher technical skill and experience is better equipped to assess the complexity of tasks involved in a project accurately. They can better understand the intricacies of the software requirements, potential challenges, and technical dependencies. This understanding leads to less uncertainty in estimates, improving the resulting HLE (and reducing the risk!).
- Mitigating unforeseen issues: People who have been through multiple software projects know they rarely go exactly as planned. And so, the team’s technical prowess affects their ability to adapt and resolve unexpected challenges that arise during development (like an unaccounted-for dependency on the external DevOps team).
Skilled teams can more effectively troubleshoot issues, leading to shorter resolution times and reduced impact on the overall project timeline. - Software quality: A technically proficient team is likely to prioritize code quality and maintainability. While this might not directly impact the initial development time, it can influence estimates in the long run. A team that takes the time to build a solid foundation will spend less time on maintenance and bug fixing later.
In summary, the team’s technical prowess is a vital input when generating high-level estimates in software engineering.
The above assumes you already have a team. In my case, I had just started the hiring process at that point.
Company rhythm
For the purpose of this post, “company rhythm” refers to the unique pace, culture, communication style, and working patterns of a company. It encompasses factors that influence how work is planned, executed, and delivered within the organization. Usually, you’d have limited visibility on that because what happens in other teams or departments would not always be accessible to or shared with, a more general audience.
Yet, even if you’re new, it is still possible to take a measurement of what the work pace is like at this company.
High-level estimates need to be aligned with the rhythm to accurately reflect the company’s capabilities, constraints, and priorities. Understanding and accounting for the company’s rhythm in high-level estimates can lead to more realistic project planning and successful outcomes. And less stress!
Cost of failure
The “cost of failure” is an essential input that informs high-level estimates in software development. It helps answer the “what if… ?” type of questions. I’ll refer to it as COF below for the sake of brevity.
Here’s my attempt at defining the COF: It’s the potential negative consequences and expenses associated with not delivering a software project on time, within budget, or meeting the desired quality standards.
Incorporating the cost of (delivery) failure into your high-level estimates helps provide a more accurate and realistic assessment of the project’s scope, timeline, and, to some extent, risks.
- Resource allocation: If the cost of failing to meet a deadline is substantial, you might decide to allocate additional team members or make an advance investment in tools and technologies that can expedite development. On the other hand, if the potential COF is lower, you might prioritize optimizing existing resources rather than increasing them.
- Risk assessment: When estimating a software project, it’s important to consider the potential risks that could lead to delivery failure. Things like unforeseen technical challenges, scope creep, resource limitations, or changes in project requirements.
- Contingency planning: Including the COF in your estimates allows you to allocate resources for contingency planning by setting aside extra time, budget, or personnel to address unexpected challenges that could arise during the project. By acknowledging the possibility of failure and planning for it, you can better handle unforeseen obstacles without significantly affecting the project’s timeline, quality, or your team’s “internal brand”.
Things I didn’t take into account (initially)
- Hiring warm-up time. It took a little while to finalize the role descriptions, perform sourcing calibration exercises, and settle on the interviewing process.
- External dependencies on teams like Design or DevOps. A good thing to do would be to inform them in advance and make clear agreements on resource allocation. Doing that helps prevent roadmap shifts, unforeseen delays, and frustrations in general.
Summary
In the complex world of software engineering, high-level estimates (or HLEs) play a critical role in guiding project planning and resource allocation. They provide a preliminary projection of the time, effort, and resources required for a software project, shaping expectations for timelines and outcomes.
However, engineering leaders often find HLEs daunting due to inherent uncertainty, pressure for commitments, potential scope changes, blame games, and the challenge of conveying complexity to non-technical stakeholders.
Based on my journey, this blog post explored the factors influencing high-level estimates in software engineering. Clear requirements, the team’s technical prowess, domain knowledge, the company’s rhythm, and the cost of failure are among the inputs that shape accurate estimates. Or, to be more precise, it allows for estimates as accurate as possible while working with limited data.
Thanks for reading, and let me end the post with this quote:
I love deadlines. I like the whooshing sound they make as they fly by.
Douglas Adams
