The advent of software product management

The concept of product management originates from a 1931 memo by Procter & Gamble’s President Neil H. McElroy. McElroy, requesting additional employees who would take on the role of managing products, packaging, positioning, distribution, and sales performance. These would be the first product managers in the world of business, responsible for ensuring that a product meets the needs of its target market and contributes to the business strategy, while managing a product or products at all stages of the product lifecycle. To this day, product managers spend most of their time doing what McElroy had wanted them to do – analyzing product distribution, optimizing distribution strategies, diagnosing, and solving distribution issues, optimizing product positioning and product marketing, and collaborating with regional distribution managers.

Software product management simply adapted the wisdom from generic product management, specifically for digital products. This adaptation is of particular interest, for several reasons

• a digital product specification is far more nebulous than, say a pack of detergent. In fact, exhaustively defining a digital product is an impossible task, and attempting such a feat is the reason why “waterfall” software development efforts have essentially been abandoned.

• a software engineering team supporting a digital product is labor intensive and delays and errors in product management decisions have a very large negative impact on project outcomes.

• somewhat counterintuitive to the above statement, a high-performing software engineering team releases product features very fast, and so the risk of rapidly building the wrong product is extremely high as well.

As product management has become more complex – by several orders of magnitude by some measures – the traditional product manager’s job of managing the external aspects of the product around marketing, positioning and distribution, as well as learning from and adapting to consumer signals has grown exponentially as well, and they are often supported by large teams of product analysts and market researchers and all manners of project, program and portfolio management experts. Simultaneously, product owners have less time for their software engineering teams who are the vehicle for their product development, at a time when the timeliness and clarity of the communication of product decisions is of paramount importance.

The software product management discipline has therefore seen the advent of the role of product owner, in addition to the product manager, whose sole accountability is to serve as an effective communication channel between the product manager (and team) and the software engineering team developing the product.

Product Manager vs. Product Owner

Like with any discipline, product management is susceptible to overloading of terminology, and the roles are product manager and product owner are often confused. To be clear, the two roles are entirely different, although both could be performed by one individual.

While the product manager has a strategic and long-term perspective with a strong focus on the market success of a product, a product owner aims to maximize the business value of the product or increment created by an agile project which can include benefits within an organization and does not explicitly relate to a product's marketability. Therefore, a product owner focuses mainly on the development of a product and may limit their product owner responsibilities to the duration of a project. The product manager role, in contrast, requires a long-term perspective of the market and product line.

An effective product manager’s “superpower” is their domain expertise and vision, strategy and plan for product positioning, marketing, distribution and monetization (not generation of revenue, necessarily, but at least the generation of funding to sustain the product). This superpower may be achieved by a team of experts, but ultimately, they have authority over the entire product management team and must have legitimacy among all product stakeholders. A product manager’s superpower may be boosted by human-centered design principles such as user research and usability testing, and through the principles of lean manufacturing but their understanding or adoption of other software engineering practices and processes rarely have any impact on their effectiveness.

By contrast, an effective product owner’s superpower is their ability to translate product goals and decisions, often perceived as changing frequently, into an actionable backlog of prioritized software features for the software engineering team so that

• the highest priority features are being worked on

• the features being worked on are sufficiently understood to not require additional discussions between the designers and developers and the product subject matter experts

• the engineering team has sufficient clarity on the acceptance criteria of the features so that the chances of rejection and rework are minimized

A product owner may benefit from their domain knowledge, and they must have the authority to make the final call on the product features, as well as the legitimacy among the product stakeholders, but for the most part their superpower comes from their ability to understand the product roadmap enough to set priorities for the engineering team and a commitment to elegant but robust implementation of features, while also giving the engineering team the leeway to make engineering investments that will speed up delivery in the future. At Caravel Labs, we love the Product Ownership in a Nutshell video that explains what an effective product owner is expected to do. The product owner role was specifically created for software product management, and this function is now considered an integral part of software engineering practices. Most software engineering process frameworks today have a very clear definition of the product owner role. For example, Scrum Alliance’s Scrum Guide has a very specific description of the product owner role.

While both product manager and product owner roles may be fulfilled by a single individual, they are not substitutes for each other. Unless both are fulfilled adequately, the engineering team will either struggle to release a quality product in a reasonable amount of time, or will rapidly build a product that struggles to meet its intended business purpose. While a product manager could be part of a larger team that may function through consensus, a product owner is always an individual and not a committee.  

Improving product management capability

Assessing the effectiveness of the product management function is usually a hard task as there is no objective measure for it. One can only measure the relative improvement or deterioration of this capability for a team over time. Furthermore, an organizational culture where basic assumptions are that the problem domain is unusually complex, or the stakeholder dynamics are unusually complicated, low levels of product management capability become part of the norm.

It is therefore more effective to be on the lookout for the common fault lines of product management and address them either proactively through processes and tools, or reactively detected during retrospectives and then rigorously eliminating them.

All humans, and therefore all product managers and product owners are susceptible to cognitive bias and therefore fortification of their judgment with effective user research and usability testing helps ameliorate the situation significantly. Caravel Labs consulting teams invariably include a UX Generalist who is professionally trained in the latest practices in these fields and serves to bolster the product management capability provided they are given access to the appropriate user representatives.

In a software development project, the efficient communication of which features to build first, their description and acceptance criteria is critical to the speed with which the right product will become incrementally available to the users. It is therefore critical that processes and tools that support the engineering team also support the product owner in being most effective. Caravel Labs product teams accomplish this in the following ways:

• During the first sprint, a human-centered design exercise, co-hosted by the UX Generalist and the Lead Engineer, ensures that the initial product roadmap is established. No matter how complex or large the overall effort, the product roadmap includes milestones no longer than three months apart that plot the path to the ultimate goal. These milestones, which we call seasons, have clear business outcomes associated, that meet the SMART criteria. The nearest milestone is broken up into a prioritized backlog of features that the product owner deems as required to meet the season or milestone goal.

• During the first sprint and thereafter at least every two weeks, the product owner, assisted by the UX Generalist and the Lead Engineer refines the backlog to ensure that the features and user stories that comprise the features meet the INVEST criteria. This allows the backlog for each milestone or season to be easily estimable by the team, ensuring both the product management and product development stakeholders have the same level of confidence on what can be achieved and creating an environment where right product priority and implementation decisions can be made to maximize chances of success.

• All backlog refinement meetings between the product owner and the engineering team are timeboxed and never exceed more than two hours a week. This ensures that the engineering team is focused on development rather than attending meetings, but equally importantly, allows the product owner to use most of their time to work with the product manager, user representatives and other product management stakeholders, as necessary. The Lead Engineer provides the product owner with a Definition of Ready for the features and user stories, a rubric for the details that the engineering team expects, thereby giving autonomy to the product owner.  

As mentioned above software engineering practices, processes and tools can help avoid but cannot eliminate product management pitfalls. Lack of a product strategy or having an incorrect product strategy – for example, not prioritizing how users will gain access to the product or how they will pay for it – is not something that an engineering team can help with, as it has no choice but to implicitly trust the product manager’s judgment on this matter. However, some common symptoms of ineffective product management are known and when detected and discussed during sprint retrospectives, serve as starting points for process improvements:

• Product decisions influenced by a powerful internal stakeholder, overruling data from user research or usability testing may relieve the team of immediate pressure but ultimately introduces product risk and kills team morale.

• Product decisions influenced by a single internal stakeholder who is confident about their gut instinct over the data from user research or usability testing can similarly introduce product risk.

• Flip-flopping on product decisions leading to wasted development efforts.

• Focusing purely on new features at the cost of allowing the engineering team to make investments that speed up future delivery, leading to a reactive cycle that hinders productivity and innovation due to piling technical debt.

• Delaying the release of completed features to users until the latest new feature is completed, shifting focus to squeezing out “productivity” from the engineering team rather than releasing early and often in order to solve immediate user problems and have more opportunities to learn from them in order to build a more efficacious and resilient product.

Caravel Labs consulting teams are good at detecting these product management inefficiencies, bring them up in sprint retrospectives, and if required escalate. There is no one size fits all solution to eliminating or even assuaging these inefficiencies, but a healthy discussion of these observations helps drive changes in the overall product management function that improve the situation drastically.

Last but not the least, just like Caravel Labs’ Engineering function works relentlessly to automate processes and build tools that empower developers, by eliminating toil, providing effective guard rails and assist in building good habits, Caravel Labs’ automated processes and tools also empower product owners in the same way.

We would love to learn about how you plan to improve your product management capabilities and share our thoughts in greater detail. Drop us a line any time.