While many portfolio planning tools make great demonstrations and look fantastic by the time the 4th month has rolled around, they fail miserably at handling true complexity. To test for this, we need to look at features somewhat differently than just checking off a long list of features. We need to test how a portfolio planning tool handles reality, hundreds of initiatives, competing funding requests for the same initiatives, inter-departmental dependencies and a Senior Leadership Team that changes its mind twice a quarter.
The first test of potential portfolio planning tools is to map out the real work that product teams do, to categorize these based on the output, and to select potential tools based on this real work rather than on the often misleading vendor categories.
Roadmaps that survive contact with change
I judge a Roadmap more on the ability to modify it without some form of ceremony than on how good it looks in the first place. Thus, I test the ability to move an Initiative and have all downstream information change (e.g. Capacity changes, dependent Work changes, milestones become incorrect).
Check out what happens when you move a date. If the answer is to manually update 3-4 other views, then that tool will go stale in a few weeks and you’ll go back to using slides to manage work.
Multiple audiences, one source
Most PM tools allow you to define different views for different users. But unless you can define a view that shows them themes and outcomes and another that shows them your Epics in sequence, then 3 separate artifacts are required for planning and tracking, which creates a lot of unnecessary work.
Prioritization that shows its reasoning
Tools for scoring provide cheap and easy to set up evaluations. But they are useless if the team behind them is not transparent about inputs and cannot change the model once it has been set up.
Look for these specifics when you evaluate.
- Custom scoring fields with configurable weights, not a single hardcoded framework
- Visible audit history showing who changed a score and when
- The ability to compare a ranked list against actual funding and headcount, so priority order confronts real constraints
- Support for qualitative rationale alongside numeric scores, since the argument matters more than the number
These need to capture the reason behind a score as teams will be unable to defend their scores 6 months on if they do not have this information.
Scenario planning as a working practice
Annual planning in terms of comparing options for potential projects usually takes weeks in any organization. A good product portfolio management software enables you to work with alternative scenarios for your strategic plan. You can create a branch, change assumptions and view the consequences in an instant.
Practical questions to ask would be the number of variables you can include in a scenario (ie. Number of initiatives, assumptions, teams, etc). That the ability to move dates is included in a tool is table stakes these days. But to truly add value the tool must also allow you to change other relevant variables as well (ie. Team allocation, remove an initiative, model a delay until a hire can join).
Comparing what scenario support looks like at each maturity level
| Capability level | What the tool allows | Where it breaks down |
| Basic | Duplicate a roadmap view and edit dates | No capacity or cost modeling, so trade-offs stay invisible |
| Intermediate | Branch plans with resource and budget assumptions attached | Comparison is manual, and merging a chosen scenario takes rework |
| Advanced | Side by side comparison with variance highlighting and one-click promotion to the live plan | Requires clean underlying data, which many organizations lack at the start |
Product data that connects to delivery
Planning tools break when the data in them is entered manually. Two-way integration of a planning tool with the delivery systems of your teams is far more relevant than any of the typical planning features. Does it reflect reality?
The direction of the sync as well as the granularity of the information synced (e.g. Can strategic context sync down to engineers working on specific work items). A one-way integration (e.g. Delivery system to portfolio management tool) creates a reporting layer that will inevitably lag behind the work of the organization, so favor a product portfolio planning tool that writes changes back into the systems your teams already work in.
Customer evidence in the same system
Feedback (support themes & win-loss), evidence to support a justification for prioritizing something, should attach to the initiatives themselves, not sit in some separate research repository where only the Product Manager for that feature has read it.
Collaboration and portfolio visibility across the organization
These access patterns can be very useful during your evaluation of potential tools to ensure adoption by stakeholders outside of the product group.
- Read-only executive views that require no training and no license cost per viewer
- Comment and request workflows so stakeholder input lands in the tool rather than in email
- Cross-portfolio dependency mapping, showing where one group’s slip affects another’s commitment
- Permission granularity that lets confidential initiatives stay hidden from broad audiences
Portfolio visibility is key to seeing all your investments in one place and comparing how money and people are spent against your strategic goals. This can mean little more than aggregation of the various things that different teams enter into a product management work planning tool. However, if that aggregated view cannot then be compared against your targets for spending and allocation, then the tool is just reporting out against what already exists.
Weighing the decision
Use the prioritization tools to score how each candidate will handle your portfolio at its worst, not at its best. Do a pilot with real data and include in the pilot the worst group of people in your organization, worst processes, worst dependency problems, worst prioritization problems. If the tool can’t handle them then it will fail to meet any of your needs.
