Getting a transformation approved is hard. Proving it worked is harder.
About a year after the first increment ships, someone in finance or on the leadership team will ask a simple question: is this working? It is tempting to answer with activity: we have converted 340 programs, refactored 60 files to SQL, and documented most of the order management application.
Those are real accomplishments. They are also not an answer. They describe what the team did, not what changed for the business. A CFO cannot tell from a program count whether the investment is paying off, and neither, honestly, can the IT leader presenting it.
A multi-year transformation is far easier to defend when the outcomes and the starting point are decided before the work starts. This post covers the measures that matter, how to establish a baseline, and what the numbers should tell you by the end of year one.
Why Activity Metrics Mislead
Activity metrics are tempting because they are easy to collect and always go up. Every month, more programs are converted. Every quarter, more code is documented. The chart looks like progress.
But activity is a means, not an end. Futurization™, our one-of-a-kind methodology for building new capability on top of proven IBM i applications instead of replacing them, exists to make the business faster, safer, and less dependent on a shrinking pool of expertise. Converting a program to free-format RPG matters because it makes that program easier to change and easier for more people to understand. If changes still take as long, break as often, and depend on the same two people, the conversion has not yet delivered its value.
The measures below focus on those outcomes. Most can be tracked with data your team already has or can start collecting with little effort.
Start With a Baseline Before the First Increment
None of these measures mean much without a starting point. A change lead time of three weeks is good news if it used to be eight, and bad news if it used to be two.
Can your organization answer basic questions about its current state today? How long does a typical change take from request to production? How many changes required a fix after release last year? Which programs can only one developer safely modify?
If the honest answer is “we have a sense, but no numbers,” that is where to start.
Capturing that baseline is the single most valuable thing you can do before Futurization begins. It does not need to be perfect. A reasonable estimate built from your ticketing history, change logs, and a few conversations with your senior developers is far better than nothing, and it gives every later measurement something to stand against.
Measure 1:
How Long a Change Takes to Reach Production
If you track only one number, make it this one.
One of the best-known frameworks for measuring software delivery comes from DORA, a long-running research program now run by Google Cloud. Its current model uses five metrics grouped into throughput and instability, and one of the core throughput measures is change lead time: the time from a code commit to a successful production deployment.
For IBM i organizations, it is worth measuring a wider window as well. The business does not experience your commit date. It experiences the gap between asking for something and getting it. Tracking both numbers, commit to production and request to production, shows you where time is actually being lost. If commits move quickly to production but requests sit for weeks before anyone starts, the constraint is capacity and knowledge, not delivery.
Futurization shortens both. Code that is easier to read and change moves faster through development, and broader team capacity means requests start sooner.
Measure 2:
How Often Changes Break Something
Speed only matters if it holds up. DORA’s instability metrics cover this side: change fail rate, the percentage of deployments that cause failures in production, and deployment rework rate, the percentage of deployments that are unplanned work to fix bugs.
An early DORA finding is that speed and stability are not a trade-off. Its research found that high performers excel at both. That is a useful standard for any transformation. If change lead time is dropping but change fail rate is climbing, the team is moving faster by taking on risk, and the gains will not last.
On IBM i, this measure is especially telling. Applications that run invoicing, order processing, and overnight batch work have very little tolerance for regressions. A transformation that improves speed while holding stability steady, or improving it, is one the business can keep trusting.
Measure 3:
The Age of Your Backlog
Backlog size gets most of the attention, but age is the more revealing number. Consider two backlogs with the same number of open requests. In one, requests move through within a quarter. In the other, the oldest requests date back years. The size is identical, but the second backlog tells you the business has been asking for things it has stopped expecting to get.
Two simple measures capture this: the median age of open requests, and the age of the oldest 10% of the backlog. As futurization expands what the team can take on, both should fall. This is also one of the measures business stakeholders understand immediately, because they are the ones who submitted the requests.
Measure 4:
How Many Programs Only One Person Can Change
For IBM i organizations, this may be the most important measure of all.
Most IT leaders can name at least one program that, in practice, only one developer can safely modify. Sometimes the code is complex. Sometimes the logic was never documented and the person who understands it has always been the one who touches it. That concentration of knowledge is a business risk, and it grows every year the developer gets closer to retirement.
The broader market makes that risk harder to absorb. In Fortra’s 2026 IBM i Marketplace Survey, IBM i skills ranked as the top concern among respondents for the first time, cited by 69%. When the person who understands a critical program leaves, replacing that knowledge from outside is getting harder, not easier.
Start by identifying your most business-critical programs. For each one, ask two questions:
- Is its logic documented in a form someone else could use?
- Anyone other than the usual owner successfully changed it in the past year?
The share of critical programs that pass both tests is a direct measure of how much key-person risk you are carrying. Futurization should move this number steadily. Free-format code, modularization, SQL-based data access, and current documentation all make programs changeable by more of your team.
Measure 5:
Where Your Senior Developers' Time Goes
Ask your most experienced RPG developers how they spent their time last month. If the answer is mostly keeping things running, through production support, urgent fixes, and answering questions only they can answer, that is an expensive use of scarce expertise.
Track the rough split between maintenance work and new capability for your senior developers, even if it starts as a monthly self-estimate. As futurization reduces the effort required for routine changes and spreads knowledge across the team, more of that senior time should shift toward work that moves the business forward.
Measure 6:
Business Value Per Increment
The previous five measures describe the health of your development organization. This one connects the work directly to the business.
Futurization is delivered in increments, and each increment should have a business outcome named before it starts. Not “convert the inventory programs,” but what that conversion is meant to enable: a faster turnaround on pricing changes, a new capability the sales team has been waiting for, or a reduction in the time it takes to onboard a new customer.
At the end of each increment, check the result against the outcome. Over a year, this creates a record that leadership can follow without needing to understand RPG at all: here is what we said each phase would deliver, and here is what it delivered.
What These Numbers Mean for Developers
Developers are often wary of metrics programs, especially when numbers might be used to rank individuals.
These measures are not about individual performance. They describe how easily work flows through the system: how long changes wait, how often they need to be redone, and how much depends on a few people. For developers, improvement on these numbers shows up as fewer emergency calls, fewer changes that require tracking down the one person who understands a program, and more time spent building things instead of keeping things alive.
The technical work of futurization drives these numbers directly. Modularizing monolithic programs makes them easier to understand and safer to modify. Converting fixed-format RPG to free-format makes code readable to a wider group of developers. Refactoring record-level access to SQL makes data logic clearer and easier to change safely. And validating every change against a program’s recorded behavior before it reaches production is what keeps change fail rate low while lead time drops.
Where CoderFlow Fits
Agentic Futurization, powered by CoderFlow, affects these measures in two ways.
First, it moves them. CoderFlow’s AI agents work inside your own environment on real IBM i tasks, modularizing monolith, converting RPG to free-format, refactoring data access to SQL, and documenting what programs actually do.
Each change is compiled and validated against the original program’s behavior before a developer reviews it, and nothing is committed without human approval. That combination of speed and validation is exactly what improves lead time without sacrificing stability, and the documentation work directly reduces key-person concentration.
Second, it makes the work visible. Because CoderFlow works through source control and logs its actions, each task leaves a clear record from assignment to validated commit. That gives you a reliable source for measurements that would otherwise have to be estimated.
What Good Looks Like After Year One
There is no universal target for these measures. Every organization starts from a different baseline, with different applications and a different team. What matters is direction and consistency.
By the end of year one, a transformation that is working should show change lead time trending down, change fail rate holding steady or improving, the oldest backlog items starting to clear, a growing share of critical programs that more than one person can safely change, and a documented business outcome for every completed increment.
If most of those are moving the right way, you have an answer for the leadership team that no program count could provide. If some are not, you have something just as valuable: an early, specific signal of where to adjust before another year goes by.
Build Your Baseline Now
The best time to decide how you will measure success is before the first increment ships. A Profound Discovery assessment maps your applications, identifies where futurization delivers value first, and gives you a clear picture of your starting point.
To talk through how to measure success in your environment, reach out to Futurization@ProfoundLogic.com.