Ask an IBM i shop what they are looking for in an AI coding tool, and the answer is rarely “a specific large language model.” It is usually something closer to “something that actually understands RPG” or “something we can trust with decades of business logic.” The model underneath is treated as an implementation detail, something the vendor handles.
That instinct is understandable, but it deserves a second look. Which model runs a given task on your codebase is not a minor technical footnote. It is a decision that affects accuracy, cost, and how much of your organization’s future is tied to one company’s roadmap. The question worth asking is not which AI tool to adopt.
It is who gets to decide which model handles your RPG and COBOL logic: you, or the vendor.
No Single Model Is Best at Everything
Large language models are not interchangeable, even within a single vendor’s lineup, and they are certainly not interchangeable across vendors. Some models are stronger at reasoning through long, tangled dependency chains.
Others are faster and cheaper for straightforward, well-defined tasks. Some have been trained on more code in general and reason well about unfamiliar syntax. Others have deeper exposure to enterprise patterns but stumble on niche language features.
RPG and COBOL sit in an unusual spot for this. They are not obscure languages, decades of code exist, but they are underrepresented in the datasets most general-purpose models were trained on compared to Python or JavaScript. That means model performance on IBM i work is genuinely uneven, and the model that produces the cleanest output on a fixed-format RPG III conversion is not guaranteed to be the same one that produces the best result documenting a complex CL job stream or refactoring a modular RPG IV program.
For a shop making a single, deliberate technology bet, this is a real problem. If your AI strategy is built around one model, you are accepting whatever that model’s specific blind spots happen to be, on every task, indefinitely, whether or not the vendor tells you where those blind spots are.
The Industry's Answer Has Been to Hide This Decision From You
The dominant pattern among enterprise AI coding platforms right now is automatic model routing. The platform decides which model handles which task based on its own internal logic, and the developer sees a single, unified experience.
IBM Bob works this way, it routes tasks across a mix of models, including Anthropic’s Claude, Mistral, and IBM’s own Granite family, based on the platform’s assessment of accuracy, performance, and cost for that task.
There is a real convenience argument for this. Nobody has to think about which model to use.
An industry analyst covering Bob’s launch put it well: automatic routing “eliminates the paralysis of choice that comes from switching models between tasks.” That is a genuine benefit, especially for teams without the time or expertise to evaluate models themselves.
But the same analyst named the trade-off in the same breath: automatic routing is “a double-edged sword, as developers can be suspicious of black box tools.”
When the platform decides which model touches your code and you cannot see the decision or override it, you are trusting that the vendor’s routing logic will always make the choice you would have made, on every task, forever.
For a lot of workflows, that is a fine trade. For the specific business logic that keeps an IBM i shop’s core operations running, it is worth asking whether “trust the routing” is a decision you actually want to hand over.
What Model Choice Actually Costs You If You Get It Wrong
This is not an abstract concern. A few concrete risks show up when model choice is made for you rather than by you:
- You cannot compare output before committing. If one model produces a subtly wrong interpretation of a business rule buried in a 30-year-old program, and you never see what a different model would have produced on the same input, you have no way to catch that except by finding the bug later, in production.
- You inherit the vendor’s roadmap, not just their product. If a vendor changes which models it routes to, deprecates a model, or shifts pricing on the backend, your results can change without you changing anything. You experience this as a mysterious quality shift, not as the vendor decision it actually is.
- You cannot meet data residency or governance requirements that are specific to your organization. Some IBM i shops have real constraints on which models are allowed to see certain data, driven by industry, geography, or internal policy. A platform that routes automatically and does not expose that decision makes it hard to enforce constraints that matter to your specific business, not a hypothetical average business.
- You lose the ability to learn what actually works for your codebase. Every RPG shop’s code is different. The programs written in the 1990s do not look like the ones written in 2015. Without visibility into which model handled which task and how well, you cannot build institutional knowledge about what actually performs well against your specific application portfolio.
None of this means automatic routing is a bad idea in general. It means it is a bigger bet than it feels like when you make it, and worth making with your eyes open rather than by default.
What Keeping the Choice in Your Hands Looks Like in Practice
CoderFlow takes a different position on this: the model is a configuration choice, not a hidden decision made on your behalf. You can run a task against Claude, Gemini, OpenAI’s models, or other supported options, including open-source models running on your own infrastructure for shops with strict data residency requirements.
CoderFlow also includes automated judging agents that can run competing models against the same task, evaluate the outputs against standards you define, and surface the strongest result along with the reasoning behind that assessment.
You still make the final call before anything is committed. Over time, this builds exactly the kind of institutional knowledge that automatic routing keeps invisible: which models actually perform best on your dependency-heavy batch jobs versus your customer-facing screens versus your documentation tasks.
CoderFlow’s agent execution happens inside your own infrastructure. Combined with visible, adjustable model choice, that means the two decisions that matter most for a shop protective of decades of business logic, where the work runs and which model touches it, stay in your hands rather than the vendor’s.
None of this is a claim that any single model is universally better. It is the opposite claim: no single model is universally better, which is exactly why the choice should not be made for you once, by default, and left there.
Making the Decision Deliberately
f you are evaluating AI tools for your IBM i environment, the model question is worth asking directly, whichever platform you are looking at:
- Can you see which model handled a given task, or only the output?
- Can you compare two models against the same input before deciding which result to trust?
- Can you change your answer later without re-architecting your entire AI strategy?
- And if your organization has specific data residency or governance requirements, can the platform actually honor them today, not on a future roadmap?
These are not edge-case questions. They are the difference between an AI strategy that adapts as your understanding improves and one that locks in whatever assumptions were built into the platform on day one.
If you want to see what model choice actually looks like against your own RPG and COBOL codebase, side by side rather than routed automatically, reach out to our team at Futurization@ProfoundLogic.com.