From unfamiliar legacy code to a reviewable application picture

Legacy modernisation rarely begins with a clean specification.

It begins with source code, jobs, screens, data definitions, old integration conventions, partial documentation and assumptions that may have survived longer than the people who originally made them. The first problem is not generating replacement code. It is establishing enough of the current system to make a defensible decision about what should change.

We wanted to test how far Harten's MaaS platform could get from an unfamiliar mixed-technology legacy corpus without pretending that automated analysis is infallible.

This is a case study of that run.

The source material comes from NightingaleHQ's public legacy-functional-docs repository. NightingaleHQ describes the project as a first-draft documentation system for Natural/Adabas, Mantis/Supra and surrounding mainframe artefacts, with deterministic extraction, citations, explicit gaps and human review. The repository is MIT licensed.

The worked examples are synthetic. NightingaleHQ explicitly describes its screen fixtures as invented material in a fictional steel-mill application rather than a client export. This is therefore not customer evidence, and NightingaleHQ did not commission, validate or endorse the Harten run.

For reproducibility, the source used for the Harten benchmark is also available in the public MaaS legacy mainframe benchmark repository.

The benchmark was deliberately messy

This is not a conventional COBOL application.

The corpus mixes Natural programs and maps, Adabas definitions, Mantis programs, Supra data definitions, JCL, screen exports, CICS-style definitions and SQL DDL. Some files represent business behaviour. Some describe data. Some establish runtime or orchestration context. Some are parser fixtures. A separate sensitive artefact is deliberately included to test whether potentially dangerous material is surfaced without being casually normalised into the application narrative.

That matters because real legacy estates are rarely equivalent to one programming language.

Code is evidence, but it is not the whole system.

MaaS uses a deterministic front end to inventory and partition material into bounded work units. Models then reason over those bounded units and the resulting evidence. This run was not configured to make a model decide the application structure before evidence existed.

The question was simpler:

Given the material MaaS could admit and understand, how much of a useful application picture could it establish in approximately an hour?

The elapsed time is a Harten-controlled observation from this run. It is not a universal benchmark and it is not a claim that every application of similar size will complete in the same time.

What came back

The saved output collection ran to 442 pages.

That number is not the claim. The export contains generated artefacts and advisory reviews, so page count is a poor proxy for value.

The important point is the shape of the output.

From the admitted evidence, MaaS produced:

• current-system documentation, including an application overview, high- and low-level design, requirements, candidate characterisation tests and open questions;
• an application specification covering behaviour, business rules, state, interfaces, data, security, operations, acceptance criteria, evidence gaps and generated diagrams;
• a proposed modernisation plan covering target architecture, change mapping, migration and rollback, verification, risks and decisions, delivery slices and an execution contract.

Both major downstream artefacts retained their authority boundary. The application specification was marked as an unapproved draft. The modernisation plan was marked as an unapproved proposal.

Generation did not become approval merely because the system could produce a plausible document.

What the system recovered

A few examples show the level at which the run was reasoning.

Mill-order release

MaaS identified a release process associated with MMP0100.

The generated account recorded that the order must exist, that its status must be CONF, that available stock is compared with order weight using a stated 2.50 per cent tolerance, and that evidenced outcomes include RLSD, PART and return codes 10, 20 and 30.

It also carried uncertainty forward. The consolidated specification did not claim that it had completely established the boundary between a successful partial release and an insufficient-stock rejection.

That particular gap is useful because the upstream source contains more detail than the first MaaS consolidation recovered. In the pinned NightingaleHQ example, MMP0100.nsp distinguishes full release, a partial release within the tolerance, and a lower-stock rejection.

We have not rewritten the first Harten run to make it look better after the fact.

The missed distinction is benchmark evidence. It can be used to improve the parser, specialist analysis or evaluation set and then tested again.

Quality hold

The run identified a second order-state pathway around MMP0400 and the MMM0150 map.

It recovered order number, hold reason and inspector identifier as inputs, return code 40 for a missing order, rejection of an already-held order, a transition to HELD, creation of a QUALITY-HOLD record, use of Natural's date function and an END TRANSACTION statement.

It also identified a call to MMN0900, described in the supplied material as an audit routine.

What it did not do was silently infer the missing audit contract. Payload, persistence, transaction participation and failure behaviour remained unresolved.

That is an important distinction. Finding an audit call is not the same as proving that the target system can replace it with ordinary operational telemetry.

Certificate enquiry

The run associated MMP0200 with certificate enquiry and an external PDFGEN dependency.

It recovered blank and unknown certificate re-prompting, retrieval or display of heat number and cast date, and an error path around PDF generation.

The generated modernisation plan did not immediately turn that external call into an asynchronous cloud workflow. It proposed isolating the dependency behind an adapter until timing, output, retry and failure semantics were understood.

That is the kind of restraint we care about.

Modernisation architecture should be a consequence of evidence, not a template applied to old code.

Sensitive material

The corpus includes a separate source artefact containing a credential-like literal associated with an identifier.

MaaS surfaced it as a potential security concern while preserving uncertainty around language classification, deployment, reachability and whether the value was live, test data or already invalidated.

We are deliberately not reproducing the sensitive literal here.

The useful behaviour is not simply that the system spotted a suspicious string. It is that the observation remained bounded by what the evidence could actually establish.

The plan was not "turn everything into microservices"

The proposed target used Microsoft Azure, ASP.NET Core/.NET, React, a Backend-for-Frontend pattern and Microsoft Entra ID.

More important than the technology list was the reasoning around boundaries.

The plan proposed keeping release and quality-hold behaviour within an order-management boundary initially because they share order state and may share transaction ownership. It treated certificate enquiry and generation as a candidate separate boundary because the data and external generation dependency differ.

It explicitly warned against creating independently deployable services without a demonstrated reason.

It also treated asynchronous messaging as conditional. Azure Service Bus was an option only where sequencing, visibility, durability and failure semantics allowed it.

Legacy return codes could be preserved at compatibility boundaries while allowing cleaner internal outcomes.

Application Insights and Azure Monitor were not treated as automatic substitutes for durable business audit.

Those are not proofs that the target design is correct. They are examples of a plan remaining coupled to the evidence and uncertainties found during discovery.

What MaaS did not establish

The run was not complete.

Some source families in the public corpus did not materially contribute to the consolidated evidence set used for this output. The final application picture was therefore stronger in some areas than others.

That does not invalidate the result.

It tells us where the next evidence should come from.

The corpus contains additional JCL, Mantis, Supra, DDM/FDT, screen and runtime material. Those artefacts can enrich future analysis and answer questions that the first run left open.

That gives us a measurable improvement loop:

1. preserve the original run;
2. identify a material unresolved question;
3. identify source evidence that should help answer it;
4. improve the deterministic admission, parser, specialist or evaluation layer;
5. rerun against the same pinned corpus;
6. test whether the uncertainty is genuinely reduced.

The objective is not to teach the system to sound more confident.

It is to make stronger claims because stronger evidence became available.

The gap register is part of the output

One of the easiest mistakes in AI-assisted discovery is to measure success only by how much the model says it knows.

We think the unresolved tail matters just as much.

A good gap tells the next engineer where to spend attention.

In this run, the open questions included transaction behaviour, audit failure semantics, exact release conditions, complete status vocabulary, PDFGEN contracts, runtime topology, database semantics and the relationship of the separate sensitive artefact to deployed behaviour.

Some of those questions can be answered from additional source.

Some require runtime observation.

Some may require local or client-side models with specialist support for customer-specific artefacts.

Some require a domain expert because the answer is not encoded in software at all.

The point is not to eliminate humans. It is to stop spending expensive human attention indiscriminately across the whole estate.

A Pareto-style operating model, without claiming 80 per cent

There is an obvious Pareto intuition here: use machines to establish the broad recoverable picture quickly, then concentrate scarce expertise on the consequential frontier.

We are not claiming an 80/20 ratio from this benchmark. We have not measured one.

The useful operating model is nevertheless clear.

MaaS can perform broad deterministic inventory, bounded analysis, synthesis and gap generation. Client-side or local models can extend analysis for specialised or sensitive artefacts. Engineers can challenge the resulting specification. Domain and operational experts can resolve questions that source alone cannot answer.

The output gets progressively stronger as evidence is added.

That is a more realistic model of AI-assisted modernisation than asking one model to read an estate and declare that it understands everything.

Could three senior engineers do better?

This run was not a controlled human-versus-machine experiment, so we will not claim a productivity multiple.

Three experienced legacy engineers could certainly find things this run missed. In a small public corpus, they may notice additional JCL, screen or data relationships quickly. They also bring contextual judgement that a model does not possess.

The more useful question is what those engineers should spend their time doing.

After approximately an hour, this MaaS run had already created a substantial review surface: current-state descriptions, recovered behaviours, state and data views, security observations, explicit gaps, diagrams, an application specification and a proposed route to modernisation.

That changes the starting point for human work.

Instead of beginning with an undifferentiated repository, engineers can begin by challenging a structured account:

• Is this rule correct?
• Is this gap material?
• Which of these behaviours actually matters to the business?
• Which uncertainty changes the target architecture?
• What needs runtime evidence?
• What can safely be left unresolved for now?

That is where senior engineering judgement has the highest value.

What this case study demonstrates

This benchmark does not demonstrate infallibility.

It does not demonstrate complete recovery of the synthetic application.

It does not demonstrate behavioural equivalence with a running production system.

It does not demonstrate customer acceptance, production migration or endorsement by NightingaleHQ.

It demonstrates something narrower and, we think, commercially important.

An evidence-first system can take an unfamiliar mixed legacy corpus and, in a short period of time, turn the evidence it can understand into a coherent, reviewable application picture. It can carry that picture forward into a proposed specification and modernisation plan. It can retain uncertainty instead of hiding it. And the gaps can become the next dataset for improving the system or directing human investigation.

That materially changes the economics and sequencing of discovery.

The claim is not that the machine understands everything.

The claim is that a modernisation team can start from a much more structured account of what appears to be true, what supports it, what remains uncertain and what should be investigated next.

That is a better place to begin changing a complex system.

---

Source and attribution

The benchmark source is derived from the public NightingaleHQ legacy-functional-docs repository, copyright NightingaleHQ and licensed under the MIT License. The worked examples are synthetic and fictional, not customer source. NightingaleHQ did not commission, validate or endorse this Harten benchmark.

Harten's public benchmark input is available at lavann/maas-legacy-mainframe-benchmark.

This case study reports a Harten-controlled MaaS run on 18 September 2026. The approximately one-hour processing time and 442-page saved export describe this run only. The page count includes generated artefacts and advisory reviews and should not be interpreted as a measure of correctness or completeness.

This case study sits within Harten's wider work on application understanding and evidence-first modernisation. Browse the Harten Papers or examine the public application examples.