# What Engineers Need From AI-Assisted Application Analysis

> Six PRSD-IWS examples show how source-linked findings help engineers preserve behaviour, investigate uncertainty and define meaningful tests for AI-assisted software change.

Paper: Technical case analysis
Published: 2026-10-17
Canonical: https://harten.io/papers/what-engineers-need-from-ai-assisted-application-analysis/

Cover illustration: [View the title image](https://harten.io/media/papers/covers/v1/what-engineers-need-from-ai-assisted-application-analysis.png?v=398d5e5dab66). Illustrative cover artwork, not an application screenshot or a record of measured results.

**Lavan Nallainathan | Harten Technical Papers | 17 October 2026**

*Technical case analysis based on an exploratory inspection of public PRSD-IWS analysis and a pinned source reference, not a controlled product benchmark.*

[Download the illustrated paper (PDF)](https://harten.io/assets/papers/engineering-analysis/what-engineers-need-from-ai-assisted-application-analysis.pdf)

## Abstract

AI-assisted application analysis is useful when it helps somebody answer an engineering question, inspect the supporting material and prepare a change without silently altering the meaning of the existing behaviour. A report can be readable and substantially correct while still leaving important work to its reader: identifying which execution path a rule describes, finding the relevant implementation, deciding whether two observations really conflict, and turning a broad acceptance statement into a test with a definite expected result.

This paper examines six questions using Harten's published PRSD-IWS analysis and targeted source-file inspection. It identifies useful distinctions already preserved in the analysis, including different completion paths and the separation of a financial guarantee's current status from its recorded decision. It also shows how a supposed contradiction can arise from comparing different operations, and how a correct boundary condition becomes more useful when its application area and enforcement layer are explicit. The resulting engineering answer records combine the broader explanation with source locators, change implications and concrete candidate checks. The study does not measure comparative product accuracy or human review-time savings. It establishes a set of observed presentation and reasoning issues that a subsequent controlled comparison can test.

## The Engineering Question Behind an Application Report

Imagine an engineer preparing to replace the shipment-completion behaviour in an established service. The documentation says that completion must not be dated before receipt. The statement is understandable, and it sounds like an entirely reasonable rule. Before implementing it, however, the engineer needs to ask where that rule applies. Is it checked when an external user uploads a certificate, when an internal operator records completion, or in both places? Is it a condition in the domain model, a form validator, or a rule enforced only by one caller?

That distinction can change what the engineer builds. A replacement that applies the rule everywhere may introduce a restriction not established for every existing path. A replacement that omits it everywhere may remove a control that the current implementation does contain. Neither choice should be made simply because one sentence is easier to implement than a qualified account of the application.

In the earlier [paper on modernisation cost, duration and confidence](https://harten.io/papers/estimating-modernisation-cost-duration-and-confidence/), we explored how application findings can affect proposed work and delivery assumptions. This paper asks a different question: what must an analysis provide before an engineer can use a finding reliably?

The comparison began with the existing PRSD-IWS material rather than a newly designed demonstration. We wanted to see what the published output already helped a reader understand and where a targeted return to source made the engineering answer more precise. The purpose was not to construct a test that Harten would necessarily win.

## What We Inspected and What the Comparison Can Establish

We selected six questions about movement completion, receipt clearing, financial-guarantee state, replacement integrity, partial-rejection validation and the preparation of tests. These were deliberately chosen from issues already visible in our discussion and application material. They are a purposive sample, not a random estimate of codebase-wide correctness.

The first material was Harten's published lifecycle synthesis, selected specification sections and candidate-characterisation chapter, recorded at Harten repository commit `96e92c1ea9cb9457406c1622efdbd02d402159e6`. The second was a targeted source-assisted investigation using GitHub search and file reading at DEFRA PRSD-IWS commit `95f7eeae92f82a8db3c609f14ce2483c6e7cb738`. The investigation used the same AI assistant that had already encountered parts of the Harten material. It was not blinded, and the source-assisted answers are not an independent coding-assistant benchmark. [1–5]

There is also an input-version boundary. The Harten case-study manifest records the analysed archive's hash but does not verify the corresponding upstream commit. We did not recover that archive in this inspection. The reference commit therefore cannot be represented as a proven identical input, and an apparent difference against it is not automatically a historical analysis error. The public package also intentionally excludes its underlying evidence reports and private evidence index. This study assesses the material that was available, not every feature of the Wayfinder workspace. [1]

These limits do not prevent us examining whether a statement supplies the scope needed to use it, whether its reasoning follows, or which additional source observation makes the next engineering action clearer. They do prevent us claiming a percentage accuracy improvement, a reduction in review time or superiority over another vendor. No application tests or proposed changes were executed. The inspection produced six comparison records and seventeen candidate checks, which remain proposals for verification.

## Different Completion Paths Need Different Explanations

The published states and acceptance material already distinguishes internal from external completion. It records an internal check against the receipt date and a check rejecting a date later than the current UTC date. It does not assert that the corresponding external path enforces those same conditions. That is a useful result: the analysis preserves a difference an implementer could otherwise overlook. [2, 4]

![Two aligned source panels compare the non-default date and file checks in Complete with the receipt-date and future-date checks in CompleteInternally. External endpoint behaviour remains to be investigated.](/assets/papers/engineering-analysis/figure-01-completion-paths.png)

**Figure 1. The two completion methods do not contain the same checks. Selected statements identify the scope of each finding; absence of a local check does not establish what an external endpoint permits.**

[External method, lines 320–326](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/Movement.cs#L320-L326) · [Internal method, lines 333–344](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/Movement.cs#L333-L344) · [Open the full-size, source-linked figure (PDF)](https://harten.io/assets/papers/engineering-analysis/figure-01-completion-paths.pdf)

The pinned source makes the distinction inspectable. In `Movement.Complete`, the method checks that the completion date and file identifier are not their default values before firing the completion trigger. `CompleteInternally` instead reads `Receipt.Date`, rejects an earlier completion date, rejects a future date and then fires its internal completion trigger. The inspected completed-receipt constructors assign the supplied values; they do not add the missing receipt-date comparison to the external method. [6, 7]

An engineer can now identify a specific characterisation test: construct an eligible movement with a receipt dated on day 10, call internal completion with day 9, and check the exception raised by the date-order guard. An external endpoint test needs more work because its controller or handler may impose an additional check before reaching the inspected method. The analysis should say that, rather than turning the absence of a local guard into a claim about the whole endpoint.

The source-assisted answer is not valuable because it contains more technical names. It is valuable because those names identify the boundary of the claim and the place where the engineer can check it. The broader explanation tells the reader why two paths matter; the source reference makes the explanation actionable.

## Two True Observations Can Be Mistaken for a Contradiction

The published business-rules material identifies a more subtle problem around receipt data. It contrasts an account in which rejection or partial rejection clears a receipt with an account in which receive methods do not clear it, and calls the contrast unresolved. Those descriptions concern different operations. A receipt method can create a receipt while a later rejection method removes it. The two statements do not, by themselves, contradict each other. [3]

This is not a reason to remove uncertainty from the analysis indiscriminately. It is a reason to identify exactly which claims are being compared. Before treating two observations as conflicting, we need to establish that they concern the same object, operation, starting conditions, execution path and version. Otherwise the report can create an investigation task that exists only because its descriptions lost their scope.

The pinned `Movement` source illustrates the distinction. `OnReceived` and `OnInternallyReceived` create receipt data. `PartialReject` clears `Receipt` when the movement's starting status is Received. The state configuration permits an internally completed transition from PartiallyRejected to Completed. Yet `CompleteInternally` reads `Receipt.Date` before firing that transition. [6]

![Receipt creation and partial rejection clearing it are shown as separate operations. A dashed connection into internal completion marks the unverified caller sequence; internal completion reads Receipt.Date.](/assets/papers/engineering-analysis/figure-02-receipt-sequence.png)

**Figure 2. Receipt creation and later receipt clearing are different operations. The dashed connection marks the execution path still requiring investigation, not a confirmed production failure.**

[Receipt operations, lines 268–308](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/Movement.cs#L268-L308) · [Internal completion, lines 333–344](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/Movement.cs#L333-L344) · [Open the full-size, source-linked figure (PDF)](https://harten.io/assets/papers/engineering-analysis/figure-02-receipt-sequence.pdf)

The useful engineering question is therefore whether the actual caller and persistence sequence can reach internal completion with a missing receipt. Another operation might restore it, or a higher-level invariant might prevent the sequence. We have not established either outcome. The next action is to inspect that path and build a representative test, not to assert a production failure.

The application analysis did something useful by identifying a possible relationship between partial rejection and completion. The improvement is to separate that risk from a supposed conflict between different operations. Keeping everything under an undifferentiated label of unknown would make it harder for the team to decide what to do next.

## A Lifecycle Distinction Matters When Another Component Uses It

The financial-guarantee lifecycle provides a clear example of something the published analysis already does well. It explicitly distinguishes the guarantee's current status from its recorded decision. A guarantee can become Released while retaining an Approved decision and the original decision date. It also distinguishes the separate release date. [8]

The reference implementation supports that account. Approval sets the decision to Approved. Release changes the lifecycle status and records the release date. The release callback only changes the decision and its date when the previous decision was None, so the earlier approval survives the ordinary approved-then-released flow. [9]

The next source observation makes the consequence clearer. The assessment view displays the recorded decision separately from the current status. Its display logic explicitly retains Released as the current status when the decision is Approved. The distinction is therefore not merely an incidental property of a domain class; this inspected consumer uses it to present different information to the user. [10]

![A guarantee changes from Approved to Released status while retaining its Approved decision and original decision date. Exact release-callback and assessment-template extracts show why both meanings matter.](/assets/papers/engineering-analysis/figure-03-guarantee-decision.png)

**Figure 3. Releasing a guarantee does not erase its approval decision. The source-derived before/after table is paired with the release callback and exact template expressions. This is not a captured application screen.**

[Release callback, lines 207–216](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/FinancialGuarantee/FinancialGuarantee.cs#L207-L216) · [Assessment template, lines 7–36](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Web/Areas/AdminExportAssessment/Views/FinancialGuaranteeAssessment/Index.cshtml#L7-L36) · [Open the full-size, source-linked figure (PDF)](https://harten.io/assets/papers/engineering-analysis/figure-03-guarantee-decision.pdf)

Suppose a replacement design proposes to merge the decision and status into one simplified field. The team now has a specific question to answer: how will that representation preserve the meanings and dates still needed by the service? The existing implementation does not prohibit redesign, but it supplies evidence that a redesign must address. A migration test should include an approved-then-released record, not only a newly approved one.

The output retains a distinction with a concrete consequence for implementation and migration. The additional source link identifies a consumer that makes the consequence easier to inspect.

## A Validation Rule Needs an Application Area and an Enforcement Layer

A similar issue appears in partial-rejection quantities. The published material says that the rejected quantity must be strictly less than the actual quantity in the cited domain path. It correctly preserves the equality boundary rather than simply repeating wording that says a rejected quantity must not exceed the actual quantity. [3, 4]

The pinned import service, `PartialRejectionImportMovement.PartailReject`, checks `rejectedQuantity >= actualQuantity`. With equal units and quantities of 10 and 10, that condition is true and the service throws. The accompanying error message refers to a quantity being more than the actual shipment quantity, but the executable condition also covers equality. A test based only on the message could therefore encode the wrong boundary. [11]

There is a second distinction. The similarly named export-side `PartialRejectionMovement.PartailReject` checks the date but does not contain the same quantity comparison in the inspected method. It would be incorrect to infer that the export endpoint accepts equality: an earlier form or request validator could reject it. But it would also be incorrect to extend the import service's local guard to every export path without checking. [12]

The source-assisted record therefore names the application area, method and layer next to the condition. It also records that the import service compares the supplied decimal quantities without visible unit conversion. The reviewer still needs to establish where compatible units are normalised. A source-level decimal comparison is not, on its own, a complete account of the business quantity rule. [11]

This example does not show that the published statement is necessarily wrong. It shows that a generally correct statement can leave an implementer with avoidable work if the supporting path is not identified where the rule is used. The path belongs with the claim, not only somewhere in a large supporting report.

## Some Questions Should Remain Open After Source Inspection

The approval flow also illustrates a limit that both the published analysis and direct source review should retain. The service looks up a currently approved guarantee, approves the new one, and changes the previous guarantee differently according to whether its reference matches the new reference. The collection lookup uses `SingleOrDefault` over guarantees with Approved status. [13, 14]

These files show a sequential replacement procedure and an assumption about the collection. They do not establish database uniqueness, transaction isolation or the result of concurrent approval requests. The published lifecycle material identifies those gaps, and the targeted source inspection provides no basis for declaring them resolved. [8]

An answer that confidently converts the lookup into a universal one-approved-guarantee invariant would be less useful than the qualified analysis. The next work needs database and transaction evidence, other mutation paths and concurrency tests. More confident wording would not remove that work.

For comparison purposes, this is not a win for either approach. It is a case where the available material sets a boundary on what we can conclude. A good evaluation must include questions of this kind, rather than rewarding every system for producing a definite answer.

## A Candidate Test Needs a Definite Expected Observation

The existing documentation includes named candidate tests and Given/When/Then structures. It also states that no successful execution is being claimed. Those are useful foundations. Some cases, however, still need substantial interpretation before they could become checks an engineer or agent can execute. [5]

For example, the movement-completion candidate allows the result to be Completed or an appropriate partial/rejection state. That formulation leaves the evaluator to decide what appropriate means. The inspected completion methods have specific inputs and transition behaviour, so a more useful handoff separates those operations from quantity and rejection tests rather than combining them into one broad scenario. [5–7]

The revised worked records identify the object and entry point, starting state, input values, expected observation, source basis and execution status. Where the source does not supply a complete expected result, the record is labelled as an investigation. A test asking what the external endpoint does with an early completion date is not the same as an approved requirement instructing the replacement to reject it.

### Worked Check: Internal Completion Before Receipt

This makes the existing candidate **Q1-T1** concrete; it does not add a newly executed test.

| Field | Proposed check |
|---|---|
| Starting condition | A valid Movement fixture in Received state, with a non-null Receipt dated 10 January 2026. Freeze the current UTC date at 20 January 2026. Use a valid creator identifier and otherwise valid fixture data. |
| Action | Call `Movement.CompleteInternally` with a completion date of 9 January 2026. |
| Expected observation | The date-order guard raises `InvalidOperationException` before the internal completion trigger is fired. |
| Why this follows | 9 January is earlier than the recorded receipt date of 10 January, so `completedDate < Receipt.Date` is true. |
| Source basis | `Movement.cs`, lines 333–344, at the pinned reference commit. |
| Execution status | Candidate characterisation check. Not yet executed. |
| Boundary | This is a method-level expectation. It does not establish external-endpoint behaviour, persistence atomicity, or an approved future business rule. |

The dates are chosen test inputs, not production observations. Freezing the clock keeps the future-date check from becoming an accidental second cause of failure. [6]

Seventeen candidate checks were prepared from the first five comparison questions. They include method-level date and identity guards, the approved-then-released guarantee, reference-dependent replacement and equality at the partial-rejection boundary. They have not been executed. Their contribution at this stage is greater specificity and clearer evidence boundaries, not a measured reduction in effort.

## The Consequence for AI-Assisted Engineering

When a human engineer reads a broad sentence, they may recognise that they need to search for its implementation before relying on it. A software agent also needs a way to recognise that boundary. Giving an agent a long report without clear execution paths or distinguishing observations from approved requirements can leave it to supply the missing decisions itself.

The examples suggest a practical division of work. Application-wide analysis explains the capabilities, relationships and questions worth investigating. A source-linked engineering answer identifies the exact observation relevant to the change. A person or authorised process decides which behaviour the future system should require. Candidate tests then express how the chosen requirement or existing behaviour will be checked.

That division does not require every paragraph to contain a large schema or every missing detail to become a new approval gate. It requires enough information at the handoff to prevent a specific misunderstanding. In these examples that means naming the internal or external completion method, keeping approval history separate from release status, and checking whether two supposedly conflicting descriptions concern the same operation.

We applied that approach to the worked answer records in this comparison. We have not claimed a corresponding generator release, changed the existing public case study, or demonstrated that independent users complete tasks faster. Those are different claims and require their own evidence.

## What a Controlled Comparison Should Measure Next

A controlled comparison needs a common, verified input snapshot and questions fixed before the outputs are generated. It should include ordinary rules, cross-component relationships and questions that cannot be answered from the available material. The complete authorised evidence experience should be tested separately from the deliberately narrower public case-study presentation.

Independent reviewers can then compare source investigation, analysis-assisted investigation and a combined approach. They should assess whether the answer is correct within its stated scope, whether the supporting source can be inspected, whether consequential conditions were omitted and whether a concrete change or verification task can be prepared. Active review time should include checking and correcting the output, not just reading its first answer. Reviewer disagreements and learning effects need to be recorded rather than hidden inside an average.

This initial inspection does not tell us which product wins that test. It does show why the test should examine more than the presence of a report, a graph or a plausible explanation. The useful output is the one from which a team can determine what the application does, what remains uncertain, which decision is needed and what must be checked before the change is accepted.

## References and Source Notes

All source inspection refers to DEFRA PRSD-IWS commit `95f7eeae92f82a8db3c609f14ce2483c6e7cb738`. Harten output locators and Git blob identities are retained in the comparison's evidence register at Harten baseline commit `96e92c1ea9cb9457406c1622efdbd02d402159e6`. The original analysed archive identity has not been mapped to the reference commit.

1. Harten, public PRSD-IWS case-study manifest, `content/case-studies/prsd-iws/manifest.json`. Original source archive SHA-256: `3452e285a66ecd243d07e3f1df3ba1379a859b6078f24425b6e159f591283e85`. The manifest states that the analysed upstream commit is not verified and lists excluded evidence material.
2. Harten, [States and Transitions](https://harten.io/case-studies/prsd-iws/specification/states/).
3. Harten, [Business Rules](https://harten.io/case-studies/prsd-iws/specification/business-rules/).
4. Harten, [Acceptance Criteria](https://harten.io/case-studies/prsd-iws/specification/acceptance-criteria/).
5. Harten, [Candidate Characterisation Tests](https://harten.io/case-studies/prsd-iws/documentation/13-candidate-characterisation-tests/), particularly CT-13-005, CT-13-006 and CT-13-008.
6. DEFRA, [Movement.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/Movement.cs), `CreateStateMachine`, `PartialReject`, receipt callbacks, `Complete` and `CompleteInternally`.
7. DEFRA, [MovementCompletedReceipt.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/MovementCompletedReceipt.cs).
8. Harten, [Financial-Guarantee Lifecycle Synthesis](https://harten.io/case-studies/prsd-iws/synthesis/lifecycle-states/).
9. DEFRA, [FinancialGuarantee.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/FinancialGuarantee/FinancialGuarantee.cs), `OnApproved`, `Release`, `OnReleased` and state configuration.
10. DEFRA, [Financial-guarantee assessment view](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Web/Areas/AdminExportAssessment/Views/FinancialGuaranteeAssessment/Index.cshtml), lines 1–90 inspected.
11. DEFRA, [PartialRejectionImportMovement.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/ImportMovement/PartialRejectionImportMovement.cs).
12. DEFRA, [PartialRejectionMovement.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/Movement/PartialRejectionMovement.cs).
13. DEFRA, [FinancialGuaranteeApproval.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/FinancialGuarantee/FinancialGuaranteeApproval.cs).
14. DEFRA, [FinancialGuaranteeCollection.cs](https://github.com/DEFRA/prsd-iws/blob/95f7eeae92f82a8db3c609f14ce2483c6e7cb738/src/EA.Iws.Domain/FinancialGuarantee/FinancialGuaranteeCollection.cs).

**Attribution:** Contains public sector information licensed under the Open Government Licence v3.0. Source software: DEFRA/Environment Agency, PRSD-IWS. The work is independent Harten analysis, not a departmental commission or endorsement. The case-study observations are not claims about the current production service.