6 min read 1,082 words 6 views
The data lineage problem hiding inside financial reports

Accounting has always been obsessed with the audit trail of a transaction. Every invoice, every journal entry and every bank movement can be traced to a document, a date and a person who approved it. What finance teams rarely trace with the same care is the path a number takes after it leaves the ledger. Between the general ledger and the slide shown to the board, figures get exported, filtered, joined to other datasets, adjusted by hand and pasted into templates. That journey is where most reporting errors are born, and it is almost never written down.

Data engineers call this missing record lineage. In a well-run data platform, lineage answers a simple question for any value on a dashboard: where did this come from, and what happened to it on the way? Finance departments ask the same question during every close, usually at the worst possible moment, and usually by opening five spreadsheets and hoping the formulas still point where they used to.

Why financial numbers lose their history

The problem is structural rather than careless. A typical monthly report draws on the ERP, a payroll system, a CRM, a billing tool and at least one workbook that only one analyst truly understands. Each handover strips away context. A revenue figure exported from billing arrives in the consolidation file without the filter that excluded refunds, or the currency rate used on the day it was pulled. Three months later, nobody remembers that the filter existed.

Spreadsheets make this worse because they hide transformations inside cells. A hardcoded adjustment typed over a formula looks identical to a calculated value. Linked workbooks break silently when a file is renamed. Teams that move reporting into dedicated financial statements software usually do it for speed, but the more durable benefit is that the path from source to statement becomes visible. When the tool records which ledger accounts feed which line, and which mapping rule applies, the question of origin stops being archaeology and becomes a lookup.

What regulators learned first

Banks hit this wall long before corporate finance teams did. After the 2008 crisis, supervisors discovered that many large institutions could not quickly add up their own risk exposures across business lines and legal entities. The Basel Committee responded with a set of principles on risk data aggregation, and more than a decade later lineage remains one of the hardest parts to get right. A recent supervisory newsletter notes that legacy systems, scattered data estates and constantly shifting lineage make end-to-end traceability difficult to confirm and maintain, even for organisations with large compliance budgets.

That finding matters outside banking. If institutions with dedicated data offices and regulators watching them still struggle to prove where a number came from, a mid-sized company running its close on email and shared drives is almost certainly carrying invisible risk. The difference is that nobody inspects the mid-sized company until something goes wrong, such as a restatement, a failed due diligence review or a covenant calculation that does not match the lender’s version.

The three places lineage breaks

Most lineage failures in finance fall into a few predictable categories.

The first is extraction. Data leaves a source system through a report or an export, and the parameters of that export are rarely stored alongside the output. Date ranges, entity selections and status filters all shape the result, yet the file that lands in a shared folder carries none of them.

The second is mapping. Chart of accounts structures differ between subsidiaries, and someone has to decide which local account rolls into which consolidated line. Those decisions often live in a lookup table that is edited without version control. When a mapping changes mid-year, comparisons against prior periods quietly stop meaning what readers assume they mean.

The third is manual adjustment. Top-side entries, reclassifications and management overlays are sometimes legitimate and sometimes the only way to meet a deadline. The risk is not the adjustment itself but the absence of a note explaining it. Good practice treats every override like a journal entry: it needs an author, a reason and a date.

Borrowing from the data engineering playbook

Data teams have spent years building habits finance can adopt without buying anything new. Assigning a named owner to each dataset is the cheapest of them. When the revenue extract has an owner, someone is accountable for documenting its filters and noticing when a source system changes. Broader governance frameworks, such as the ones Dataconomy has written about, stress defining roles and measuring progress so that the effort does not stall after the first enthusiastic quarter, and it helps to see how other teams assign ownership before designing your own.

Validation rules are the second habit. Engineers test data automatically before it moves downstream: totals must reconcile, signs must make sense, dates must fall inside the reporting period. Finance can build the same checks into its close. A balance sheet that does not balance is an obvious failure, but subtler checks catch more. Does intercompany revenue net to zero? Does headcount in the payroll extract match the HR system within a small tolerance?

There is public evidence that automated checks work. When the XBRL US Data Quality Committee introduced free validation rules for structured SEC filings, filers cut the errors those rules covered by 64 percent in the first quarter the rules applied, compared with a year earlier, and large accelerated filers saw an even steeper drop, a result the standards body reported in 2016. The rules did not make accountants more careful. They simply caught mistakes at the point where they were cheapest to fix.

Making every figure answerable

The practical goal is modest: any number in a management report should be answerable within a few minutes. Someone should be able to click or look up where it came from, which transformations touched it and who approved any manual change. That standard does not require a data lake or a dedicated engineering team. It requires documenting extraction parameters, putting mapping tables under version control, logging adjustments with reasons, and running automated reconciliations before figures reach a reader.

Finance teams that get there notice a side effect. Close cycles shorten, not because people work faster, but because fewer hours disappear into tracing discrepancies backwards. Auditors ask fewer follow-up questions. Board members stop receiving two versions of the same metric. Lineage sounds like a technical concern, yet its real payoff is trust, and in financial reporting trust is the product.


Featured image credit