Every BES data team I have worked with has a moment they do not talk about publicly. EGM or SEDDK sends a list of ten participant certificate numbers and asks for the full computation trail behind a specific field — usually a state contribution eligibility flag, a fon dağılımı at a specific valör date, or a giriş aidatı deduction. The team knows the monthly submission was correct. The reconciliation totals matched. The aggregate reports passed every control. And yet nobody can reproduce those ten records on demand without rerunning the entire pipeline against a full month of data and grep-ing the output.
This is not a data problem. It is an architectural problem, and it is almost universal in Turkish pension and insurance shops.
The Batch Assumption Baked Into Everything
BES pipelines are almost always designed around one operating mode: process the full participant population, join against fon fiyatları, apply devlet katkısı rules, calculate hak kazanma, produce the EGM file. The logic is written assuming the batch context exists — the temp tables are populated, the reference snapshots are loaded, the calendar tables are joined, the previous month's balances are available in staging.
Remove that context and the code cannot execute. Some concrete examples I have seen:
- A devlet katkısı eligibility calculation that depends on a window function ranking all participants by contribution date within the fund. Run it for one participant and the ranking is meaningless.
- A fon getiri distribution that reads from a session-scoped temp table populated by an earlier step in the DAG. The step exists nowhere as a callable function.
- A hak kazanma süresi calculation that joins against a derived table of all sözleşme events across the population to detect ara verme periods. Single-record execution returns null because the self-join has no peers.
- Ayrılma calculations that reference a batch-level parameter table written by the orchestrator at run start and dropped at run end.
The pipeline is correct. It is also completely opaque at the record level.
Why This Fails Under Regulatory Sampling
EGM and SEDDK do not audit datasets. They audit records. Their forensic methodology is explicitly designed around small samples — five, ten, twenty participant files — because the point is to trace the logic, not verify the arithmetic. When the auditor asks "why did this participant receive 23.47 TL of devlet katkısı in March instead of 24.10 TL", the expected answer is a deterministic trace: input values, rule references, intermediate calculations, final output.
What the auditor typically gets instead:
- A screenshot of the monthly reconciliation totals.
- An explanation that involves the phrase "the batch calculates it this way."
- A promise to rerun the pipeline and extract the record.
- A rerun that produces a slightly different number because reference data has since changed.
That last point is the one that turns a routine spot check into a finding. If you cannot reproduce the exact number that was submitted, you have implicitly admitted that your submission is not reproducible. And non-reproducibility, in a regulated pension context, is not a technical issue — it is a control weakness.
The Reproducibility Test Nobody Runs
Here is the test I recommend to every BES data lead: pick ten random certificate numbers from last month's submission. Without rerunning the batch, and using only code that exists in production, reproduce every calculated field for those ten records and produce the input values, the rule versions, and the intermediate steps.
Most teams fail this test on the first attempt. The failures cluster in predictable places:
- Reference data snapshots were not versioned. The fon fiyatı table has since been updated.
- Rule parameters (katkı payı oranları, devlet katkısı tavan) are stored as current values, not as-of-date values.
- Intermediate calculation results are not persisted. Only the final field lands in the submission table.
- The calculation logic exists only inside a stored procedure that requires the full staging environment to run.
None of these are exotic problems. They are the natural consequence of optimizing for throughput and reconciliation totals rather than for record-level explainability.
What Record-Level Auditability Actually Requires
Building a BES pipeline that survives regulatory sampling means treating single-record execution as a first-class use case, not an afterthought. Practically:
- Pure functions for every rule. Devlet katkısı eligibility, hak kazanma, fon dağılımı — each should be a callable function that takes explicit inputs and returns an output, with no dependency on batch context. The batch pipeline calls the same function the audit tool calls.
- As-of-date reference data. Every parameter, rate, and threshold used in a calculation must be resolvable for any historical date. This means bitemporal tables, not overwritten current-value tables.
- Persisted intermediates. For every calculated field in the submission, the intermediate values that produced it should be stored alongside it — not derivable, stored. Storage is cheap. Regulatory findings are not.
- Deterministic rule versioning. Which version of the devlet katkısı logic ran for this record on this date? The answer should be a lookup, not an archaeology project.
- A single-record replay tool. Give an analyst a certificate number and a date, and the system should produce the full trace without touching the batch infrastructure.
The architectural shift is significant. It means separating rule logic from orchestration logic, which most BES pipelines conflate. It means investing in reference data versioning, which most teams have deferred for years. It means accepting that the pipeline that processes ten million records and the tool that explains one record are the same system, not two.
The Uncomfortable Reframe
Auditability is not a report you generate. It is not a control you attest to. It is a property of whether your pipeline can answer, in isolation and after the fact, the question "why is this number what it is" for any single record you have ever submitted.
If the answer requires rerunning the batch, you do not have an auditable pipeline. You have a pipeline that has not yet been audited properly. The regulator's ten-record sample is not a stress test of your data. It is a stress test of your architecture. Most BES pipelines in Turkey would fail it today, and the teams running them know it.
The fix is not more controls on the output. The fix is designing the pipeline, from the beginning, to explain one record as clearly as it explains all of them.