Version: 1.0 Status: Explanatory (Non-Normative)
1. Purpose and Scope ————————
This document identifies recurrent failure modes observed in Results-Based Funding (RBF) systems in the health sector and explains how analogous failures would manifest in education systems if not addressed explicitly.
For each failure mode, this document identifies the specific GEOS Data Pipeline (GEOS-DP) specification(s) designed to prevent or mitigate the risk.
This document is explanatory only. It introduces no requirements and does not modify or supersede any GEOS normative specification.
Its purpose is to clarify why particular Data Pipeline standards exist, drawing on well-documented historical experience from health RBF systems (including WHO- and DHIS2-based programs), in a manner consistent with WHO and FATF practice.
2. Failure Mode 1: Ambiguous Object of Certification ——————————————————--
Early health RBF systems often failed to define precisely what was being certified. In practice, certification alternated inconsistently between reports, databases, indicators, institutions, or ad hoc summaries.
Before the standardization of DHIS2 indicator metadata and WHO indicator reference definitions, countries certified "results" that were structurally non-comparable, undermining funder confidence.
If education systems certify platforms, datasets, outcome claims, or institutions interchangeably, certification loses meaning and RBF4Ed claims become non-credible.
GEOS-DP-001 — Data Pipeline Conceptual Model
GEOS-DP-002 — Data Pipeline Certification Criteria
3. Failure Mode 2: Uncontrolled Inputs ——————————————
Health reporting pipelines historically accepted heterogeneous and weakly specified inputs, making downstream indicators sensitive to undocumented variation at the point of entry.
Paper registers, informal spreadsheets, and manual summaries entered HMIS pipelines without standardized entry constraints, requiring costly ex-post validation.
If education Data Pipelines accept poorly specified inputs, downstream outcome signals cannot be relied upon for finance, regardless of later processing quality.
GEOS-DP-004 — Data Pipeline Entry Specification
GEOS-DP-006 — Data Pipeline Traceability & Lineage Requirements
4. Failure Mode 3: Opaque Transformations ———————————————
Many health indicators failed audit because transformations between raw data and reported indicators were undocumented or non-reproducible.
Indicator discrepancies across DHIS2 deployments were frequently traced to undocumented aggregation or transformation logic.
Education outcome claims could not be independently recomputed, making certification non-credible to capital allocators.
GEOS-DP-006 — Data Pipeline Traceability & Lineage Requirements
GEOS-DP-003 — Data Pipeline Conformity Assessment Procedures
5. Failure Mode 4: Non-Comparable Outputs ———————————————
Health RBF systems failed when outputs differed subtly across countries or time periods despite nominally similar indicators.
WHO repeatedly revised indicator output definitions to restore cross-country comparability for financing decisions.
Education outcomes that appear similar but are structurally different undermine cross-jurisdictional RBF4Ed.
GEOS-DP-005 — Data Pipeline Exit Specification
GEOS-DP-008 — Data Pipeline Versioning & Evolution Rules
6. Failure Mode 5: Loss of Audit Trail ——————————————
Some health programs could not reconstruct historical results because lineage and provenance were lost as systems evolved.
Both WHO and FATF emphasize immutable audit trails following repeated failures to reconstruct past funding decisions.
Education RBF decisions become politically and fiduciary indefensible if historical outcomes cannot be reconstructed.
GEOS-DP-006 — Data Pipeline Traceability & Lineage Requirements
GEOS-DP-007 — Data Pipeline Integrity & Tamper Resistance Requirements
7. Failure Mode 6: Silent Standard Drift ——————————————--
Health indicators sometimes changed quietly over time, breaking longitudinal analysis without explicit signaling.
WHO indicator revisions now explicitly version definitions to preserve longitudinal integrity.
Apparent outcome improvements may reflect standard changes rather than real learning gains.
GEOS-DP-008 — Data Pipeline Versioning & Evolution Rules
GEOS-DP-012 — Data Pipeline Certification Artifacts & Registry Schema
8. Failure Mode 7: Excessive Opacity or Excessive Disclosure —————————————————————-
Health systems oscillated between over-disclosure (privacy risk) and under-disclosure (trust erosion).
WHO indicator metadata balances public transparency with protection of underlying sensitive data.
Either Ministries expose sensitive student data, or funders lack sufficient visibility to rely on outcomes.
9. Failure Mode 8: Confusion Between Measurement and Finance Logic ———————————————————————-
Some health programs embedded funding rules directly into measurement systems, creating conflicts of interest.
Best-practice health RBF separates indicator production from financing decisions.
Education systems could be accused of "teaching to the funder" or gaming metrics.
GEOS-DP-010 — Data Pipeline Relationship to Outcome Signal Portfolios (GeOSP)
GEOS-DP-011 — Data Pipeline Non-Requirements (Explicit Exclusions)
10. Failure Mode 9: Unclear Post-Certification Artifacts ————————————————————
Health funders struggled when certification produced no stable, inspectable artifacts.
Modern RBF systems rely on registries of certified indicators and reporting pipelines.
Funders cannot discover, reference, or rely on certified education pipelines.
11. Concluding Observation ——————————
The GEOS Data Pipeline specification set reflects hard-earned lessons from decades of health-sector RBF experience. Each specification exists to prevent a specific class of failure that has historically undermined trust, comparability, or scalability.
This document exists to make those causal relationships explicit—so that education systems do not repeat health's early mistakes, and so that RBF4Ed can scale on a firmer foundation.
END of "GEOS-EX-002 — Failure Modes in Results-Based Funding: Lessons from Health"