Educational companion dossier · Fact, interpretation, lived experience, clinical education, fiction, and mechanics are labeled separately. Scope & safety
PsychologicalWar.orgPeople before labels.
REAL-WORLD INTERPRETIVE

SIMULATION LITERACY

Calibration, Validation, and Sensitivity

How to tune a model, test it against evidence, and reveal which assumptions drive its conclusions.

Read the working brief Choose a learning path
LEVEL 1

ORIENTATION

Why this matters

REAL-WORLD INTERPRETIVE

One-sentence brief

A model that reproduces one historical curve may still fail elsewhere or for the wrong reasons. Validation and sensitivity prevent a fit from becoming false authority.

REAL-WORLD INTERPRETIVE

Three key points

  1. Calibration selects parameters; validation tests performance on evidence not used for tuning.
  2. Sensitivity asks how conclusions change when assumptions change.
  3. Passing a bounded test is evidence within scope, not proof of reality or production readiness.
LEVEL 2

WORKING BRIEF

Evidence, context, and limits

REAL-WORLD INTERPRETIVE

Calibration

Calibration estimates or selects parameters so model outputs align with chosen data or constraints. It should document targets, loss functions, search ranges, prior assumptions, and overfitting risk.

  • Separate calibrated from fixed parameters.
  • Use data quality and uncertainty in the objective.
  • Do not hide manual adjustments.
REAL-WORLD INTERPRETIVE

Validation

Validation compares the model with observations, holdout periods, alternative datasets, or known accounting identities. A model may be useful for one purpose and invalid for another.

  • Define success before testing.
  • Use out-of-sample or forward evaluation where possible.
  • Report failures and subgroup gaps.
REAL-WORLD INTERPRETIVE

Sensitivity analysis

Sensitivity analysis varies inputs, structures, and rules to identify what controls the result. Local sensitivity tests small changes; global sensitivity explores combinations and interactions.

  • Prioritize assumptions with high impact and low evidence.
  • Show direction and magnitude of change.
  • Do not present one parameter set as inevitable.
REAL-WORLD INTERPRETIVE

Identifiability and equifinality

Different parameter combinations or mechanisms can produce similar outputs. A good fit does not automatically identify the true causal process.

  • Compare multiple plausible explanations.
  • Use diagnostic observations, not only aggregate fit.
  • Mark parameters that cannot be estimated from available data.
REAL-WORLD INTERPRETIVE

External validity

A model calibrated in one period or jurisdiction may fail under different institutions, mobility, climate, access, reporting, or behavior. Transfer requires new evidence and review.

  • Recalibrate or bound transfer claims.
  • Do not use one country as the universal baseline.
  • Publish where the model has not been tested.
REAL-WORLD INTERPRETIVE

Release decision

Automated validation can support a release gate, but it does not replace legal, ethical, accessibility, scientific, regional, or lived-experience approval.

  • Keep human approval state explicit.
  • Rerun after material changes.
  • Preserve failed baselines as regression evidence.
LEVEL 3

COMPLETE DOSSIER

Limitations, game links, and review context

DISPUTED / MULTIPLE ACCOUNTS

Known limitations and gaps

  • This is a non-operational educational transformation: it does not build or implement a game, executable simulator, forecasting service, emergency tool, or decision system.
  • This page is an educational transformation of supplied research leads; it does not authenticate every citation, equation, product claim, or institutional attribution in those files.
  • A scenario, model run, or map is not an observation, forecast, legal finding, public-health instruction, emergency warning, or proof of future behavior.
  • No source code, nuclear-effects formula, casualty calculation, target-selection method, cyber-intrusion procedure, exploit chain, or instruction for bypassing safeguards is published.
  • Geography, nationality, ethnicity, religion, language, migration, disability, health, poverty, or political identity are not inherent danger, compliance, intelligence, competence, or worth variables.
  • Specialist scientific, public-health, accessibility, legal, regional, ethics, security, and lived-experience review remains pending; the page stays open to correction.
LEVEL 4

RESEARCH EDITION

Sources, methods, and stable links

REAL-WORLD INTERPRETIVE

Linked reports

  • Building Epidemiological ModelsProtected research reference; no public report route in this release.
  • IARPA SimEarth Game ArchitectureProtected research reference; no public report route in this release.
  • Planetary Simulation Game ArchitectureProtected research reference; no public report route in this release.
REAL-WORLD VERIFIED

Method and corrections

This page follows the public method for provenance, confidence, source independence, alternative accounts, limitations, review state, and visible correction.

Read the methodView corrections
NEXT

CONTINUE

Related learning

Simulation LiteracyAccessibility, Performance, and Low-Bandwidth ReviewA standard for making complex maps and simulations inspectable without high-end hardware, constant broadband, motion-heavy 3D, color-only encoding, or inaccessible controls.Simulation LiteracyAgent-Based, Compartmental, Cellular, and Network ModelsA comparative guide to four common model families and the kinds of heterogeneity, interaction, and uncertainty each preserves.Simulation LiteracyCompetitive AI Spread as a Fictional ScenarioA non-operational way to use a helpful-versus-harmful AI scenario to examine network assumptions, patching, uncertainty, governance, and recovery.Simulation LiteracyData Provenance, Assumptions, and VersioningA traceable chain from source data and transformations to model version, scenario inputs, output, correction, and supersession. Learning pathSimulation Literacy FoundationsLearn to distinguish scenarios from forecasts, select model families, inspect scale, validate assumptions, trace provenance, and read maps without false precision.