The Metalog Distribution
The single distribution family behind every probabilistic surface in Compass.
Every distribution Compass renders — GDPNow headlines, ACH cell ratings, edge weights, compass-node scores, scenario terminal values — is stored and manipulated in a single mathematical form: the metalog distribution. Choosing one family everywhere is a design decision worth defending.
Why one family?
Classical probability literature picks a distribution family per problem: Normal for measurement error, Beta for rates, Gamma for waiting times, Lognormal for multiplicative effects. Each family has its own parametrization, its own fitting algorithm, its own tail behavior, and its own gotchas. A cross-domain product like Compass — which stores posteriors, rating distributions, edge weights, and GDP nowcasts side- by-side — would become a compatibility nightmare under that approach.
The metalog, introduced by Tom Keelin in 2016, solves this by being genuinely general. A single family with a tunable number of terms can fit Normal-looking, skewed, bimodal, and heavy-tailed shapes equally well. Storage is 18 floats per distribution regardless of shape. Sampling is exact and fast: one inverse-CDF evaluation per sample.
How it works
A metalog quantile function is a polynomial in logit(p) with a specific basis of low-order terms. Fit proceeds by ordinary least squares on a handful of elicited percentiles (typically p05, p25, p50, p75, p95). With 5 points and 5 terms, the fit is exact — the metalog passes through each percentile. With more terms it can capture more shape; 10–14 terms is typical for richer elicitations.
Compass uses bounded metalogs wherever the semantic range is known: edge weights on [0, 1], coupling weights on [0, 2], ACH ratings on [−2, +2]. The bound is implemented via a logit transform inside the fit — the output distribution is strictly inside the bound, no tail clipping needed.
The five-percentile elicitation standard
Across the platform, every AI prompt that elicits a distribution from an LLM uses the same five-percentile shape:
{
"p05": <number>,
"p25": <number>,
"p50": <number>,
"p75": <number>,
"p95": <number>
}This consistency has a purpose: when you see a 5-percentile elicitation anywhere in the codebase, the downstream fit is identical. Porting a new kind of distributional input only requires producing the 5 percentiles; the fit + storage + visualization layer handles everything from there.
What happens when the fit fails
Not every elicitation produces a well-posed distribution. If percentiles are non-monotone (p75 < p50), or the spread is degenerately small, or the LLM returns duplicates, the fit can fail. Compass handles these cases with a cascade:
- Clamp endpoints inward by ε to avoid ±∞ at the bounds.
- Jitter internal ties proportional to spread.
- Try bounded fit. If singular, try unbounded + post-clip.
- Fall back to a constant-at-median metalog — the caller always gets a valid 18-float list.
The system surfaces “distribution unavailable — using point estimate” at the UI level when the degenerate case fires, so users know their confidence is being derived from a scalar rather than a proper distribution.
When the metalog isn't the right tool
The metalog is an excellent working distribution. It is not a theoretical truth. Two known limitations:
- Extreme tails. For true heavy-tailed phenomena (power laws, t-distributions with low ν), the polynomial tail may over- or under-state tail mass depending on how the 5 percentiles were elicited. Compass is deliberately not trying to nowcast black swans — the metalog fits the body well and that's where most decisions live.
- Strongly multimodal. The metalog handles bimodality but only up to a few modes. Mixture-of-Normals is the right tool if you genuinely need 4+ modes.
Keelin, T. (2016). “The Metalog Distributions.” Decision Analysis 13(4): 243–277.