In-context learning is commonly interpreted as a form of conditional inference, in which the prompt specifies a context and the model’s output is treated as an estimate of the corresponding conditional distribution. If this interpretation holds, then LLM estimates should satisfy basic probabilistic identities. In particular, the law of total probability requires that prior-weighted conditional estimates over any valid partition of a population aggregate back into the population-level marginal. We test this requirement using binary conditioning trees, which recursively partition a population into increasingly fine-grained subpopulations, each verbalized as context for the model.
Applying this protocol across survey questions and forecasting tasks, we find widespread violations of self-consistency, even among frontier models. Strikingly, we observe a macro fallacy: aggregates reconstructed from fine-grained subpopulation estimates align better with human reference data than direct population-level estimates. This suggests that models possess relevant subgroup knowledge but fail to propagate it into aggregate estimates. Building on this observation, we propose statistical self-consistency as an unsaturated, reference-free criterion for evaluating LLMs.