Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models

MPI-IS Tübingen & ETH Zürich, July 2026
Preprint
Paper Code

In-context learning is commonly interpreted as a form of conditional inference, in which the prompt specifies a context and the model’s output is treated as an estimate of the corresponding conditional distribution. If this interpretation holds, then LLM estimates should satisfy basic probabilistic identities. In particular, the law of total probability requires that prior-weighted conditional estimates over any valid partition of a population aggregate back into the population-level marginal. We test this requirement using binary conditioning trees, which recursively partition a population into increasingly fine-grained subpopulations, each verbalized as context for the model.

Setting: binary conditioning tree, level-wise aggregation, direct evaluation and self-consistency
A binary conditioning tree partitions a population into subpopulations, whose LLM estimates are aggregated via the law of total probability and compared against the direct estimate.

Applying this protocol across survey questions and forecasting tasks, we find widespread violations of self-consistency, even among frontier models. Strikingly, we observe a macro fallacy: aggregates reconstructed from fine-grained subpopulation estimates align better with human reference data than direct population-level estimates. This suggests that models possess relevant subgroup knowledge but fail to propagate it into aggregate estimates. Building on this observation, we propose statistical self-consistency as an unsaturated, reference-free criterion for evaluating LLMs.

Citation

references.bib
@misc{wolf2026partitionpromptaggregatestatistical, title = {Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models}, author = {Patrik Wolf and Thomas Kleine Buening and Andreas Krause and Celestine Mendler-D{\"u}nner}, year = 2026, url = {https://arxiv.org/abs/2607.15277}, eprint = {2607.15277}, archiveprefix = {arXiv}, primaryclass = {cs.CL} }