LLMs are Bayesian, in Expectation, not in Realization