The question of whether people store absolute magnitude information or relative local comparisons of magnitudes has remained unanswered despite persistent efforts over the last three decades to resolve it. Absolute identification is one of the most rigorous experimental benchmarks for evaluating theories of magnitude representation. We characterize difficulties with both absolute and relative accounts of magnitude representation and propose an alternative account that potentially resolves these difficulties. We postulate that people store neither long-term internal referents for stimuli, not binary comparisons of size between successive stimuli. Rather, they obtain probabilistic judgments of size differences between successive stimuli and encode these for future use, within the course of identification trials. We set up a Bayesian ideal observer model for the identification task using this representation of magnitude and propose a memory-sampling based approximation for solving it. Simulations suggest that the model adequately captures human behavior patterns in absolute identification.