How do people learn underlying properties, such as mass and friction, from objects' interactions in complex scenes? Such inferences are difficult: the parameters cannot be directly observed and have nonlinear effects on the physical dynamics. Yet, people learn them. Participants predicted the stability of blocks stacked in complex tower configurations. After observing the true outcome, they answered, "which blocks are heavier?". Their responses indicate rapid learning of the blocks' relative masses. We view such learning as probabilistic inference in a generative model of Newtonian rigid-body dynamics, and express this hypothesis in a model observer that infers parameters using a procedure of approximate physical simulation. While participants' judgments qualitatively matched the model's, they also deviated in key ways that may be explained by resource limitations. This work advances our understanding of how people infer unobserved physical properties, and offers a framework for modeling such behavior in complex, real-world scenes.