Some researchers have argued that mass perception, causal ascriptions, and predictions in simple billiard ball interactions can be modeled as inductive Bayesian inference over a (noisy) Newtonian representation of the world. However, there are phenomena, such as the asymmetrical ascription of forces to colliding objects, that are conceptually incompatible with the symmetry of Newtonian physics. We propose that human inference in physical scenarios operates over a pre-Newtonian physical representation that is based on impetus intuitions. Impetus theories assume that object movements are caused by an internal force, impetus, that is transferred and reflected when objects collide with each other. Moreover, impetus interactions are inherently asymmetric. We present a mathematical model that implements impetus theory and show that the theory is well suited to model perceived causal asymmetry. Moreover, the theory can also explain phenomena that so far have been presented as unique evidence for (noisy) Newtonian representations.