A major puzzle in recognition memory has been the process by which participants set reasonable old/new decision criteria when the study and test lists are comprised of items of widely varying types, with differing degrees of baseline familiarity and experience (e.g., words vs. random dot patterns). We present a model of the recognition process that addresses this issue. Its core assumption is that recognition decisions are based not on the absolute value of familiarity, but on how familiarity changes over time as features are sampled from the test item. We model recognition decisions as the outcome of a race between two parallel accumulators: one that accumulates positive changes in familiarity (leading to an "old" decision) and another that accumulates negative changes (leading to a "new" decision). Simulations with this model make realistic predictions for recognition performance and latency regardless of the baseline familiarity of study and test items.