Left-right asymmetries have been noted in tasks requiring the classification of many visual stimuli, including Navon figures, spatial frequency gratings, and faces. The Double Filtering by Frequency (DFF) model (Ivry & Robertson, 1998), which postulates asymmetric frequency filtering on task-relevant frequency bands, has been implemented to account for asymmetric processing of each stimulus type above, but does not provide a fully mechanistic explanation, nor does it have direct neural correlates. The Differential Encoding (DE) model (Hsiao, Shahbazi, & Cottrell, 2008), which postulates that a known asymmetry in patch connectivity drives visual processing asymmetries, has previously been used to account for only one stimulus type. Here, we refine the DE model to match the published patch asymmetry more precisely and show that the DE model generalizes to three of the four datasets mentioned above. Examination of the failure to match all datasets suggest a possible reinterpretation of the original dataset itself.