The present study investigated the relationship between eye movement pattern in face recognition and cognitive performance during natural aging through modeling and comparing eye movement of young (18-24 years) and older (65-81 years) adults using Hidden Markov Model (HMM) based approach. Young adults recognized faces better than older adults. Older adults’ recognition performance, on the other hand, correlated with their cognitive status assessed by the Montreal Cognitive Assessment. Eye movement analysis revealed two different strategies, namely “analytic” and “holistic”. Participants using the analytic strategy had better recognition performance (particularly in the false alarm rate) than those using the holistic strategy. Significantly more young adults adopted the analytic strategy; whereas more older adults adopted the holistic strategy. Interestingly, older adults with lower cognitive status were associated with higher likelihood of using the holistic strategy. These results suggest an association between holistic eye movement patterns and cognitive decline in the elderly.