A previous study (Hwang et al., 2011) found evidence for semantic guidance of visual attention during the inspection of real-world scenes, i.e., an influence of semantic relationships among scene objects on overt shifts of attention. In particular, the results revealed an observer bias toward gaze transitions between semantically similar objects. However, these results are not necessarily indicative of semantic processing of individual objects but may be confounded by knowledge of the scene gist, which does not require object recognition (Torralba et al., 2006), or by known spatial dependency among objects (Oliva & Torralba, 2007). To examine the mechanisms underlying semantic guidance, in the present study, subjects were asked to view a series of displays with the scene gist removed and spatial dependency varied. Our results confirm the previous finding of semantic guidance and show that it is not entirely due to either the effect of scene gist or the spatial dependency among objects. Even without scene gist or spatial dependency, subjects still retrieved semantic information to guide their attention. This strategy may facilitate scene understanding and object memorization.