The way that we look at images is influenced by social context. Previously we demonstrated this phenomenon of joint perception. If lone participants believed that an unseen other person was also looking at the images they saw, it shifted the balance of their gaze between negative and positive images. The direction of this shift depended upon whether participants thought that later they would be compared against the other person or would be collaborating with them. Here we examined whether the joint perception is caused by beliefs about shared experience (looking at the same images) or beliefs about joint action (being engaged in the same task with the images). We place our results in the context of the emerging field of joint action, and discuss their connection to notions of group emotion and situated cognition. Such findings reveal the persuasive and subtle effect of social context upon cognitive and perceptual processes.