The Heider-Simmer Illusion

“I argue that we sometimes visually perceive the intentions of others. just as we can see something as blue or as moving to the left, so too can we see someone as intending to evade detection or as aiming to traverse a physical obstacle. i consider the typical subject presented with the heider and simmel movie, a widely studied ‘animacy’ stimulus, and i argue that this subject mentally attributes proximal intentions to some of the objects in the movie. i further argue that these attributions are unrevisable in a certain sense and that this result can be used to as part of an argument that these attributions are not post-perceptual thoughts. finally, i suggest that if these attributions are visual experiences, and more particularly visual illusions, their unrevisability can be satisfyingly explained, by appealing to the mechanisms which underlie visual illusions more generally.”

“Simple displays of moving shapes can give rise to percepts of animacy. these films elicit impoverished narratives in some individuals, such as those with an autism spectrum disorder. however, the verbal demand of producing a narrative limits the utility of this task. non-verbal tasks have so far focused on detecting animate objects. lacking from previous research is a task that relies less on verbal description but more than animacy perception. here, we presented data using a new social interaction judgement task. healthy young adults viewed the heider and simmel movie and pressed one button whenever they perceived social interaction and another button when no social interaction was perceived. we measured the time points at which social judgement began, the fluctuation of the judgement in relation to stimulus kinematic properties, and the overall mean of social judgement. participants with higher autism traits reported lower levels of social interaction. reversing the film in time produced lower social interaction judgements, though the temporal profile was preserved. our study suggests that both low-level motion characteristics and high-level understanding contribute to social interaction judgement. the finding may facilitate future research on other populations and stimulate computational vision work on factors that drive social judgements.”

“Human theory of mind (tom) is so automatic and pervasive that we spontaneously attribute mental states to animated abstract shapes, as evidenced by the classic heider–simmel findings. the extent to which this represents a fundamental characteristic of primate social cognition is debated. prior research suggests that monkeys spontaneously predict behavior and attribute basic goals to conspecifics, but it remains unclear whether, like humans, they spontaneously ascribe mental states to animated shapes. here, we address this question by analyzing rhesus monkeys’ viewing patterns of the classic heider–simmel animations. we hypothesized that if rhesus monkeys also spontaneously attribute mental states to animated shapes, then, like humans, they would have the longest fixation durations for theory of mind animations, medium duration fixation for goal-directed animations, and shortest fixations for animations with random motion. in contrast, if attributing mental states to animations is specific to humans and perhaps other apes, then we predict no differences in looking time across animation categories. unlike humans, monkeys did not fixate longer on tom videos. critically, monkeys’ viewing patterns did not correlate with humans’ viewing patterns or intentionality ratings from previously published research. the only major difference in viewing patterns between animation categories tracked differences in low-level visual motion. thus, monkeys do not view the classic heider–simmel animations like humans do and we found no evidence that they spontaneously attribute mental states to animated shapes.”

“In 1944, heider and simmel reported that observers could perceive simple animated geometric shapes as characters with emotions, intentions, and other social attributes. this work has been cited over 3,000 times and has had wide and ongoing influence on the study of social cognition and social intelligence. however, many researchers in this area have continued to use the original heider and simmel black-and-white video. we asked whether the original findings could be reproduced 75 years later by creating 32 new colored animated shape videos designed to depict various social plots and testing whether they can evoke similar spontaneous social attributions. participants (n = 66) viewed our videos and were asked to write narratives which we coded for indicia of different types of social attributions. consistent with heider and simmel, we found that participants spontaneously attributed social meaning to the videos. we observed that responses to our videos were also similar to responses to the original video reported by klin (2000), despite being only 13–23 s and portraying a broader range of social plots. participants varied in how many social attributions they made in response, and the videos varied in how much they elicited such responses. our set of animated shape videos is freely available online for all researchers to use and forms the basis of a multiple-choice assessment of social intelligence (brown et al., 2019).”

“Two distinct literatures have emerged on the functionality of the anterior temporal lobes (atl): in one field, the atls are conceived of as a repository for semantic or conceptual knowledge. in another field, the atls are thought to play some undetermined role in social-emotional functions such as theory of mind. here we attempted to reconcile these distinct functions by assessing whether social semantic processing can explain atl activation in other social cognitive tasks. social semantic functions refer to knowledge about social concepts and rules. in a first experiment we tested the idea that social semantic representations can account for activations in the atl to social attribution stimuli such as heider and simmel animations. left atl activations to heider and simmel stimuli overlapped with activations to social words. in a second experiment we assessed the putative roles of the atls in the processing of narratives and theory of mind content and found evidence for a role of the atls in the processing of theory of mind but not narrative per se. these findings indicate that the atls are part of a neuronal network supporting social cognition and that they are engaged when tasks demand access to social conceptual knowledge. © 2009 elsevier inc. all rights reserved.”

“Certain simple visual displays consisting of moving 2-d geometric shapes can give rise to percepts with high-level properties such as causality and animacy. this article reviews recent research on such phenomena, which began with the classic work of michotte and of heider and simmel. the importance of such phenomena stems in part from the fact that these interpretations seem to be largely perceptual in nature – to be fairly fast, automatic, irresistible and highly stimulus driven – despite the fact that they involve impressions typically associated with higher-level cognitive processing. this research suggests that just as the visual system works to recover the physical structure of the world by inferring properties such as 3-d shape, so too does it work to recover the causal and social structure of the world by inferring properties such as causality and animacy. copyright (c) 2000 elsevier science ltd.”

“People are adept at perceiving interactions from movements of simple shapes, but the underlying mechanism remains unknown. previous studies have often used object movements defined by experimenters. the present study used aerial videos recorded by drones in a real-life environment to generate decontextualized motion stimuli. motion trajectories of displayed elements were the only visual input. we measured human judgments of interactiveness between two moving elements and the dynamic change in such judgments over time. a hierarchical model was developed to account for human performance in this task. the model represents interactivity using latent variables and learns the distribution of critical movement features that signal potential interactivity. the model provides a good fit to human judgments and can also be generalized to the original heider–simmel animations (1944). the model can also synthesize decontextualized animations with a controlled degree of interactiveness, providing a viable tool for studying animacy and social perception.”

“One of the great feats of human perception is the generation of quick impressions of both physical and social events based on sparse displays of motion trajectories. here we aim to provide a unified theory that captures the interconnections between perception of physical and social events. a simulation-based approach is used to generate a variety of animations depicting rich behavioral patterns. human experiments used these animations to reveal that perception of dynamic stimuli undergoes a gradual transition from physical to social events. a learning-based computational framework is proposed to account for human judgments. the model learns to identify latent forces by inferring a family of potential functions capturing physical laws, and value functions describing the goals of agents. the model projects new animations into a sociophysical space with two psychological dimensions: an intuitive sense of whether physical laws are violated, and an impression of whether an agent possesses intentions to perform goal-directed actions. this derived sociophysical space predicts a meaningful partition between physical and social events, as well as a gradual transition from physical to social perception. the space also predicts human judgments of whether individual objects are lifeless objects in motion, or human agents performing goal-directed actions. these results demonstrate that a theoretical unification based on physical potential functions and goal-related values can account for the human ability to form an immediate impression of physical and social events. this ability provides an important pathway from perception to higher cognition.”

“Although individual differences in the ability known as social intelligence are important to many areas of psychological science, there are few validated measures of this capacity. existing measures often involve complex verbal materials, focus on images of human faces, or are based on self-report judgments. here we describe the development and validation of a new measure of social intelligence: the social shapes test (sst). the sst is inspired by the work of heider and simmel (1944), klin (2000), and recent applications of their work in social and clinical neuroscience studies of mentalizing and theory of mind. each sst item consists of a short video of 4–6 simple geometric shapes that are animated to simulate social interactions, paired with an objectively scored, multiple-choice question. scores on the sst predict performance on measures of social intelligence beyond the effects of cognitive ability. sst scores are also distinct from cognitive abilities and do not appear to vary based on sex, race/ethnicity, or current affect. with just 23 engaging items and a duration of about 10 minutes, the sst is a convenient tool for researchers to measure social intelligence, alone or in combination with other established measures.”

“More than sixty years ago fritz heider and marianne simmel conducted an experimental study, which can be seen as the starting point of attribution theory research. moving symbols were shown during a short animated cartoon which ss unanimously described as living objects (mostly people). this quite often quoted study has been replicated several times in germany in the last few years. these replications show that (1) stories produced by ss are formed by personal motivations of ss, and that (2) content of stories can in part be attributed to the research settings and finally (3) it is shown in recent replications that ss are surprisingly far less inclined to describe the symbols in terms of acting persons or animals than in the original study. possible reasons for these results are discussed.”

“The ability to ascribe mental states, such as beliefs or desires to oneself and other individuals forms an integral part of everyday social interaction. animations tasks, in which observers watch videos of interacting triangles, have been extensively used to test mental state attribution in a variety of clinical populations. compared to control participants, individuals with clinical conditions such as autism typically offer less appropriate mental state descriptions of such videos. recent research suggests that stimulus kinematics and movement similarity (between the video and the observer) may contribute to mental state attribution difficulties. here we present a novel adaptation of the animations task, suitable to track and compare animation generator and -observer kinematics. using this task and a population-derived stimulus database, we confirmed the hypotheses that an animation’s jerk and jerk similarity between observer and animator significantly contribute to the correct identification of an animation. by employing random forest analysis to explore other stimulus characteristics, we reveal that other indices of movement similarity, including acceleration- and rotation-based similarity, also predict performance. our results highlight the importance of movement similarity between observer and animator and raise new questions about reasons why some clinical populations exhibit difficulties with this task.”

“People naturally anthropomorphize the movement of nonliving objects, as social psychologists fritz heider and marianne simmel demonstrated in their influential 1944 research study. when they asked participants to narrate an animated film of two triangles and a circle moving in and around a box, participants described the shapes’ movement in terms of human actions. using a framework for authoring and annotating animations in the style of heider and simmel, we established new crowdsourced datasets where the motion trajectories of animated shapes are labeled according to the actions they depict. we applied two machine learning approaches, a spatial-temporal bag-of-words model and a recurrent neural network, to the task of automatically recognizing actions in these datasets. our best results outperformed a majority baseline and showed similarity to human performance, which encourages further use of these datasets for modeling perception from motion trajectories. future progress on simulating human-like motion perception will require models that integrate motion information with top-down contextual knowledge.”

“The classic animation experiment by heider and simmel (1944) revealed that humans have a strong tendency to impose narrative even on displays showing interactions between simple geometric shapes. in their most famous animation with three simple shapes, observers almost inevitably interpreted them as rational agents with intentions, desires, and beliefs (‘That nasty big triangle!’). much work on dynamic scenes has identified basic visual properties that can make shapes seem animate. here, we investigate the limits on the ability to use narrative to share information about animated scenes. we created 30 second heider-style cartoons with 3-9 items. item trajectories were generated automatically by a simple set of rules, but without a script. in experiments 1 and 2, 10 observers wrote short narratives for each cartoon. next, new observers were shown a cartoon and then presented with a narrative generated for that specific cartoon or one generated for a different cartoon having the same items. observers rated the fit of the narrative to the cartoon on a scale from 1 (clearly does not fit) to 5 (clearly fits). performance declined markedly when the number of items was larger than 3. experiment 3 had observers determine if a short clip of a cartoon came from a longer clip. experiment 4 had observers determine which of two narratives fit a cartoon. finally, in experiment 5, narratives always mentioned every item in a display. in all cases of matching narrative to cartoon, performance drops most dramatically between 3 and 4 items.”

“Recent work suggests that biological motion perception is supported by interactions between posterior superior temporal sulcus (psts) and regions of the posterior lobe of the cerebellum. however, insufficient attention has been given to cerebellar contributions to most other social cognitive functions, including ones that rely upon the use of biological motion cues for making mental inferences. here, using adapted heider and simmel stimuli in a passive-viewing paradigm, we present functional magnetic resonance imaging evidence detailing cerebellar contributions to animacy attribution processes in healthy adults. we found robust cerebellar activity associated with viewing animate versus random movement in hemispheric lobule vii bilaterally as well as in vermal and paravermal lobule ix. stronger activity in left crus i and lobule vi was associated with a greater tendency to describe the stimuli in social-affective versus motionrelated terms. psychophysiological interaction analysis indicated preferential effective connectivity between right psts and left crus ii during the viewing of animate than random stimuli, controlling for individual variance in social attributions. these findings indicate that lobules vi, vii, and ix participate in social functions even when no active response is required. this cerebellar activity may also partially explain individual differences in animacy attribution.”

“Perception of dispositions of others revealed by movement is an essential ingredient of adaptive daily-life social behavior. brain imaging points to several brain regions involved in visual processing of social interaction represented by motion of geometric shapes. however, temporal interrelations among these regions remain unknown. keeping in mind that successful visual social perception depends on intact communication throughout the brain, we focus here on analysis of the induced gamma neuromagnetic response to social interaction revealed by motion. a peak of induced gamma activity of 62 hz was found at 1 s from the stimulus onset over the right parieto-temporal junction. two further enhancements in gamma response of lower frequency of 44 hz occurred at 1.4 s over the medial prefrontal and posterior temporal cortices in the right hemisphere. subsequent boosts of 44 hz were found at 1.6 s over the left temporal and right posterior temporal cortices. for the first time, the findings identify the cortical network engaged in visual processing of social interaction revealed by motion and help to better understand proper functioning of the social brain circuitry. © the authors 2010.”