Face to Face : facial close-ups and joint attention in Science and the Visual Arts

This essay examines the contrasting visions of the expressive powers of the human face—both from neuroscientific approaches rooted in Darwin which argue for a codified system of six basic emotions universally recognized (happiness, sadness, anger, fear, disgust, surprise) and from the visual arts of cinema, television and portraiture painting that rely on facial close-ups to represent an emotional fluidity that is always subjective. As a means of reconciling the two approaches, it turns to the current study of infants by developmental psychologists (like Peter Mundy and Daniel Stern) who stress the importance of an infant‘s ability to read the mother‘s face, which facilitates joint attention, the acquisition of verbal language and social interactions with the world. Although Stern‘s imaginative dialogues sound literary and subjective, his description of the infant‘s encounter with the mother‘s face is actually consistent with the explanation by neuroscientist Antonio Damasio (1999) of how consciousness is first launched in the ―core self.‖ By treating the mother‘s face as the crucial object in the infant‘s early development and by perceiving this encounter awash in reflective feelings (which Damasio distinguishes from basic emotions shared with other species), he helps explain the dichotomy between the two systems of emotive facial expressions: reading the specific codified emotions (in humans and other species) versus experiencing the flow of (what Damasio calls) ―background feelings‖ that continuously play across the human face. By emphasizing the theories of Béla Balázs and films of Ingmar Bergman and Chick Strand, which literally teach us how to read these background feelings moving across the human face, this essay claims facial close-ups do not distract us from our social circumstances or political action as Walter Benjamin argued. Instead they can have an ideological edge in a wide range of genres as they enable us to see this emotional engagement in joint attention both as a form of interpellation and as a means of survival—not only for infants but for all those engaged with the visual narrative arts.


The Expressive Powers of the Human Face
Facial expression is the most subjective manifestation of man, more subjective even than speech, for vocabulary and grammar are subject to more or less universally valid rules and conventions, while the play of features… is a manifestation not governed by objective canons, even though it is largely a matter of imitation.The most subjective and individual of human manifestations is rendered objective in the close-up….The expression of the whole face cannot cover up the expression of its details, if these details betray a different, more profound truth.(BALÁZS, 1945) The evidence suggests that the emotions of all normal members of our species are played out on the same keyboard.The most accessible signs of emotions are candid facial expressions.In preparing The Expression of the Emotions in Man and Animals, Darwin circulated a questionnaire to people who interacted with aboriginal populations on five continents.…Darwin summed up the responses: -The same state of mind is expressed throughout the world with remarkable uniformity.‖Contemporary research has borne out his conclusion.When the psychologist Paul Ekman began to study emotions in the 1960s, facial expressions were thought to be arbitrary signs that the infant learns when its random grimaces are rewarded and punished.If expressions appeared universal, it was thought, … no culture was beyond the reach of John Wayne and Charlie Chaplin.Ekman assembled photographs of people expressing six emotions.He showed them to people from many cultures… Everyone recognized happiness, sadness, anger, fear, disgust, and surprise.(PINKER, 1997) How do we reconcile these contrasting visions of the expressive powers of the human face?On the one hand, the -scientific‖ approach has identified a specific location in the brain that controls facial recognition and a brief list of emotions that generate facial expressions which are universally recognized across diverse cultures.
Rooted in Darwin's narrative of natural selection, this codified system is the basis for security systems relying on facial recognition, for animation software that makes the depiction of the human face more realistic, and for the current on-line popularity of emoji that slightly expand both the emotional and cultural range of the author's facial expressions.On the other hand, we have all experienced the free play of feelings that moves with great fluidity across the human face, particularly in facial close-ups at the movies or on television.For many, these subtle expressions are the essence of good acting.For Hungarian film theorist Béla Balázs (1945) they led him to argue that the human face is the greatest resource for cinematic art and to apply this theory (not to

Lumina
Face to Face: facial close-ups and joint attention in Science and the Visual Arts Juiz de Fora, PPGCOM - UFJF, v. 11, n. 2, p. 36-59, mai./ago. 2017 39 John Wayne and Charlie Chaplin like Pinker) but to Maria Falconetti in Carl Dreyer's 1928 silent classic, La passion de Jeanne d'Arc.
In the last years of the silent film we saw… conversations between the facial expressions of two human beings who understood the movements of each others' faces better than each others' words and could perceive shades of meaning too subtle to be conveyed in words….

The Mute Dialogue between Mother and Child
A provocative answer is provided by developmental psychologist Daniel Stern (1990), who presents a phenomenological (rather than psychoanalytic) account of an infant's early encounters with his mother's face and explores what's at stake emotionally in the interaction [2].Although this interaction occurs before the child acquires verbal language, Stern speaks for the infant (whom he calls Joey), presenting the newborn's point of view in a poetic text that expresses his perceptions and feelings.This phenomenological strategy suggests the infant's emotional life and survival actually depend on his ability to read the mother's face.Though, at first, he treats her face as an external object, he gradually realizes he is being transformed by this crucial encounter which will shape his subsequent interactions with the world.
I enter the world of her face.Her face and its features are the sky, the clouds, and the water.Her vitality and spirit are the air and the light.It is usually a riot of light and air at play.But this time when I enter, the world is still and dull.Neither the curving lines nor the rounded volumes are moving.Where

Lumina
Face to Face: facial close-ups and joint attention in Science and the Visual Arts is she?Where has she gone?I am scared.I feel that dullness creeping into me.I search around for a point of life to escape to.I find it.All her life is concentrated into the softest and hardest points in the world-her eyes.…I stare into their depths.And there I feel running strong the invisible currents of her vitality.They churn up from those depths and tug at me.I call after them to surface, to see her face again, alive.(STERN, 1990, p. 58) Stern treats mother and child as partners in a dance, a performance that arouses a desire for what developmental psychologists call -joint attention‖-our ability -to coordinate our attention with someone else…in order to talk about the same topic, to be able to shift topics, to know when somebody else is attending to something else either mentally or not‖ (MUNDY, 2013).It's a -mute dialogue‖ between two faces, a dialogue worthy of Carl Dreyer's cinema, as described by Béla Balázs, but emphasizing fusion rather than conflict.According to Stern, In reaching across to touch him, her smile exerts its natural evocative powers and sets in motion its contagiousness.Her smile triggers a smile in him and breathes a vitality into him.It makes him resonate with the animation she feels and shows.Her joy rises.Her smile pulls it out of him.Then Joey himself fully releases it from inside… He is both responding and identifying now.(STERN, 1990, p. 65) In contrast to the painful interactions in Bergman's Persona and Cries and Whispers, these moments of joint attention-of contagion and alternation, projection and identification-are welcomed rather than feared.
Once a pair of smiles has passed between a mother and a baby of this age, a process has already been set in motion.What happens is this.Joey's smile and his mother's are slightly out of phase with each other….By remaining out of phase they keep restarting the other and prolong the duet….Such an alternating pattern between mother and baby becomes common after three months.It occurs in vocalizing back and forth as well as in smiling back and forth.It is the baby's first and principal lesson in turn taking, the cardinal rule for all later discourse between two people.(STERN, 1990, p. 66) The lack of eye contact in this early stage is one of the first signs of autism, for joint attention prepares the child for all future identifications and interactions with others.Recent research in child development has shown, once a child has mastered joint attention, it's then time to take the lead-to be the one who initiates the interest in a new object and who makes sure his mother's eyes will follow.This alternation soon leads to vocalizations that similarly go back and forth and then to the acquisition of language, which enables him to become more fully immersed in culture.According to Peter Mundy (2013), Director of Educational Research at the MIND Institute in Sacramento: When babies come into the world they're not able to coordinate their attention with somebody else….It develops between about 4 months and 24 months of life... First, we learn to follow the gaze and follow the attention of somebody else.That's like learning to comprehend language.But then we start to be creative, we start to generate language and we also start to generate bids to direct the attention of other people.(MUNDY, 2013) are the ideological implications of this condition, or of cinematic facial close-ups that capture and convey underlying feelings?
The Ideological Implications of Facial Close-ups Probably the best answers to these questions concerning ideology were offered by Marxist philosopher and cultural critic Walter Benjamin (1936), who (unlike Balázs) focused on the impact of the close-up itself rather than its frequent subject, the human face.Linking the cinematic close-up with psychoanalysis, he claimed both turned the invisible into a visual spectacle.Arising around the same historical moment, cinema and psychoanalysis introduced us to unconscious optics and impulses respectively.While both provided new insights and pleasures, they also distracted us from the material conditions of our life.By turning our attention inward, they made us less attentive to the outside world and its discontents and less likely to engage in meaningful political action or even fulfill his famous dictum-the best way to acquire a book is to write it yourself.
By close-ups of the things around us, by focusing on hidden details of familiar objects, by exploring common place milieus under the ingenious guidance of the camera, the film, on the one hand, extends our comprehension of the necessities which rule our lives; on the other hand, it manages to assure us of an immense and unexpected field of action.Our taverns and city streets, our offices and furnished rooms, our railroad stations and our factories appeared to have us locked up beyond hope.Then came film and burst this prison-world asunder by the dynamite of the tenth of a second, so that now, in the midst of its far-flung ruins and debris, we calmly and adventurously go traveling.(BENJAMIN, 1936, p. 745-747) Benjamin concedes that the cinematic close-up enables us to isolate and analyze details -much more precisely and from more points of view than those presented on paintings or the stage,‖ an ability that promotes -the mutual penetration of art and science.‖Yet this interaction still distracts us from political engagement, which he deems more important, particularly since he was writing as a German Jew during the rise of fascism.This reaction may help explain why there was such a leftist backlash against Bergman when he died in 2007, the same year that Italian filmmaker Michelangelo Antonioni also passed.Despite the aesthetic power and formal originality of both filmmakers, whereas Antonioni's films were widely praised for their prescient Marxist insights about global politics, Bergman's were dismissed for their intense yet narrow focus on bourgeois characters trapped within a psychoanalytic framework.when he is about to be whacked and finally realizes that his opponents are IRA terrorists rather than local mobsters, that this movie is a European political thriller rather than an Anglicized gangster film, and that he has seriously misunderstood the power dynamics both of the genre and of globalism-realizations he conveys solely with facial expressions.
These ideological facial close-ups are not restricted to cinema but also appear on television.Yet the conventions for treating facial close-ups are different within that medium, for actors, journalists and interviewees are all encouraged to look directly into the camera so they can directly address the viewer.During the early days of television, the smaller screen made the reliance on big -talking heads‖ all the more crucial, enabling them to function like an expanding system of emojis.
Increasingly, it has enabled them to be recognized as a reliable source of transmedia information not only in dramas, documentaries, science shows and news but also in commercials and commentary.
The serial structure of television dramas also provides a more expansive narrative field, which makes the emotional transformation of characters more plausible.We can see this effect in a brilliant crime series like David Simon's The Wire (2002)(2003)(2004)(2005)(2006)(2007)(2008), which presents not only a systemic study of corruption in the city of Baltimore but also an ensemble of emotionally compelling characters (on both sides of the law) who are struggling to understand the changing urban dynamics and survive.When charismatic drug dealer Stringer Bell (Idris Elba) tries to go legit, despite his intelligence and driving ambition, he is still too naïve to understand the power dynamics driving the so-called legitimate worlds of business, law, and politics.
As in Hoskins's transformative fatal close-up at the end of The Long Good Friday, Stringer's last-minute gain in systemic understanding-represented in a stunning close-up that shows an array of thoughts and feelings playing across his facereconciles him to his premature death.In both cases, this leap in understanding is a fitting tragic payoff for their respective transformations.Yet, The Wire's broader narrative field enables Stringer's death to be merely another stage in the on-going process of social change.

Documentary: The Case of Soft Fiction (1979)
Whether viewed in a movie theater or on television, on a computer, ipad or any other electronic device, a number of films reinforce the neuroscientific approach to narrative and its reliance on the facial close-up as a means of conveying emotions and cultural values.This dimension is a particularly valuable resource in documentary-whether of the traditional, ethnographic or experimental variety.An excellent example is Chick Strand's Soft Fiction (1979), a short film that documents the empowering qualities of storytelling, particularly for women.
Soft Fiction can be seen as an anthology of excerpts from serial autobiographies, all of which demonstrate how narrative empowers individuals (in general) and women (in particular) to take control over their lives, even in the most dangerous circumstances.It presents six women telling stories, five of which focus on a traumatic situation: an incestuous relationship with one's grandfather; a struggle against addiction-first to a dangerous lover and then to heroin; a gang rape by cowboys at a rodeo; the march to internment in a concentration camp; and a performance of Hayden's Death and the Maiden.The film shows how it is possible for these women to reject the role of victim and take control of their own experience through the act of storytelling-even if they are merely performing a story (like the gang rape or encounter with Death) allegedly written by someone else.
By leaving this question open (of who was the woman at the rodeo), Strand encourages us to question the truth value of the entire film and its multiple narrators.
This ambiguity brings to mind neuroscientist Michael Gazzaniga's (1998)  the -Interpreter,‖ an individual‗s consciousness located in the left hemisphere of the brain, which -constantly establishes a running narrative of our actions, emotions, thoughts, and dreams… and creates our sense of being a whole rational agent.‖Though it is not limited to the truth and usually presents us as more powerful than we actually are, his Interpreter -produces the wonderful sensation that our self is in charge of our destiny‖ (GAZZANIGA, 1998, p. 174-175)-which is precisely the feeling we get in watching Soft Fiction.
Thus, one meaning of the provocative title is the softness of the boundary between truth and fiction.We know that Hayden wrote -Death and the Maiden‖, yet this narrative is performed by a female singer and by the filmmaker, two women who bring their own emotions and memories to this powerful collaborative performance.
And we know that one of the narrators claims the event she is narrating (sex with several cowboys) didn't actually happen to her; like us, she is merely reading the text.
Yet her comic tone and rejection of the role as victim are narrative strategies that are compatible with her disclaimer about authorship.And if she is the author, then it's also possible that she was a consenting participant in the sex.Her disclaimer also raises the issue of collaboration: to what extent does her empowerment depend on her identification with the spectators of the film as well as with those who -perform‖ the other tales.Although the other women don't deny their authorship or the veracity of the remaining narratives, there's still the possibility that they are all fiction, particularly since these women are all performers-actresses, filmmakers, writers or singers.As in the case of Gazzaniga's left brain Interpreter, truthfulness seems less important than the narrative function of the tale-the empowering effect of narrating and thereby controlling the traumatic event.
The title Soft Fiction also evokes -soft core‖ porn, a genre related to the tales of incest, sexual addiction, and gang rape and to the many depictions of female nudity.Though it also suggests softness (like weakness and emotionality) is part of the female stereotype, here it is redefined as a positive term, especially in contrast to the hardness of males.In fact, Strand had intended to do a sequel titled Hard Fiction (and had asked my husband to contribute a story) but never got around to making it.
Still, her choice of visual style is anything but soft.
Most of these women are presented in huge, stark black and white facial close-ups that frequently capture only part of their face.These shots are intercut with expressive close-ups of their hands.We read these gestures to gauge the truthfulness of the tale, and to establish the tone (or what Damasio calls -background feelings‖).
As with Bergman's gigantic close-ups, they put us in the position of the child gazing up at our mother's face, on the path to joint attention and language acquisition.Ironically, although the sixth episode seems to be an exception to this reading, it actually gets to its core.As the first story we hear in the film, it trains us how to read the others.As if drawn by a magnetic invisible presence, the camera slowly approaches a woman (film theorist Beverle Houston), wearing a black gown, seated alone in a room, smoking.As she begins to describe a strange experience she recently had at a Pasadena museum, the film cuts to a huge facial close-up that features her eyes and her mouth and that is intercut with close shots of her fingers holding the cigarette.She confides that when she touched the spiral handrail of a staircase at the museum, she suddenly had a strong desire to become that objectthat smooth, coiled railing.And she felt that physical wave of desire moving through

Lumina
Face to Face: facial close-ups and joint attention in Science and the Visual Arts Juiz de Fora, PPGCOM -UFJF, v. 11, n. 2, p. 36-59, mai./ago.2017 51 her body-she felt it in her cells and in her small parts, until it made her bones go soft.
Hardly traumatic, this story makes it difficult to see her as a victim.Yet it uncannily illustrates Damasio's (1999) description (already quoted above) of the crucial encounter with an object that launches the -core self.‖And it also helps us understand his distinction between emotions (which are physical and which we humans share with other animals) and feelings (which provide a self-reflexive look at our own emotions and the way our body experiences them).Beverle had told many of her friends about this experience (including me, her collaborator with whom she had written two books and several essays).But only Chick recognized its significance -Beverle's extraordinary ability to describe and interpret her own feelings with such physical and mental precision.And five years later when Beverle was diagnosed with colon cancer, the physical shape of that coiled railing and its spiraling curves took on new meaning.And another five years later when she died of that disease, her story took on even greater significance, now prefiguring the final performance of -Death and the Maiden‖ and serving as a perfect introduction to those other tales of survival.
Did filmmaker Chick Strand know all this?Did her roots in ethnographic filmmaking help her, or her own experience of personal trauma that she never overcame?Did she somehow see or intuit the connections to neuroscience?Probably not, since the film was made long before Damasio published his theories.But Chick Strand instinctively went for these narrative strategies and expressive facial close-ups because she could feel and harness their emotional impact.

Face to Face: from Masks to Mona Lisa
I don't mean to imply that this revelatory reading of the human face began with cinema.It can also be found much earlier in the making of masks throughout many parts of the world and in portraiture painting, two visual media in which it's the fixed nature of the facial expression that is the source of its power rather than its fluidity as in cinema and television.
With a collection of over a hundred masks on display on the walls of my living room, I often think about their relationship to the facial close-up.Although masks are static, they are usually part of a codified system that either affirms one's membership within a tribal group or sub-culture, or prevents others from recognizing your unique identity, which is more typical of modern masquerades.Because of their frequent association with dance and ritual performance, there is always the potential contrast between the moving body and the static face hidden behind the mask.Right -The J. Paul Getty Museum website [6] One could easily read these dynamics in Althusserian terms, as an interpellation of the spectator as a devoted subject eager to absorb the values of a Christian world view.Yet, the dynamics in all three portraits also evoke the early visual bond between mother and child, a humanizing joint attention that facilitates the infant's ability to identify and empathize with the parent and eventually imitate and acquire her use of gestural and verbal language.Far from being contradictory, these two readings suggest that joint attention may be the earliest form of ideological interpellation and an empathic theory of mind.It's the conflation of these two contexts that make these paintings as powerful as the cinematic facial close-ups in

Figure 2 -
Figure 2 -Two facial close-ups of the nurse and actress from Bergman's Persona, first side by side, then fused.

Figure 3 -
Figure 3 -Reading the human face in Cries and Whispers.Boy reaching up to image of his mother's face in Persona.

Figure 5 -
Figure 5 -One narrator raises doubts about the rodeo narrative while Beverle Houston trains us how to read the others in Chick Strand's Soft Fiction.
With a filmmaker like Bergman, who specializes in the gigantic facial close-up, we see how he can deftly turn these expressive shots into moving masks whose emotional gestures we are eager to read-like the demon in Hour of the Wolf removing her rubber face; or the half-masked performers in The Rite(1969), who frighten their critical judge to death; or the mute actress listening to music in Persona, where the slow, subtle movement of her face is almost imperceptible.By including such images, his films constantly make us wonder what lies beneath the mask.Instead of making us look forward to television or computers, his films make us look backward in time-not to the theater, a medium in which he had a rich career but which has no close-ups-but to painting, and more specifically to portraiture, which is also concerned both with capturing and complicating the unique emotive power of the human face.In portraiture, we find dynamic moves that enhance the expressive power of the human face, especially in Leonardo DaVinci's intriguing portrait of Mona Lisa, whose emotive ambiguity almost disavows the painting's static nature.His famous portrait owes some of its power to Renaissance diptych painting, which was featured in a 2014 exhibition at the Huntington Library in San Marino, California.Titled Face to Face: Flanders, Florence, and Renaissance Painting, the show was designed to demonstrate the importance of Flemish portraiture during the Renaissance and its influence on Italian Masters, including DaVinci.The exhibit may leave some of us wondering whether the Renaissance diptych and its legacy are the basis of the Lumina Face to Face: facial close-ups and joint attention in Science and the Visual Arts Juiz de Fora, PPGCOM -UFJF, v. 11, n. 2, p. 36-59, mai./ago.2017 53 cinematic facial close up-of both its spiritual and secular powers-particularly in a work like Dreyer's Passion de Jeanne d' Arc.In her -Introduction‖ to the catalogue, co-curator Catherine Hess (2012, p. 9) defines the diptych as -two hinged painted panels that could open and close like a book,‖ a portraiture format that -became a widespread art form in fifteenth-century Flanders‖.Often the diptych established a significant connection between two faces, one linked to the world of spirit and the other to the material world of commerce usually evoked in the surrounds.Hess observes: -One panel displayed a religious subject [frequently the Madonna and Child] while the other contained a portrait of the donor who had commissioned the work.‖For example, in the diptych of Maarten Nieuwenhove by Hans Memling (1487), we see a three-quarter portrait of a young man praying, as he gazes intently at the Virgin Mary and Infant Jesus in the adjacent panel.Over time, stylistic changes altered the nature of these implied relations, especially as the portraits moved from earlier idealized profiles (considered more modest and therefore more appropriate for female subjects) to the more modern, realistic three-quarter or frontal facial close-ups (whose openness was deemed more suitable for male subjects).Yet, later in Italy in DaVinci's Mona Lisa (1503-16), the secular female subject, whose face is seen in the open three-quarter angle and who is surrounded by a worldly Flemish-style landscape, still retains the captivating spiritual power of the Madonna.

Dreyer' s
Passion de Jeanne d 'Arc and in Bergman's Persona and Face to Face.

Lumina Face to Face: facial close-ups and joint attention in Science and the Visual Arts actress
and then as we hear the exact repetition of the story but this time with expanding facial close-ups of the nurse.Paradoxically, at the same time that Bergman single facial close-up, a goal he came close to fulfilling in his short documentary, Karin's Face (1984)-a film based on stills from his family album, primarily of his mother Karin.

Face to Face: facial close-ups and joint attention in Science and the Visual Arts
How do we ever begin to be conscious?Specifically, how do we ever have a sense of self in the act of knowing?...This account is a simple narrative without words.It does have characters (the organism, the object).It unfolds in time.And it has a beginning, a middle, and an end.The beginning corresponds to the initial state of the organism.The middle is the arrival of the object.The end is made up of reactions that result in a modified state of the organism.(DAMASIO,1999,p. 168)By treating the mother's face as the crucial object in the infant's early development and by perceiving this encounter awash in reflective feelings (which Damasio distinguishes from basic emotions shared with other species), he helps explain the dichotomy between the two systems of emotive facial expressions: reading the specific codified emotions (in humans and other species) versus experiencing the flow of (what Damasio calls) -background feelings‖ that continuously play across the human face.Not restricted to the codified list of six -universal emotions‖ studied by Darwin and other scientists (fear, anger, sadness, disgust, surprise, and happiness), Damasio's list of reflective -background feelings‖ expands to include several binaries such as: fatigue/energy, wellness/sickness, Stern, we are still left wondering what if the mother (or caretaker) is not available for this interaction or simply is more interested in the world outside.And what, if any, Lumina