The Mind's GoldenEye

by Douglas Blumeyer

In GoldenEye (1997), the legendary action game title for the Nintendo 64 platform, the player is represented by a human avatar in a three-dimensional enclosed game space called the level. Using the controller stick, the player can move his avatar in any direction as a relatively normal human could, and his objective is to stay alive while killing as many of his opponents as possible. Each match is a contest, say, "first to five kills," that is, players can and will "die" multiple times during the course of a match, only to be returned anew to the level at one of several "spawn" points. Each of the "runs" the player is granted consists of a single uninterrupted long take from the first person (indicated by the presence of the currently equipped weapon in the bottom right hand corner of the frame), which begins at spawn and ends in death. The multiplayer mode of this game, in which up to four human players engage each other via the same screen split into separate subscreens, presents a new moment in the suture theory – as it now applies to montages of simultaneity – raising issues of subjectivity and embodiment of the "screenwatcher": one who watches the game through not only his own screen, but his opponents' as well.

I'd like to think that Bazin would love GoldenEye. In the concise words of Anne Friedberg, "Bazin's defense of deep focus and the long take was rooted in his emphatic rejection of montage and its 'violations' to the 'realism' of space and time." And Heath mentions Bazin's "version of the spatial realism he ontologically cherished provided by Italian Neorealism; a version that might show the possibilities of the long take away from an absorbed dramatic space." Of course, being interactive, GoldenEye imposes no specific dramatic arc on the avatars; the players are not absorbed by a constantly intervening series of messages from the game designer as would a spectator at a film edited for continuity. They are free to explore the three-dimensional space at will, in "jerk"-less continuity.

Ironically, when dealing in terms of a computer generated image instead of a photographic image, deep focus is not an advanced technology, but rather a technical limitation. While modern combat games in the spirit of GoldenEye such as Gears of War utilize graphic code which actually introduce filmic characteristics to the image such as depth of field, realistically blurring more distant buildings and objects to produce a more immersive atmospheric environ, GoldenEye's limited technology and memory resources do not allow for such effects, confining it to primitive polygonal modeling, resulting in silly bounding boxes and other highly simplified geometry, only barely compensated for by repetitive texturing mapped onto their surfaces, with no shaded relief. This geometry regresses to an obvious single vanishing point and foregrounds this fact with both its simplicity and infinitely deep focus. Stephen Heath notes that "deep focus allows composition for a high degree of perspective," and so it is in GoldenEye, an extreme case of the Quattrocento system; an image which is more its three-dimensional determinants than its two-dimensional result. This is partially by design, allowing players to embrace the skill required to engage opponents in three-dimensional combat, on one of the first games for one of the first gaming platforms to be designed primarily for three-dimensional gaming. Again, while not "realism" in the sense of creating the illusion of naturalism, GoldenEye foregrounds elements of Bazin's desire for the integrity of spatiotemporal experience.

So, unlike a typical cinematic photograph, a screenshot of GoldenEye gives a very strong spatial impression. Rudolf Arnheim describes the cinematographic image as a combination of its "actual happening" and "pictureness." The "actual happening" is the three-dimensional spatial impression aspect, which allows the spectator to feel embodied in both space and time within each individual take, as opposed to the "pictureness," which is what allows the spectator to break that embodiment and understand it as part of a montage sequence in the service of a greater discourse. "If film photographs gave a very strong spatial impression," Heath suggests, "montage probably would not be possible." So because of the downplay of the "pictureness" in the primitive computer-generated GoldenEye image, and analogous emphasis on the "happening," montage of images in GoldenEye would become fundamentally quite difficult on the reconstructive machinery of the spectator.

But because the experience of a single GoldenEye player-avatar never is played out directly from any angle other than an embodied first person as described before or an optional "over-the-shoulder" just outside the body first person variant, it is silly to speak of montage within a single player's gaming experience. But it is worthwhile to explore a form of proto-montage a player can activate for himself within a multiplayer match. Since GoldenEye was developed for the Nintendo 64 platform, which predated the internet-capable console, when one competes in a multiplayer match, one is competing against either computer intelligences or other humans actually in the room. All players, therefore, regard the same screen as the interface with their game avatar. Consequently, any player may look at an opponent's screen, with little more effort than looking a few inches above, below, or to the side. Despite the aforementioned theoretical difficulty of montage of the GoldenEye image, in this case, the increase in spatial impressioning does not interfere with the transfer of presence in one image to the next: because the orientation of the view to the perspective is exactly identical in every frame, to cut oneself from one sequence to another is more comparable to a jump cut than a montage of images from varying distances, focal lengths, angles, etc. These implications are tremendous: to look away from your screen and at an opponent's is to easily cut from one position in the three-dimensional level to another, to break the integrity of the spatiotemporal continuity which Bazin "cherishes." And to do so frequently throughout the course of the match is to experience the match as a montage, to play as a "montagist."

Anne Friedberg proposes an intriguing interpretation of what it means to be a television viewer, when she suggests that "the armchair televisual viewer is a montagist, composing a sequenced view from a database of channels and delivery formats." Subsequently, Friedberg proposes that this same televisual viewer, now experienced as a montagist, is as a "film spectator… increasingly equipped to engage with such fractures in attention; televisual spectatorship much more directly encouraged the habits of a split-attentive viewer… channel switching… implied the inherent potential to engage in a mode switch." Our modern media-oriented brains are accustomed to accepting the fact that multiple streams of information are possible to tap into at any given moment, and comfortable with control over which one we choose to engage with, and the rapidity with which we allow ourselves to move between them. There exists extensive literature on the increase in rapidity of cutting in the film editing aesthetic to reflect the popularity of television and specifically movies presented on television. More literally and completely embracing this aesthetic is the recent film Timecode, which is presented in the same four-way split screen of simultaneous and co-present long takes (93 minutes each!) as is the GoldenEye multiplayer mode. The film reflexively addresses its own viewing by opening one of its stories with an image of yet another four-way split screen at a security guard's console, as if to say that the act of watching this film is going to be like playing the role of a security guard. Friedberg explains in her discussion of Timecode that "despite the assumption that even in multiple display one watches only one screen at a time, we actually watch all of the screens at the same time. Rather than demonstrate our split attention, the film demonstrates our ability to follow all four screens."

I take issue with her suggestion that the spectator can read for meaning all four screens at the same time. I find a near self-contradiction in this statement, considering that she herself points out that the sound in Timecode is mixed so that at any given moment three of the images are nearly silent, directing our attention at the scene that is heard.[1] Furthermore, I find a troublesome ambiguity between her explanation of reading simultaneous visual texts as both separate and together. Though her statement is specifically about a computer monitor with multiple open windows, she posits that in it one "can be two (or more) places at once, in two (or more) time frames, in two (or more) modes of identity, in a fractured post-Cartesian cyberspace, cybertime," (my emphasis) yet in the same environment one is also be placed in "several contexts at the same time… [ones] identity on the computer is the sum of [ones] distributed presence" (my emphasis). This ambiguity between fractured and sutured experience warrants exploration, and I intend to do so now through the guise of the GoldenEye-multiplayer-match montagist.

"When sequence yields to the simultaneous, one is in the world of structure and configuration," Friedberg states, and this couldn't be more applicable to my transfer of discourse from the sequential televisual montagist to the simultaneous GoldenEye multiplayer montagist, especially since there remains between these two personas the control that the feeds which provide the sources of material for the montage continue to exist before and after the edits.

Before I can proceed, I must discuss some basic elements of the sensations of game play and consequential strategy to performing well in a GoldenEye multiplayer match, by way of suture theory.

Stephen Heath proposes a firm distinction between the concepts of "space" and "place," the former being the profilmic three-dimensionality of the film set, and the second being the experience of the spectator in the narrative arc. He would explain a continuity editor's resolve to use over-the-shoulder and matches on gaze as tools in service of constructing a unity "conceived from the narrative models of the novelistic that cinema is dominantly exploited to relay and extend." That is, if we were to read a scene editing for continuity as a text rather than watch it as a film, our minds are, in a process we take for granted, noting the positions of symbolic references to objects and characters in the subject and object positions in sentences, using the author's pacing and positioning of the words within sentences to immediately weight them and juggle them around on our "field of consciousness"[2] as they come and go and our eyes scan from one sentence to the next. Heath quotes Burch: "It was necessary to be able to film objects close up – to isolate a face, a hand, an accessory (as the discourse of a novel does) – but avoiding any disorientation of the spectator in respect of his or her own 'reasoned' analysis of the spatial continuum." The continuity film, that is, aspires to recreate for the spectator the same proto-visual experience of reading the same information, not literally displayed on the screen, but indirectly through flagging the same concretes and abstracts at rates corresponding to the relative speed of reading them in order. Thus, the conventional continuity film no more aspires to give the viewer a concrete sense of embodiment in the three-dimensional "space" of the set at which it was filmed than a paragraph in a fiction novel does; the goals of construction of space in a continuity film are fundamentally narrative, and only incidentally reconstructive: "The vision of the image in its narrative clarity and that clarity hangs on the negation of space for place… negatively, the space is presented so as to not distract attention from the dominant actions… important settings, character traits ('psychology'), or other causal agents." What space does get reconstructed in such a practice is considered by Arnheim "neither absolutely two-dimensional nor absolutely three-dimensional, but something between" – only three dimensional enough such that the viewer can suture the views together into a coherent a whole. "It is this process of construction, indeed," Heath says, "which is often regarded as the power of cinema… the spectator, victim of the 'trick effect,' spontaneously perceives the space as unitary," but of course in service of the place.

Bazin explains that montage projects information "onto the field of consciousness of the spectator." That is, he believes that there is an informational field in our consciousness on which is tended all of the information the filmmakers are providing us. The type of information can range from abstract intellectual information, such as metaphors generated by Eisensteinian montage, to simply a construction of the diegesis from the use of continuity editing montage. This "field" of information is simultaneous and overlapped with all basic sensory information. This is not paradoxical, and can be illuminated by a simple test. Imagine the image of your own face. Though you now certainly have some form of proto-visual information stored on your field of consciousness – you are "seeing" yourself in some capacity – this is not interfering with your ability to continue to see this text and other things in your more literal visual pyramid; we can see both simultaneously. In film, this intersection is facilitated by the nature of the apparatus; though the information on the screen is the format that our eyes see, it is manipulated in ways that only our mind's eye can do. As Metz puts it, I do not feel strange when the camera pans but my neck does not turn, because of my "identification with the movement of the camera being that of a transcendental, not an empirical subject." By the same token, when I watch a continuity film and am asked to suspend a couple over-the-shoulder shots of characters, and punch-ins on detail (probably eyeline matches) over the master shot, as well as information about character's motives and immediate intents in the form of information, these are as well reducible to the same simultaneity of visual and proto-visual on my mind's eye.

Continuity editing emphasizes the "extreme importance of providing an overall view, literally the 'master shot,' that will allow the scene to be dominated in the course of its reconstruction." The master shot does not have to come absolutely first, but relatively soon, so that the spectator does not remain disoriented very long. The returning viewer of a film has advantages over a newcomer; being already familiar with everything from plot points to character traits to visual signs, the reconstruction of narrative place is less hampered by sorting elements of space which may turn out to be extraneous. The returning viewer already has the master shot archived in his memory of images, ready to contextualize a shot in a scene even if it precedes the master shot chronologically. In the same sense, the frequent GoldenEye gamer has archived both solid three-dimensional models of each level (scene), as well as two-dimensional mini-sequences, which provide him with an advantage over the "newbie" player.

Here I will explain what I mean by a mini-sequence. The typical GoldenEye level consists of a set of rooms and a series of corridors. Because of the narrowness of the corridors and the limitations of the primitive gameplay and rendering, there are only so many ways to move from one end of a corridor to another, and all ways look roughly the same. Therefore, once one has entered a corridor with a mental commitment to making it to the other side of the corridor, this moment in the game play no longer proceeds in a richly interactive way. It proceeds as a segment of meaning in another form of proto-montage which will eventually constitute one "run" through a match. And repetition of a mini-sequence will also effect for the player an accrued inventory of associations between images from different locations in the level: the player will learn, if not consciously and three-dimensionally that the Bunker Room is next to the Hallway with All the Crates, then at least subconsciously by frequent temporal proximity and causal association of the two-dimensional images of these rooms with each other.

The spaces in GoldenEye are in stark contrast to those of more modern three-dimensional games such as Superman Returns and Grand Theft Auto, which collectively are known as "sandbox" games, for the reason that the game designers have provided a kind of open world with little necessary direction imposed on the gamer. The limitations of the memory in earlier games such as GoldenEye necessitate that minimalistic décor is used, and to compensate and justify a room from a design standpoint, each must have a defining, distinguishing functional characteristic, usually in the form of a special item contained therein. The gamer feels the deliberate hand of the game designer here, but is not insulted by it; he embraces it not as an element compromising reality, but yet another abstract element to a game which more incidentally represents real life than aspires to recreate it. In this sense, the place and space of GoldenEye are identical. The possible narrative arcs of runs are equivalent with the spatial aspect of their existence. When a gamer familiar with the level enters a room, he knows what the room's item is and where in the room it lay. Thus, even rooms rarely get explored to their full extent, being reduced by "natural selection" (I will unpack this term shortly) to their most efficient and coherent use. I feel compelled to repeat this key quote from Heath: "The vision of the image in its narrative clarity and that clarity hangs on the negation of space for place… negatively, the space is presented so as to not distract attention from the dominant actions… important settings, character traits ('psychology'), or other causal agents."

As Heath demonstrates through an exploration of Yasujiro Ozu's style, a filmmaker does not necessarily have to choose exclusively between using the cinematographic image for the construction of narrative place or for diegetic space. In fact, Heath considers it a mark of modernity that Ozu has achieved an independence of the space and the place, while sacrificing neither. Ozu tends not to respect conventions of continuity editing such as the 180 degree rule. The virgin viewer of an Ozu film struggles to bind the space and the place, but it is only logical at the points where Ozu deliberately does so. The veteran viewer of an Ozu film knows to stick to the narrative place, and treat the diegetic space as potential fun to explore. The virgin player of a GoldenEye level behaves chaotically within each room, probing its space for its narrative place. The veteran GoldenEye player saves his chaotic behavior for the narrative space beyond the individual room; within the room, he knows its purpose, he enacts it mindlessly.

Thus playing a match in a GoldenEye level becomes in great part a beast of chance: at the beginning of the level, each player is placed randomly at one of the "spawn points" (of which there are also a very limited number, say six per level). Depending on the proximity of the spawn point one happens to appear at to items of strategic value such as body armor, the best weapon, or access to a strategic position, the player may begin with a strong advantage over his opponents. Other than this chance factor, in a game played by all experienced players, the game can become borderline deterministic. Each player edits for himself little more than a "proto-montage" of what I have described as "mini-sequences": each mini-sequence in-and-of-itself familiar, and the only difference is the order and combination that the player chooses to take them. Of course, eventually the deterministic flow is thwarted by encounters with opponents, and may end in death, victory, or mutual disengagement, the latter two of which result in a return to the independent montage project.

For example: The match begins in the level Basement. I spawn in The Ventilation Duct. I have two directions I can go. Having played this level hundreds of times before, I know that one direction leads to "The Room Where I Will be Quite Vulnerable and To Which Only Idiots Go," while the other leads to what I know as "The Body Armor Room," of course, for the fact that it contains the level's source of body armor (which periodically regenerates after being taken by any player). I choose the direction that will take me to The Body Armor Room. Once this decision has been made, my thoughts on the matter need not constantly occupy themselves with the fact; I am free to contemplate other things as my avatar races through The Ventilation Duct "corridor" in more or less the exact same way I have seen it do hundreds of times before. I arrive in The Body Armor Room and make a beeline for the body armor. Now, once again, the interactivity foregrounds itself, and I take my place as a montagist of mini-sequences: there are two other entrances to The Body Armor Room, and I choose to take either of them or retrace my steps through The Ventilation Duct. Whatever I choose, each of these corridors in turn leads to another set of divergences. Thus, my "run" in one of these matches can be mapped as a path through a search tree of possibilities, to borrow a term from computer science. I may manage to kill an opponent or two in moments of interruption of my sequence. And eventually my all-too-inevitable and all-too-soon death comes, and I spawn once more at another spawn point, birthing another node in an unfurling tree of possible mini-sequences that the next run will consist of.

These individual runs, in turn, accumulate in my field of consciousness, as both visual and causal information. Even as I continue to hunt and be hunted through this level, I contemplate my own recurring behaviors, either failures or successes, accruing in patterns, relative to the spatial structure of the level. If I didn't already know The Room in Which I Will Be Quite Vulnerable and in Which Only Idiots Go as such, I will eventually learn this fact in that process of "natural selection." With each of my runs fundamentally resembling each other, I in effect play the role of a film editor splicing segments of film together. While the editor watches his test attempts at creating the scene through the building blocks of master shots and glance cues and punch-ins, in an attempt at creating a pace and coherency he desires, making slight adjustments to each version until he gets what he wants, I vary my behavior during the course of my runs, experimenting with splicing "mini-segments" together in new orders, choosing corridors I would not normally choose, and, using natural selection over an evolutionary period, determining which sequences give me the best "scene," that is, the most strategic run. The time between encounters in a match of GoldenEye obtains a predictable, narrative form, and eventually the actual process of getting from one place to another in the game becomes an unconscious function of the gamer, who can then focus on other things.

For one for whom the progression and dynamic splicing of mini-sequences in a given level is such second nature that he needs only "check in" with his own avatar periodically, such access to information provides immense advantage. Just as one, when defensively driving a motor vehicle, frequently checks his side-view and rear-view mirrors as well as his dash for speed and engine conditions, without fear for that instant of something drastically changing on the road directly ahead, the advanced GoldenEye multiplayer gamer spends at least as much time following his opponents' screens as his own. In this sense, the gamer disembodies himself from his avatar to degrees in order to "distribute" his embodiment across all avatars.

And here we have finally returned to Friedberg.

So far I have proposed two varieties of montage in GoldenEye: those within a single player's subscreen, and montage across those subscreens. I intend here to relate the two concepts directly.

When one is a new player to GoldenEye, the units of meaning he encounters are the individual rooms and items. He is confined to embodiment in his avatar, busy constructing via proto-montage the archetypical narrative of a single avatar's run.

When one is an advanced player in GoldenEye, he has overcome the primacy of such units, relocating them to his subconscious. He is now occupied with constructing the narrative of the entire match, by observing moments in his opponents' behavior in a proto-montage, and then, out of his knowledge of his archived set of archetypical narratives of single avatar runs, predicting their subsequent behavior. The GoldenEye master now approaches that role of the "omniscient narrator" in film – selecting the best angle on the action at any given time to compose for himself the most coherent narrative not of his own avatar, but of the match itself. His interactive bond with one of the avatars becomes an incidental element. Each of the four avatars on the split screen represents, in addition to a body that might be killed, a camera on the action. Like the security guard spectator of Timecode watching an array of security cameras, the GoldenEye master incorporates all these flows of information into his decision matrices.

Perhaps it would be more accurate to consider this master gamer less as a security guard and more as a spy, who has placed video taps on each avatar; this label contains the negative connotations oft associated with such practice. A practiced "screenwatcher" may become so much better at the game than his human opponents that the game ceases to be fun for them. In fact, there exists a large percentage of gamers who categorize screenwatching derisively and as just as offensive to the enjoyment of a match as other "cheap" strategies such as turtling (stubbornly refusing to leave a point of defensive advantage), sniping (hiding in a place where one is free to fire upon opponents without risk of return fire), and camping (spending time near spawn points in hopes of picking off players immediately as they spawn, terminating their run before they even have a chance to acquire a weapon or body armor). A common saying among gamers is, "Snipers get herpes, Campers get syphilis, and Screenwatchers get AIDS."

All of these criticized strategies share the aspect of compromising the utilitarian enjoyment of playing the game by betraying its understood design intent. It is difficult to design a game for which there are no ridiculously unfair strategic positions or items (known as "broken" points); debatably the most important phase of game development is testing, where experienced gamers are allowed to become master gamers at the beta version of the game, and point out potential risk points to the designers where it may be seductively easy for gamers to resort to these negative strategies. It is almost impossible, in the end, to produce a game which has no broken points, and it is up to the body of gamers to commit to a social contract at the outset of the match in which they agree not to take advantage of these elements which, though technically embedded in the game, are unintentional, and compromising to the maximum enjoyment of the designer's intentions.

When playing against a computer opponent in GoldenEye, the segment of the screen which would be used to display the point of view of its avatar is left blank, implying that visualization of an avatar's point of view is designed primarily as a point of action, not information gathering. Also, when playing similar multiplayer shooter games to GoldenEye via the internet, the opponent's point of view is rarely displayed. These facts suggest that the split-screen display of a multiplayer GoldenEye match is not an intended design, but an unfortunate necessity, given the limitations of the system. And this is not the only element of gameplay which is indicative that the design of the multiplayer system in GoldenEye is not realized to the designers' complete desires: when playing with three and four players, the number of calculations on visual polygons that the Nintendo 64's computer chip must run exceeds its rate of output to the television display, resulting in annoyingly slowed gameplay. A final piece of evidence that screenwatching is officially undesirable behavior is the presence of the radar – an imbedded screen which gives a primitive overhead display of the level which slides around underneath a central node representing the player's avatar, and undifferentiated yellow dots representing the locations of the opponents. Because the radar supplies some information but very little (not even contours of walls or indications of elevation), it implies that this is all the information the game designers intended for a player to have, for the sake of maximal strategic enjoyment by each player.

Regardless, screenwatching is a more difficult strategy to police and visually prove than camping, sniping, or turtling. A screenwatcher can hide his craft under the guise of simply being a better player. This amusing cover-up points to the fact that, interestingly, unlike a game full of snipers, a game full of campers, or a game full of turtlers, a game full of screenwatchers is still quite fun, intense, and strategically rigorous. The difference here is perhaps that the former three strategies reduce the effort required to perform well, while the screenwatching strategy requires ungodly and highly respectable amounts of effort, concentration, and quick decision-making.

Yet screenwatching is still considered taboo, independent of the strategic content it does not necessarily compromise. It seems that the fundamental element of gameplay compromised by screenwatching is not strategic, but substantive. One can see how this method of playing could defeat part of the essence of the fun of the game: the fantasy of suture into an avatar running around in an enclosed environment shooting at other individually sutured avatars. Friedberg provides an important bridge between discussion of the multiple frame and suture theory:

The shot-countershot can occur in the same master frame in a multiple-frame, multiple-screen format. Separate “points of view” – of seeing and being seen – can be combined, compared, placed simultaneous and adjacent. In terms of theories of suture, replacing the sequence of shot-countershot with this form of nonsequential simultaneity skews the sense that the spectator is somehow sutured into the film, between the shots. While the single-screen moving image offers multiple perspectives through the sequential shifts of montage and editing, the multiple-frame or multiple-screen moving image offers the same via adjacency and contiguity.

Yet, again, Friedberg's language leaves worrisome ambiguities. Though she suggests that simultaneity "skews" the effect of suture, implying a change in the effect, connotatively in a negative way, she follows up by saying that the multiple-framed moving image offers a same set of multiple perspectives. The language "same" implies the opposite of "skew," though she also seems to shift the topic. Suture seems to be less about the ultimate offering of multiple perspectives, regardless of simultaneity or sequentiality, as it does seem to be about our access through one specific perspective in each moment.

Heath explains that "point of view, that is, depends on an overlaying of first and third person modes." In his suture theory, he is referring to the sense in which we access the narrative place in each moment through identification with a character in his moment of action. Relatedly, Pasolini mentions a literary term known as "free indirect discourse," which is when the supposedly objective narration of a novel takes on the voice or limited knowledge of a certain character. In a sense, though, the novelist does not even have to go that far whenever a character is being described in a text. If only for a sentence, that character has "taken the reins" of the text. We still identify in the third person as the omniscient narrator, but also temporarily as the individual who has been "deified" as the owner of the current reality. For a unit, not particularly of time, but more of meaning, the character has become the will of the universe. And such a moment translated onto film is an object of such desire that the viewer sutures himself into the diegetic reality.

The key element in GoldenEye is the knowledge that even when no player is in sight of another player, the tension for potential meeting exists. When Timecode begins, the four spaces are disparate, none clearly being the same space or even proximate to one another, and our experience of the montage by simultaneity is purely intellectual. But as the film proceeds, a complex fabric of connections is woven as the spaces portrayed pass and intersect each other. As the spaces become intertwined, the narrative tension is elevated – this is the point where our field of consciousness begins to move from the Eisensteinian intellectual mode of seeking association to the spatial continuity mode. We seek to know who embodies the camera, because it is now no longer “alone”. The presence of a second "ghost eye" threatens to undo Metz's imaginary signifier. This is the essential moment in the gameplay of GoldenEye – when two avatars meet, they appear on each other's screens, fundamentally violating Metz's imaginary signifier: "…thus (in that it reduces the three dimensional environment to two dimensions, as well as contextualizes your conscious experience in a body which finds itself spatially among those objects) film is like the mirror. But it differs from the primordial mirror in one essential point… there is one thing and one thing only that is never reflected in it: the spectator's own body." Being confronted with the embodied self in the third person is a terrifically alienating feeling – in GoldenEye, the "transcendental" eye of one’s subscreen is undone by the image of the "empirical" incarnation of the eye in a body on another subscreen. And then the body is shot dead. Your subscreen is washed red with blood, and fades to black; with nowhere else to look, you watch your opponent's subscreen, see the aftermath of your own death, completing a common fundamental human fantasy.

To be sutured into a montage of simultaneity, as Friedberg suggests may be the consequence of the multiple frame, is to become sutured into the narrative of the match that the GoldenEye master uses strategically through his screenwatching. Were the screenwatcher to become enmeshed in the GoldenEye match as the godlike eye that perceives the best possible moments in each available camera, the screenwatcher would lose that intimate bond with the avatar which makes the game fun. He would feel nothing when his avatar dies, having already been watching it from the third person, through his opponent's screens' eyes.

But this is just flat-out not the case. One cannot be sutured into the third person alone. One becomes sutured into that third person by summation of the individual sutures with each first person. Every time the GoldenEye master cuts from one screen to another, he becomes temporarily embodied in that avatar, even if it happens to be shooting at the one he controls. We cannot read more than one sentence at once anymore than we can watch more than one screen at once. Rapidly scanning across screens as well as rapidly scanning through sentences will produce an effect of suspended simultaneity, but this is not true simultaneity. As one can accomplish viewing an Ozu film, the narrative place of the progression of the match does not need come into conflict with the moments of diegetic space accessed in rapid montage as the GoldenEye master cuts from subscreen to subscreen; they can both be suspended and exist simultaneously on the gamer's "field of consciousness": their mind's eye.

Heath explains, “Film works at a loss, the loss of the divisions, the discontinuities, the absences that structure it – as for example, the 'outside' of the frame, offscreen space… Such absence is the final tragedy of a Bazin, who wants to believe in cinema as a global consciousness of reality, an illimitation of picture frame and theater scene – 'The screen is not a frame like that of a picture, but a mask which allows us to see a part of the event only. When a person leaves the field of the camera, we recognize that he or she is out of the field of vision, though continuing to exist identically in another part of the scene which is hidden from us. The screen has no wings.'" If the suture in the montage of simultaneity truly sutures us into the third person alone, how do we account for the moments when no avatar is on any other avatar's screen? We are aware of the presence of the other avatars though direct embodiment in their first persons. The suture of the montage of simultaneity is conclusively one of rapid transfer from one first person to another.

---

[1] GoldenEye does not (can not) do this: in fact, during a multiplayer match, all sounds for all players are heard simultaneously, and at equal volumes. When one plays the game, this is somehow not bothersome; as Heath suggests, "The soundtrack is subservient to the image track," and it seems that a player bases his presence in the GoldenEye level diegesis on the visual rather than aural information, "finding" the appropriate accompanying sound from the cacophony of sounds for each visual occurrence on his own particular screen.

[2] A term from Bazin.

[Read Full Article]

[Collapse to Introduction]

Hybrid Cinema: Plugging Humanity into the Digital Infinite

by Darren Franich


PROLOGUE: 300 and the Digital Future

Midway through 300, with some of the best decapitations yet to come, Gerard Butler eats an apple. In a film that features enough radically kinetic activity captured in balletic slow motion – spears thrown, shields raised, cliffs ascended, elephants toppled – to fill at least 300 male-demographic-baiting movie previews, it seems almost ill manners to point out one of the quietest scenes in the film. Yet several critics have mentioned the “apple eating” scene, and director Zack Snyder himself admits that it is one of his two favorite parts of the movie (Weiland 1). The strange magic of the sequence lies in its casual flavor, for in a film where activity is perpetually etched against incredibly beautiful skylines – a film that begs you to admire the beauty of fake blood spraying from every limb in every direction – for just a moment, the camera admires an actor in the process of eating an apple. Onscreen mastication is one of the simple pleasures of the movies: think of Tim Holt stuffing strawberry shortcake down his fat mouth in “The Magnificent Ambersons,” or Augusté Lumiere and his wife feeding their baby at the dawn of cinema. Lisa Schwarzbaum once commented, “I love to watch people eating and drinking on screen: Unlike the simulation of movie coupling, real chewing and swallowing goes on – a very sexy sight” (Schwarzbaum 1). Schwarzbaum hits on the effectiveness of this sequence: it is quietly, casually real, whereas so much else in the movie is bombastically, gloriously, decadently fake.
As I write this, 300 is making summer-blockbuster money in mid-March, lacking any recognizable stars and with a relatively bargain-basement price tag of $60 million. It is only the third major studio release filmed on a digital backlot, with real-life actors acting in front of a computer-generated landscape, and it comprises both the best and worst predilections of the modern sphere of CGI cinema. It overdoses on its own visual grandeur, often creating a stilted environment of overcontrolled beauty, yet it also represents three positive movements in the modern aesthetic of special effects towards further integrating the digital and the physical into a cyborg form of hybrid cinema: installing a human element by combining real-life actors with effects, either through motion-capture or blue screens; using the effects to hide their own fakery or, alternately, heightening the unreality of the effects to the point of cartoonish defamiliarization, avoiding the Uncanny Valley effect by obscuration or overpresentation, respectively; and, lastly, expanding the scope of digital creations towards infinity by programming enough variables into computerized worlds and wholly digital actors to create the spontaneous reality of randomness, granting the ultimate human virtue of choice. All three of these methods are tied in closely to the technological and artistic expansion videogames in the last two decades, and so, by examining the current stage of videogame evolution, it is perhaps possible to make an educated guess about the future of the cinema.

PART 1: Uploading Humanity

The nomenclature of the “digital actor” phenomenon obscures the evolving role of motion-capture and assorted other effects geared towards incorporating actual human actors into creation of computer-animated-characters. Final Fantasy: The Spirits Within was supposed to be the watershed moment for computer-created characters, with the filmmakers buzzing over the incredibly terabyte density that went into creating the main character’s flowing hair; but the film was a critical and commercial bust, a stilted blockbuster that presented the awkward spectacle of Alec Baldwin’s voice coming out of what looked like a digital wax model of Ben Affleck. The real breakthrough came two years later with the introduction of Gollum in The Two Towers. Onscreen, Gollum is entirely digital, but his movements, his voice, and the actors’ response to his physical presence all derive from the performance of Andy Serkis, an actor who wore a motion-capture suit. The tactility of the Gollum character, and his physical interaction with the human actors, is miles removed from the digital-human interactions of a film like The Phantom Menace, which offers the discerning viewer the pleasant profilmic image of gifted actors like Liam Neeson and Ewan McGregor staring blankly into empty greenscreen space which animators will fill in post-production. Serkis’ supporting role in Peter Jackson’s Lord of the Rings trilogy, and his ensuing starring role in Jackson’s King Kong, places him at the fore of a burgeoning generation of actors who star in movies without ever actually literally appearing onscreen: Alan Tudyk was the motion-capture model for the titular machines in I, Robot, and Doug Jones has cultivated a working relationship with Guillermo Del Toro in Pan’s Labyrinth and Hellboy. The latter film is particularly noteworthy as an important entry in the current wave of comic book films. In the wake of the poor reception of 2003’s Hulk, which presented a wholly digitalized leading man, similar characters like the Thing in Fantastic Four and Hellboy himself were hybrid creations of computer animation and actors under heavy make-up. It would be wrong to describe either character as entirely “real,” since in action sequences their persona is largely computer-created, but the characters’ non-kinetic gestures – what used to be called acting – belongs entirely to Michael Chiklis and Ron Perlman, respectively. The notion of plugging real people into digital systems has been present for years, from the rotoscoping movements in Prince of Persia to the vogue for motion capturing celebrity athletes in video games, right down to “the way the upper lip slides up over the teeth” when the digital Tiger Woods smiles in his golf franchise (Seabrook 3).
Even as modern filmmakers work towards a hybrid form of actor and computer towards the creation of film characters, they have also shifted the presentational form of atmospheric and kinetic non-human special effects to incorporate true-life bodies into the frame, in a manner that is strikingly reminiscent of the wave of three-dimensional third-person action-adventure games. One of the most striking sequences in “The Two Towers” exemplifies the manner in which the subtle layering physical reality and digital fantasy is fast becoming the norm in Hollywood. The camera tracks in front of Saruman as he walks through an interior set of his tower, then moves behind him as he steps outside onto a balcony that overlooks an army comprised of thousands of digital-Uruk Hai. The camera moves in a woozy movement behind Saruman’s head, admiring the hordes, but at no point does Saruman himself leave the frame. Whereas the limitations of primitive special effects generally engendered a style of crosscutting, following the eyeline of the human actors onto a scale model or some other pre-digital effect, modern digital effects are inserted seamlessly, if not always subtly, into the frame with human actors. As in this shot, the subjective focus of the camera remains on the physically real: Saruman occupies the center of the frame, in a third-person over-the-shoulder movement that is quite similar to the usual camera angle preferred by third-person adventure games. Peter Jackson’s Weta Digital effects company prefers layering digital effects with actors and physical scale models: the final shot in the sequence is a combination of the three styles, and Weta’s devotion to scale models represents its overall loyalty towards the bare scent of reality, compared to the completely digital worldscapes of the Star Wars prequel trilogy.
Consider a side-by-side comparison of two versions of War of the Worlds: the 1950s version exemplifies the cross-cutting style, with the camera following the actors’ frightened stares offscreen toward the models of the Martian spaceships; fifty years later, Steven Spielberg almost always keeps the Tom Cruise character in the foreground, following the stares of the audience towards the digital tripods in the distance. Advances in technology allow for many of the sequences to be filmed entirely in long unbroken shots; particularly noteworthy is the sequence in which the camera spins around and eventually through a car moving at high speed through a deserted freeway. The long deep-focus shot, an indicator of reality since before Bazin began pondering the cinema, has been co-opted by special effects cinema as a method for defamiliarizing the fakery of the digital by juxtaposing it with the human. Here again, the comparison to third-person games is appropriate, since they tend to favor a free-roaming 360-degree camera that moves gracefully in solid movements, rather than cutting. The most profound example of what could be termed third-person cinema is Children of Men, a film that shares several stylistic devices with Spielberg’s War of the Worlds. Both films follow the perspective of a single main character through a massively redefined worldscape: earth invaded in War, a post-child dystopia in Children of Men. Both offer no explanation of events outside of the ken of that main character, excepting the Morgan Freeman voiceover which bookends War. Both films favor long traveling shots that follow the main character through desolation. The difference, ultimately, is that War of the Worlds must feature gigantic alien Tripods; Children of Men’s effects are far subtler.

PART 2 – Hiding the Magic

Alfonso Cuaron’s aesthetic in Children of Men represent an important hallmark in modern effects: the use of digital effects to reflexively hide themselves, either entirely or through anachronistic imperfections that paradoxically make the film more “realistic” by acknowledging a realistic, rather than digital, unreality. The latter requires explanation. Zoic Studios, one of several small special effects companies gaining clout in a field which was once almost entirely dominated by George Lucas’s Industrial Light and Magic, have created a particularly intriguing style of digital shot in a pair of science-fiction television shows, Firefly and Battlestar Galactica. Both shows involve shots created entirely within a computer of one or multiple star ships moving through space. Just as John Dykstra’s work on the original Star Wars introduced intriguing avenues of movement by creating panning movements across star fields, Zoic’s work on these shows pushes towards what was once considered a documentary style: the shaky camera movements and seemingly haphazard zooms create the sense of an actual camera where none ever actually existed. In one particularly noteworthy shot, a piece of space debris actually “hit” the camera: by creating a sense of particularly filmic unreality, the creators of the effect add a layer of subconscious believability to the admittedly fantastic images onscreen. Video games were there years ago: in the first 3-D platform game, Super Mario 64, staring directly into an in-game mirror reveals the camera (held by a turtle floating on a cloud, which explains the extraterrestrial nature of the swooping 360 degree camera); in the Metal Gear Solid series, emerging from one of the swimming levels leaves droplets of water falling down the camera; more recently, the third-person action game Gears of War innovated what is generally referred to as the “roadie run,” when the avatar crouches into a run and the perspective moves close behind him in a shaky style reminiscent of the handheld intensity of Saving Private Ryan. The aesthetic of Gears of War bears a striking resemblance to an important sequence towards the end of Children of Men, when the camera follows behind Clive Owen through an exterior warzone into three floors of a besieged building. The shot involves several digital explosions and is actually four separate shots pasted together, but the overall effect is a seamless tracking shot operating on multiple planes of action. Most digital effects present the impossible; Children of Men uses the digital as a method for re-examining reality.
On the other end of the spectrum lies the expanding vogue in over-the-top special effects environments, digital fantasias that call attention to their overt unreality by pushing the visual aesthetic towards cartoonish proportions. The “digital backlot” as a genre is thus far represented by three major studio releases, Sky Captain and the World of Tomorrow, Sin City, and 300; one could argue that the Star Wars prequels also deserve a spot on the list, but whereas they set out to create a believable, if fantastic, reality, the aforementioned trinity are set in self-conscious recreations of cinematic genre myth. Sky Captain is the Flying Ace serial fed as a retro-futurist action dystopia, a sort of post-modern Flash Gordon; Sin City is noir on German Expressionist hallucinogens; 300 is the Greek epic stripped to the bare essentials of sword, sandal, and blood by the gallon. The visual scheme of each film steps backward toward monochrome: both Sky Captain and Sin City are essentially black-and-white, while 300 is bleached blood red. Whereas much of the evolution of special effects in the 1990s was based on the progression toward realism, these films represent a sideways leap into hyperreality, or even anti-reality, and it is tempting to read the contrast between their forthright artifice and Children of Men’s manufactured realism as a millennial answer to the image/reality rift Bazin traced back to the Lumiere outdoor actualities and Melies’ cinemagic spectacles. What is certain is that the schism represents the tremendous diversity at play in the field of special effects: the drive toward realism replaced by a multi-faceted field of possibility.
This ties in closely to the current wave of videogame console wars. Whereas in the past the evolution of home gaming entertainment systems was based closely on the advance of visual technology – the nomenclature of the Nintendo 64 was even derived from the number of bits in its processing unit – the diversionary tactics of Sony’s Playstation 3 and Nintendo’s Wii represent the new multiplicity of possibility in the visual and technical fields of gaming. The PS3 advertises “better graphics and better sound than the Xbox 360” and backward compatibility with earlier Sony games, along with a next-generation DVD system and an internet connection that moves the system beyond the realm of pure gaming into home entertainment (Thurott 1). The Wii goes in the opposite direction, with less impressive graphics technology but the first major controller innovation since the dawn of the home console revolution: a motion-controlled Wii Remote (Thurott 1). The diversity in visual and technical effects in the videogame market represents a new diversity in gaming demographics, just as the expanding universe of special effects represents the expansion of visual aesthetics in digital cinema.
However, both forms of masking digital effects, either through subliminal or superluminal methods, ultimately run the risk of being overcontrolled, and certainly all of the digital backlot films suffer from a stilted, even synthetic precision. There is a reason why the style has thus far only been utilized in overtly self-conscious genre exercises based on melodramatic narrative and emotional twists: the meticulously staged precision required for incorporating real humans into largely digital worlds simply does not give the actors much space in their performances, nor does it allow them the atmospherics of reaction, either to digitalized characters or to the world around them. Thus, even the best actors seem cut off from the art directed plasticity of the digital world around them. Mark Harris identifies 300 as an extreme but potent example of young directors’ tendency to utilize modern effects technology for perfectionist tendencies that border on monomania and deprive the filmmaking of a sense of spontaneity. He describe David Fincher’s films as particular examples of overproduced perfectionism, with “every bruise and dripping wound is rendered with exactitude, and each camera move is choreographed to the millimeter, but the films seem oxygen-deprived, hermetically sealed” (Harris 1). This brings up the final link in the current evolutionary chain of digital effects cinema: the creation of infinite variables within the processing engine that creates the special effects, and the secession of control toward a computerized facsimile of infinite randomness. Can a computer surprise its creators?

Part 3: Computers With Choice

Weta Digital’s Massive, a special effects program, represents an important step forward in the creation of randomized computer individuals, or agents, who operate within a series of variables which, when theoretically stretched to infinity, create the possibility of a completed world within the movie screen. For the massive crowd scenes which swell throughout the trilogy, steadily building from the darkened Mines of Moria through stormy Helm’s Deep to the wide open sunny expanse of the fields of Pelennor, Peter Jackson contracted Stephen Regelous to create a system that would allow for the creation of true individual agents, rather than the same orc-avatar recopied several thousand times. The sheer scale of the battle scenes made individual animation impossible, so Regelous programmed each agent to make “subtle respondings to its surroundings with fuzzy logic… it’s not crowd control but anarchy” (Macavinta 1). “I wanted to take the processes of nature and apply them to generate computer imagery,” Regelous explains, and the method involved was so successful that the animator admits, “I can’t tell what’s Massive and what’s not anymore” (Macavinta 1). It is important to note that the movements of the agents were created from stunt men, and that practically every shot produced using Massive incorporates some element of physical reality, either with New Zealand exteriors, scale models, or real actors in the foreground. And of course, it is years, perhaps even decades, too early to declare the agents as true computerized individuals. Yet a look towards the future of computer effects, in the form of the concept of Procedural Generation, offers a tantalizing glimpse of a post-Massive future.
The algorithmical process of Procedural Content Generation represents a tentative step towards the creation of a genuine infinity inside the box. As the computerized environments and worlds within video games have expanded both in size and visual intensity, budgets have skyrocketed, as more and more designers are needed to perfect even the tiniest details: every building in the sandbox urbania of a Grand Theft Auto has been individually designed, with a corresponding build-up in the overall computational size of the gamespace. The system of Procedural Generation, in which basic algorithms are used to generate content during gameplay, seeks to change that. Elite, an early space trading game, utilized a rudimentary form of the system, but procedural generation has lately come to the fore of the gaming world because of Will Wright, creator of The Sims, whose upcoming Spore “seeks to replicate algorithmically the conditions by which evolution works, and render the process as a game” (Seabrook 1). The game begins at the molecular level and allows each individual game player to create a species and carry it through the stages of evolution: primitivism, civilization, modernity, and even space travel. Henry Jenkins describes how cinematic storytelling “has become the art of world building, as artists create compelling environments that cannot be fully explored or exhausted within a single work or even a single medium” (Jenkins 115). Wright is pushing the narrative boundary into the fourth dimension: not merely giving his players the tools to build a world, but to observe the entire infinite span of its existence.

Conclusion: The Effect on Story
Spore indicates the infinite possibility within the realm of digital effects: the ability to capture the evolutionary state of an entire universe. For a cinematic society already besotted with virtual worlds, this indicates two very likely possibilities: the onscreen digital environments will become more convincing, and they will arrive at an ever faster pace. A generation of filmmakers raised on trilogies wants to create their own sagas: the millennial accumulation of Pirates, Matrices, and Spider-Men will soon be joined by more Sin Cities, and Frank Miller is supposedly considering a hopefully-unnumerical sequel to 300. Yet if Spore points to the future of digital effects cinema in its sheer infinite extremity, it seems to tell the modern viewer very little about the evolution of narrative. There is no story to Spore, really, except for eternity. Yet, in this sense, Spore harkens back to the god’s-eye-view of evolution in 2001, albeit by shifting the perspective from the hapless creatures evolving to the invisible godhand who pushes them forward. Indeed, despite the immensity of activity which Spore encompasses, the game appears constructed as almost meditative: contemplative of the infinite rather than kinetic, in the manner of most special effects since the Star Destroyer roared over the camera in Star Wars. Indeed, the raw gigantism of Spore’s visual gamescape recalls the earlier mode of effects created by Douglas Trumbull, who once espoused the virtue of “creating some crazy illusion that looks so great that you can really hang on it like a big master shot of an epic landscape” (Bukatman 94). Many of the effects of 300 and Lord of the Rings are energetic, yet just as many focus more on the juxtaposition of the human form against an epic grandeur that could only be created: consider the sweeping helicopter shot in Return of the King that spins around mountains through clouds, watching gigantic immortal torches lit across the wide expanse of Middle Earth. As digital effects become more impressive and the manufactured worlds more real, the importance of genuine humanity only becomes magnified
Thus, the future of cinema may lie in a videogame that occupies the exact opposite end of the cosmic spectrum from Spore. Shadow of the Colossus is a third-person adventure game set on a gigantic peninsula in some vaguely medieval fantasy world. It strips the story elements of the genre to its bare essentials: your avatar has a sword, an arrow, and a horse, and he must kill 16 monstrous Colossi to resurrect his beloved. The large majority of the game, both in space and time, is given over to emptiness: the player must find the Colossi hiding in the wide, varied climate of the peninsula, the large majority of which is incredibly beautiful space that serves no real “point,” in the classical sense of the goal-oriented aesthetic of video games. To add to the confusion, Shadow is constructed in a uniquely ambiguous moral gray zone: you lack any context of understanding where your character came from; the Colossi rarely attack you, and mournful music plays as they die; most tellingly, and subtly, your avatar actually degrades in appearance with each killed Colossi, turning pale and sickly. Yet because the rest of the game is empty of activity, the only other option is to simply be, rather than play. Because Shadow is a Japanese produced work of visual narrative art, it recalls what Scott McCloud describes as an eastern comic book “tradition of cyclical and labyrinthine works of art… they so often emphasize being there over getting there” (McCloud 81). Very little is explained in Shadow, and it is left to the player/viewer to explore the atmospherics of the game to glean the deeper implications of the story; even then, much of the world remains purposefully mysterious. In this, Shadow strongly recalls the structure of Children of Men, following a single character through a fantasy landscape rooted in a realist aesthetic – the former pastoral, the latter urban. Yet, in its elliptical structure and its fascination with the lonely human etched against the wide natural world (in digital form) I would argue Shadow represents no film so much as Terrence Malick’s The New World, which overdoses on natural beauty the way younger directors tend to with digital fantasias. Can it be that Malick’s plotless wandering might resemble the narrative structure of future digital cinema? Can there be a special effects art cinema, a digital backlot Antonioni? Malick’s mysterious project variously known as Tree of Life or Q, a metaphorical story of the evolution of planet earth, sounds suspiciously similar to Spore. What is clear is that special effects are simultaneously expanding inwards and outwards, both in creative method and in aesthetic. It is obvious that the digital world will become more epically expansive; yet, as Shadow shows, it will also become introverted, even contemplative. Digital effects have long explored the lofty grandeur of fantasy. Both Spore and Shadow of the Colossus point to the a different sort of future: digital effects that restore a quiet grandeur to reality.




Works Cited

Bukatman, Scott. Matters of Gravity. Duke University Press: London, 2003
Gillis, Joe. “Waiting for Godot.”
Harris, Mark. “Micro Mangling.” Mar 15, 2007. http://www.ew.com/ew/article/0,,20015210,00.html
Jenkins, Henry. Convergence Culture. New York University Press: New York, 2006.
Kosak, Dave. “Will Wright Presents Spore… and a New Way to Think About Games.” March 14, 2005. http://www.gamespy.com/articles/595/595975p1.html
Macavinta, Courtney. “Digital Actors in Rings can Think.” Dec 13, 2002. http://www.wired.com/news/digiwood/0,1412,56778,00.htm
Macri, Dean and Pallister, Kim. “Procedural 3D Content Generation.” http://www.intel.com/cd/ids/developer/asmo-na/eng/20247.htm
McCloud, Scott. Understanding Comics. 1993.
Seabrook, John. “Game Master.” November 8, 2006. http://www.newyorker.com/archive/2006/11/06/061106fa_fact
Schwarzbaum, Lisa. “Scenery Chewers.” Nov 04, 2005. http://www.ew.com/ew/article/0,,1123339,00.html
Thurott, Paul. “Choosing a Video Game System.” March 20, 2007. http://www.connectedhomemag.com/HomeTheater/Articles/Index.cfm?ArticleID=95514
Weiland, Jonah. “The Reel King: Director Zack Snyder Talks 300.” Feb 14, 2007. http://www.comicbookresources.com/news/newsitem.cgi?id=9666

[Read Full Article]

[Collapse to Introduction]

Super Mario Movie: The Work of Art in the Age of Aura-less Images, Access to Tools, and Digital Manipulation

by Stephen Hirsch


The future fabricating machine in performance will invent images as patterned after cliché vision as those of the camera, and its results will suffer a similar claim to “realism,” IBM being no more God nor even a “Thinking machine” than the camera eye all-seeing or capable of creative selectivity…Yet increased human intervention and control renders any process more capable of a balance between sub-and-objective expression, and between those two concepts, somewhere, soul…

- Stan Brakhage, Metaphors on Vision

I think when we play Mr. Italiano, it’s training our minds for something.

- Paper Rad, "Welcome to M B"

Access to computers – and anything which might teach you something about the way the world works – should be unlimited and total. Always yield to the Hands-On Imperative!...All information should be free…Mistrust Authority, Promote Decentralization…You can create art and beauty on a computer…Computers can change your life for the better.

- Steven Levy’s “hacker ethics”

In 2005, computer artist Cory Arcangel collaborated with Paper Rad – a three-person art collective specializing in comics, animation, music, and installations – on a project entitled Super Mario Movie. They created this Mario Movie by hacking a standard Super Mario Bros. (1985) Nintendo Entertainment System videogame. The Movie, which tells the story of a decaying game world in which Mario is suffering a kind of existential crisis, runs off of the hacked cartridge played on a standard Nintendo Entertainment System. This project suggests a new relationship between the spectator and the work of art in the digital age, a relationship in which the spectator actually takes control of the work and utilizes it for something new. In many ways, Walter Benjamin provides this transformation’s theoretical background in his classic 1935 essay on technologies of mechanical reproduction. Benjamin’s relevance to the computer technology involved in the Mario Movie is complicated by the essential differences between the technology of Benjamin’s historical period and the technology of the digital age. While a common vision of technological utopianism emerges, the Mario Movie illustrates the way in which digital technology offers revolutionary possibilities beyond the limitations and reservations evident in Benjamin’s essay. Ultimately, a continuum emerges in which Benjamin provides the foundational description of a process of technological transformation of the world (primarily through decay of the aura) and the Mario Movie – within the context of digital utopianism – begins to outline some form of that process’s fulfillment. The appropriation and reorientation of a mass-produced work or object that characterizes the Mario Movie ultimately asserts the transformative, revolutionary possibilities of the innovative user.

In his essay “The Work of Art in the Age of Mechanical Reproduction,” Walter Benjamin writes that because of the increasing extension of the printing press an increasing number of readers became writers:

And today there is hardly a gainfully employed European who could not, in principle, find an opportunity to publish somewhere or other…Thus, the distinction between author and public is about to lose its basic character. The difference becomes merely functional; it may vary from case to case. At any moment the reader is ready to turn into a writer.

He then states: “All this can easily be applied to the film, where transitions that in literature took centuries have come about in a decade.”[1] Benjamin here argues that an increasing availability or accessibility of media technologies as tools, as well as an increasing availability or accessibility of the knowledge required to use those tools, leads to a situation in which the only difference between a reader and a writer is which function the individual person – potentially both reader and writer – happens to be fulfilling at the moment. With the increasing accessibility of reproduction technologies, every reader is always potentially a writer, every spectator is potentially an actor.

This observation of Benjamin’s certainly seems prophetically accurate with regard to media technologies like blogs and video cameras and YouTube, which facilitate to an extreme degree the widespread availability of the active roles of author or filmmaker. The passive spectator can easily become the active creator.

But in my view Cory Arcangel and Paper Rad’s Super Mario Movie presents something different in relation to Benjamin’s observations. While it does illustrate the shift in which the passive, receptive spectator takes on an active and creative role, it does more than that. It illustrates a profound transformation in the spectator’s relationship to the work of art (art defined broadly here to include the Super Mario Brothers video game). In Benjamin’s description above, the nature of being a spectator does not really change. What does change is the availability or accessibility of the creator role. The reader becomes a writer when he publishes an article of his own; the film spectator becomes an actor when he appears as an extra or playing himself in a film. But when reading a novel or the articles of others, the writer again becomes a reader. Basically, the spectator can sometimes be a creator and the creator can sometimes be a spectator; a certain mobility between roles is created by the situation described by Benjamin, but the roles themselves are not necessarily altered.

The Super Mario Movie presents something different, a further and more profound transformation in the relationship between a work of art and the individual who encounters it. The spectator becomes a critical and innovative user. The difference here is essentially this: the spectator who becomes a creator sees a film and says something like, “I can do that too,” whereas the spectator who becomes a user-innovator sees a film (or in this case a video game) and says, “I can do something else with this.”

I should note that I don’t see this as any kind of refutation or finding fault with Benjamin’s argument, but instead as an elaboration. Obviously there is an overlap between the spectator who becomes a creator and the spectator who becomes a critical user-innovator, for example Godard and the French New Wave critics-turned-filmmakers who use all kinds of classical Hollywood motifs in innovative, critical ways. I am simply exploring one direction in which the situation described by Benjamin can develop.

Benjamin’s concept of the aura and its disappearance through mechanical reproduction suggests one way we might go about theorizing the transformation of the spectator into a user-innovator with respect to the Super Mario Movie. In particular, the spatial metaphor Benjamin uses to explain the aura of a work of art provides a foundation for identifying how a viewer begins to understand a work as something that he or she can use and change as desired. Benjamin’s use of the term “aura” essentially refers to a work of art’s unique existence, its authenticity, its status as a singular object with a fixed place in historical tradition as well as a history of its own. Mechanical reproduction destroys this singularity and history. As Benjamin puts it: “…the technique of reproduction detaches the reproduced object from the domain of tradition. By making many reproductions it substitutes a plurality of copies for a unique existence.”[2] By thus eroding an object’s uniqueness and tradition, mass-reproduction deprives the work of much of its distance from and power over the spectator.

Benjamin’s use of the spatial metaphor to explain the disappearance of the aura proves particularly insightful with regard to explaining the transformed relationship with a work of art that makes the Super Mario Movie possible. Benjamin writes:

We define the aura as the unique appearance of a distance, however close it may be…This image makes it easy to comprehend the social bases of the contemporary decay of the aura…Namely, the desire of contemporary masses to bring things ‘closer’ spatially and humanly…[3]

Due to its uniqueness, its origin and existence in its own particular time and place, the object with an aura stands at a distance from the spectator. It is a foreign, autonomous object, occupying a position of power and authority in its relationship with any particular spectator. Those who want to view the object with an aura must go to it, because it will not come to them. One cannot adjust such an object to suit one’s demands, but instead one must adjust oneself to the object’s unique existence as it is. One can approach such a work but not touch it; one may observe it but not change it. The destruction of the aura through mass-reproduction of the object destroys this distance between the spectator and the work. This closeness with the object, this tactile intimacy, characterizes the relationship of the innovative user with the work of art.

With the Super Mario Movie the distance between the viewer and the work has been so completely annihilated that the viewer becomes a user with the power to reach right into the guts of the work – what it’s made of, it’s material elements. To create the Super Mario Movie, Cory Arcangel – who did all of the actual programming work on this collaboration – took apart a Super Mario Bros. game cartridge and altered its insides. Briefly, this entails opening up the cartridge, removing the program chip from the circuit board (while leaving the original graphics chip intact), soldering in a socket for a new chip, placing the new chip into the socket, and then screwing the cartridge back together. The new chip itself is programmed on a normal personal computer. Arcangel wrote all of the code for the Movie on his computer and used a chip-burning USB device called an EEPROM Burner to transfer the code onto a blank Nintendo cartridge chip.[4] This whole process is characterized by Arcangel’s tactile intimacy, his closeness, with the Mario game. At one point in his essay, Benjamin compares the cameraman to a surgeon, who diminishes the distance between himself and the patient to such a degree that he penetrates the patient’s body and works with his internal parts and organs.[5] Arcangel hacking the Mario game cartridge offers an even more striking correspondence to the surgeon’s invasive proximity and internal alteration of the patient. The distance created by the aura is non-existent here as Arcangel actually gets inside of the work and changes it. The relationship between object and spectator governed by the aura is inverted. The work itself is subordinated to the demands of the spectator, who now becomes a user with the ability not only to put hands on the work, but in it as well. In this way Arcangel and Paper Rad rework the existing work, changing and manipulating it to fulfill their own goals and demands.

The fact that Benjamin’s analogy with surgery seems more apt in relation to the digital work of hacking the game cartridge than it does in relation to the film work of cinematography points towards the question of why Benjamin’s theory appears so relevant to computer artwork such as the Mario Movie. Benjamin writes his essay in 1935, responding to technologies of mechanical mass-reproduction – at the pinnacle of which stands cinematic technology – and their effects on the realm of art. As detailed above, the primary effect Benjamin identifies is the decay of the aura possessed by the traditional work of art. With the Mario Movie we are dealing with a different kind of technology. Digital images found in video games are not mass-reproductions of some original image or work, as is the case with prints of a painting, recordings of a musical performance, or images on a film strip of a profilmic event. Instead, digital images are mass-produced. Instead of capturing and propagating some original, the digital image of an NES video game is painted or drawn on a television directly from the code embedded in its mass-produced, factory-soldered microchips.

Thus, whereas Benjamin writes in a historical period in which he can perceive the decay of the aura through technologies of reproduction, in the computer age of mass-produced images no aura exists to begin with. Benjamin witnesses the decay of the aura, not its annihilation. Technologies of mechanical reproduction copy, preserve, and disseminate some original image or work, thus allowing some vestige of the original’s aura to remain in their content. Furthermore, analog reproduction technologies – the only kind available in Benjamin’s day – are plagued by a noticeable degree of generation loss, and thus a well-preserved original print of a film, for instance, possesses qualities such as superior clarity and fidelity that can lend it an aura of its own. The digital image is not subject to these effects that maintain or create an aura; it does not reproduce anything original that had an aura to begin with, and all copies should be exactly the same. In fact, by referring to games as “copies,” one implies that each individual cartridge is an exact copy of every other cartridge rather than a copy of some original. Thus, with regard to digital images, the aura has not decayed, but instead has been completely annihilated. In this way it becomes evident that much of Benjamin’s essay might become even more relevant with regard to this technology of an entirely different nature. After all, the consequences implied by the disappearance of the aura could only fully manifest themselves in an age when the aura has truly disappeared.

The relevance of Benjamin’s essay to the Super Mario Movie becomes even less surprising, and more clearly defined, when one places the two within a broader context of utopian responses to technological developments. As described above, Benjamin views the extension of technologies of mechanical reproduction and their aura-destroying effects as progressive, full of revolutionary, utopian possibility. For one thing, such technologies allow the masses more and more to adopt creative roles, becoming authors and actors instead of mere passive spectators. Furthermore, the decay of the aura jeopardizes the authority of the work. This has numerous liberating effects: the spectator can adopt a critical position in relation to the object or work, a criticism denied by the cult value associated with the aura[6]; the spectator can bring the work close “spatially and humanly,” entering into a more hands-on, personal relationship with the work[7]; the spectator can perceive the work in a state of distraction rather than attentive, receptive contemplation, absorbing the work rather than being absorbed by it, receiving it through a filter of more pressing, personal concerns.[8] It is all of these concepts and effects that Benjamin refers to in his preface as full of utopian promise: “…they are completely useless for the purposes of Fascism. They are, on the other hand, useful for the formulation of revolutionary demands in the politics of art.”[9] This revolutionary irreverence for the aura-less work, this close, distracted, critical relationship with it, certainly underlies the Super Mario Movie and its treatment of the original game.

Importantly, the Mario Movie takes Benjamin’s utopian vision of the possibilities offered by technology and its relation to art farther than Benjamin does. Benjamin’s view is marked by some significant reservations. In his epilogue he writes: “Fascism sees its salvation in giving these masses not their right, but instead a chance to express themselves. The masses have a right to change property relations; Fascism seeks to give them an expression while preserving property.”[10] This reservation is an extension of my comments above on Benjamin’s observation that every reader can at any time become a writer, every working-class spectator can become an actor. I wrote that this does not significantly transform the structure of the individual human being’s relationship to the work of art, since although one may freely switch between the roles of spectator and creator, these roles themselves remain unchanged. The structure or system remains the same. Benjamin qualifies the utopian aspect of his essay with the same observation: the availability of the creative role, of “expression,” is nothing without a more fundamental change in the system of property relations governing objects and works. Even the Soviet practice of using regular, non-actor workers to portray themselves in films[11] becomes a false revolutionary act, an illusion of empowerment and participation, when not accompanied by a shift in property relations all along the line. In other words, the true utopian shift would allow for widespread appropriation and use of the work, including the knowledge and tools behind it, by the masses. It would be something more like Arcangel and Paper Rad hacking Mario Bros, utilizing it and the tools that made it to fulfill their own objectives. The limit that Benjamin’s hopeful vision of technology ultimately runs up against is that of accessibility. The masses simply don’t have the tools required to hack the film.

The Super Mario Movie goes beyond this historical limitation Benjamin recognizes in his vision of technology, fitting into the context of the later technological utopianism of the digital age. In his book From Counterculture to Cyberculture, Fred Turner details the emergence of this digital utopianism through a kind of dialectical synthesis that occurred primarily during the 1970s between the technological advances of military-industrial-academic research in the science of computers and the countercultural ideals of the “New Communalists” striving to pioneer alternative, utopian ways of living. Turner identifies Stewart Brand and his Whole Earth Catalog as the nexus for this synthesis: “With the Whole Earth Catalog, Brand offered a generation of computer engineers and programmers an alternative vision of technology as a tool for individual and collective transformation.”[12] A continuum of technological utopianism emerges here. Benjamin points toward technology’s ability to undermine and destroy established structures and hierarchies, whereas Brand points toward technology’s ability to be harnessed by individuals for personal fulfillment and the creation of new worlds in place of the undermined established structures.

In fact, the accessibility of technological tools – limited and limiting in Benjamin’s case, while underlying the Mario Movie – becomes the main focus of digital utopianism. Besides the title, date, and price, the cover of the first Whole Earth Catalog simply shows the earth viewed from space and, above it, reads, “access to tools.” In a passage that underlines the significance of digital art such as the Mario Movie, Turner writes of the Whole Earth Catalog: “…readers could glimpse the possibility of an entirely new world system, one in which American industry supplied tools that could be appropriated for purposes of transformation. The tools would be deployed first by an elite and later by the whole population.”[13] Arcangel and Paper Rad, as digital artists, act as members of this “elite,” appropriating the technological tools and products of industry and deploying them in a personally fulfilling and transformative way. In the process Arcangel encourages the “whole population” to do likewise through his hobbyist approach, posting all of his code as well as detailed instructions on hacking game cartridges on his website. The Super Mario Movie is an artistic testament to “access to tools,” with all of its essential elements appropriated from industry: the images (the original factory-soldered graphics chip is left intact), the material base (the cartridge, the program chip), the exhibition mechanism (a standard Nintendo Entertainment System).[14] In this way, the Mario Movie stands as a representation and some form of fulfillment of a long history of technological utopianism. Arcangel and Paper Rad appropriate an aura-less digital work, hack and utilize it as a tool – a means to some kind of personal, artistic ends – with no regard for the game’s prescribed uses or programmed objectives.

This replacement of a game’s intended use and objectives with something new – an idiosyncratic use or vision – has an effect analogous to photogénie in cinema. Early French cinema critics of the late nineteen-teens such as Jean Epstein, Louis Delluc, and Louis Aragon saw a certain revelatory power in the cinematic image that they termed photogénie. This concept, which echoes Russian literary theorist Viktor Shklovsky’s idea of defamiliarization, refers to cinema’s ability to focus one’s attention on everyday objects or gestures – a typewriter, a banknote, a person lighting a cigarette – in such a way that they take on new aspects, new significance, a poetic beauty. Aragon writes:

Poets without being artists, children sometimes fix their attention on an object to the point where their concentration makes it grow larger, grow so much it completely occupies their visual field, assumes a mysterious aspect and loses all relation to its purpose…Likewise on the screen objects that were a few moments ago sticks of furniture or books of cloakroom tickets are transformed to the point where they take on menacing or enigmatic meanings.[15]

Photogénie involves taking an object rendered invisible or unnoticed by the habits of everyday life and viewing it with such magnified attention – with the new eyes of a not-yet habituated child – that its prescribed purpose falls away, replaced by some newfound significance. A machinima work such as the Mario Movie has the same effect on the objects and gestures of the game world. The Mario Bros. game is so goal-oriented, presenting the player with objective after objective, that incidental details of animation, character design, or background objects go unnoticed, taken for granted, rendered invisible by force of game-play habits. One is too busy making Mario jump onto a cloud to notice how he jumps or the details of the cloud. When an artist removes the game’s programmed objectives, it magnifies the attention paid to such incidental details. The beginning of the Mario Movie illustrates this. For the first few minutes of the movie Mario simply stands on top of a box with a question mark on it (the kind that, in the actual game, one is constantly bopping with Mario’s head to get coins), wondering whether he should jump off or not. He fidgets back and forth on the box, magnifying the abrupt way that Mario moves, as well as the eight-bit intricacies of his character design and that of the question-box. Another Arcangel project, Super Mario Clouds (2003), provides an even more extreme example of machinima photogénie. On this hacked cartridge, everything in the game is removed except for the clouds scrolling by. This magnification of attention on the clouds provides an utterly defamiliarized vision of the Mario Bros. game world, while at the same time completely subverting the game’s programmed objectives.

Undermining a game’s intended use and objectives in this way – replacing them with something personal and transformative, some artistic or poetic vision – certainly has a political dimension. To refer back to Benjamin, it is one concrete, localized way in which “the masses” can exercise the right to transform property relations. Arcangel and Paper Rad’s act of hacking the cartridge states that, despite the fact that Mario Bros. is a mass-produced object of the entertainment industry, it is not solely the property of Nintendo; this corporation does not have exclusive power in prescribing how one uses Mario. In our digital age, with its utopian ideal of unlimited access to tools, personal appropriation and utilization of a mass-produced object or image, or work of art, is absolutely possible. In this way, the Mario Movie serves as a digital version of the Situationist practice of detournement, which involves taking some existing work or object – originally the city itself – and utilizing it in a new, transformative way that subverts established meanings and uses. The Mario Movie’s disregard for and subversion of the uses and objectives of the original game refers back to Alex Galloway’s comment, cited in note two above, that with this project Arcangel and Paper Rad display their understanding that “all media is dead media.” They do not recognize the Mario game as having any kind of life, power, or autonomy of its own. It only comes alive only when they use it, when they make it come alive to fulfill their own idiosyncratic artistic desires. As Retort writes:

Commodities can embody human purposes, and are capable of inflecting and developing such purposes, only if they are constantly subject to reorientation – change of function, change of valuation, recall to their mere instrumentality – in a world of meanings vastly exceeding those that any things can conjure up.[16]

Arcangel and Paper Rad do precisely this with the Mario Movie, subjecting the mass-produced digital image object to a radical reorientation. Thus, for all of its under-the-radar quirkiness, its light-heartedness and its fun, the Mario Movie actually stands as a refreshingly reasonable artistic act in its assertion of the primacy of human concerns in a society flooded more and more by images and objects erroneously claiming to possess some kind of power or magic – some kind of aura. Of course one must note the locality and the small, hobbyist scale of this project – after all, they merely hacked a twenty-year-old videogame, showing it at some galleries and museums and distributing it for free on the internet. This small scale serves as a reminder of the amount of power that remains in its established place. If a project such as the Mario Movie were to become truly threatening on any large political (or financial) scale, it is this established power that has the option to either take legal action (the music industry v. peer-to-peer file sharing) or to take recuperative action (George Lucas’ control of the Star Wars fan-film culture). On the other hand, one also cannot forget the Mario Movie as a concrete example of a transformative artistic act facilitated by new, digital media – an act that asserts the possibilities, as well as the rights, of the innovative user.

---

[1] Walter Benjamin, “The Work of Art in the Age of Mechanical Reproduction,” Film Theory and Criticism eds. Leo Braudy and Marshall Cohen , 6^th edition (New York: Oxford University Press, 2004), 802-3.

[2] Benjamin, 794. It is worth noting here that this point of Benjamin’s offers one possible explanation for why these artists chose Super Mario Brothers (1985) as opposed to a more contemporary game. This explanation goes beyond mere nostalgia. If objects are detached from the domain of historical tradition by being mass-reproduced, then questions of an object’s contemporaneousness are beside the point. Thus, with respect to history (and the degrees of relevance it can imply), there is no significant difference between Mario and the latest Grand Theft Auto. The artists’ choice of Mario serves to emphasize this tradition-less coexistence of old and new objects. Alex Galloway offers a similar explanation in his essay on the Super Mario Movie: “Super Mario Bros might be nostalgia to you. But it's not to them. All media is dead media, that's what Paper Rad and Cory understand” (“Cory Arcangel (Beige) & Paper Rad’s The Mario Movie,” available at: http://www.deitch.com/projects/press_text.php?pressId=29).

[3] Benjamin, 795.

[4] All of this information on hacking Nintendo cartridges is available at Cory Arcangel’s website: http://www.beigerecords.com/cory/Things_I_Made_in_2003/.

[5] Benjamin, 804.

[6] Benjamin, 800.

[7] Benjamin, 795.

[8] Benjamin, 809.

[9] Benjamin, 792.

[10] Benjamin, 809-10.

[11] Benjamin, 803.

[12] Fred Turner, From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism (University of Chicago Press, 2006), 105.

[13] Turner, 97.

[14] It should be noted that while the example I have chosen appropriates an entertainment product, the appropriation of the tools of industry (power) by digital artists can have a much more explicitly political dimension. The Radical Software Group, with which Arcangel is affiliated, has a project called “Carnivore,” which takes the very same digital wiretapping/surveillance system used by the FBI and, by way of a number of “clients” designed by digital artists (including Arcangel), interprets the collected data as dynamic visual or aural works of art. More info – and the program itself – available at: http://r-s-g.org/carnivore/.

[15] Louis Aragorn, “On Décor,” French Film Theory and Criticism: A History/Anthology, 1907-1939, ed. Richard Abel (Princeton: Princeton University Press: 1988), 166.

[16] Retort, Afflicted Powers: Capital and Spectacle in a New Age of War (New York: Verso, 2006), 179.

[Read Full Article]

[Collapse to Introduction]