Concept artists and games designers William Chyr and Matt Nava are concerned with combining research and videogame design. Their creative activities are founded in theoretical research including scientific, experiential and academic enquiry. This is and installation called ‘Ondine, The Happiness Project, October 2011’. It is by William Chyr who researches the intersection of art and science. He has recently released a videogame called ‘Manifold Garden’. A puzzle game that subverts physical laws enabling the player to navigate space in an abstract and theoretical way.

Chyr, W., (2013), ‘On the intersection of art and science’. Blog page.
Available at: williamchyr.com/on-the-intersection-of-science-and-art/

Indie videogames are pioneers of exploring the connections between videogames & art, videogames & politics and videogames & philosophy. In the videogame ‘Ennuigi’, by Josh Millard, the player is Luigi in a post apocalyptic Mushroom Kingdom. Moving the character between ever changing screens smoking and wondering through an inner monologue of thoughts and regrets. This is an example of how videogames are inventively telling meaningful or political stories through devices such as self-referencing.

In researching the structure of videogame design, particularly game play, I have been looking at patterns. Anni Albers’ Notebook from 1970 to 1980 is full of hand drawn pattern and plans for textile weaving. Visually the flatness of the drawings relate to flat planes of platform videogames.
The role of visual arts and the connection with videogame is a current debate in concept art and illustration. Questions of how videogame relates to the cannon of visual art history and context is under discussion by a number of academics, game developers and creative practitioners including the designers, authors and lecturers, Eric Zimmerman & Ian Boggost, curator Kristian Volsung, and Chicago based art gallery the VGA (Video game art gallery) which ‘seeks to increase cultural appreciation, education of video games and new media through exhibition, study (and) critique’.

“The tectonic is both the expression of the monumental reminders of the distant past, some, in the case of the cathedrals for example, still surrounded by the ancient rites of a hierarchical and dogmatic formal system, and at the the same time, challenge to individual creation on the threshold of the new reality to come (the cathedrals yet to be built).” (Zeidler. 2010 ).

If Bataille’s terms in Documents were Formless, Architecture, The Human Body, Animal and Low then, for comparison, the terms his collaborating editor Carl Einstein preferred were:

In an analysis titled Play of Concepts, Connor Joyce compares the differences he finds in Bataille and Einstein’s critical theories during their collaboration in Documents. Where Bataille was concerned with the irrational Einstein was concerned with productive revolt. (Zeidler. 2010). Or more specifically metamorphic revolt. It was in metamorphosis where Einstein found the tectonic. ‘Einstein sees the tectonic as capturing … multiple levels of reality in one formal expression’. (Quigley. 2007). Einstein illuminated the tectonic through his critical descriptions of Cubism. For Einstein Cubism represented the layering of gestalt images. Images that, although in stasis, were ultimately ungrounded, had no foundation. Later, during Documents Einstein critiqued the work of Masson and Klee, which allowed him to develop the tectonic. Masson and Klee’s work were diagrams of something ‘truly metamorphotic’ (Zeidler. 2010), where the image passes from nothing to something then to nothing again, where the viewer is passive, becomes active through recognition and then passive once more. This process of metamorphosis Einstein called hallucinatory. This, believes Joyce, is the dialectic of Einstein, the binary opposition of the tectonic (the convergent) and the hallucinatory (the divergent). The tectonic seized the slippage, it is the ‘arrestment of flux’. (Zeidler. 2010), and the hallucinatory metamorphosis created fluidity. Reality, the real, is in the tension the duality creates. ‘Diagramming the metamorphosis of form on canvas, an affective artist enacts the metamorphosis of human subjectivity within the real’. (Zeidler. 2010).

The real was not a present human consciousness or experience. Rather, for Einstein, realities were within the mechanisms of interaction. Interaction between the viewer and an image was a reality, interaction between individuals became another reality, and so on. ‘The real in Einstein is the very alternation between realities and counter-realities’. (Zeidler. 2010). James Paul Gee writes similarly, “As people are simultaneously members of multiple lifeworlds, so their identities have multiple layers that are in complex relation to each other. No person is a member of a singular community. Rather, they are members of multiple and overlapping communities.” (Gee. 2007. 71). Therefore could the multiple overlapping identities and interactions aligned with temporal grided images provided by videogame be tectonic?

The terms of Einstein’s dialectic (Joyce. 2003) could be employed to analyse the space of videogame within the conceptual capacity of the tectonic if used as the parameters of a semiotic square:

The expanded terms might be composed thus:

Or in a Klein bottle the diagram expands to include:

By using a structuralist Kraussian analysis to negotiate the concepts and navigate the terms, there is certainly a suggestion that videogame may be tectonic. An example of a tectonic videogame is Hyper Light Drifter. It is a multiplatform two dimensional role play game. Aesthetically and through the gameplay, the design references 8 and 16 bit games from the early 1990’s. Movement is constructed using up, down, left and right commands, such that to move away, further into the games environment the command is up, to return, to move back out of the game the command is down. Progression through the game relies on moving through framed spaces with the illusion of height and depth. The ground shifts, shudders and falls away during movement through the spaces. The environment is statically fluid, stable but ever changing. The spaces are filled with beautiful coloured forms and figures, but also charged with horror and nightmare. The game is without written or verbal instruction, and the objective of each part of the game is discovered through patterns and symbols.

If videogame is tectonic, then the space it offers is what Einstein called ‘the real’. Hyper Light Drifter demonstrates many of the elements of the tectonic, and the mechanism of playing the game can be explained using the Klein bottle experiment in Figure 15. This passage from Zeidler reflects the experience of the videogame. “On the one hand, the real is liberatory. It ensures that something like history is possible at all. It enables us to transgress the given and produce new realities … On the other hand, the real enables the transgression of our realities by others in turn. Our freedom and our mortality, the very possibility of new worlds as well as their radical impermanence, go together.” (Zeidler. 2010).

Another example of videogame that equally illuminate a tectonic analysis are Superbrothers Sword and Sorcery. Also a role play adventure game which uses similar mechanics moving through framed spaces and landscapes, although the territory does not shift in the way experienced in Hyper Light Drifter. The landscapes are differently transformative with yawning faces in rocks and crumbling ruins of previous civilization.

Carl Einstein understood ‘the real as a nonhuman structure of the human interaction with one another and with their world history’. (Zeidler. 2010). Einstein’s context was early twentieth century painting. Videogame, a ‘new’ medium, can equally be described as a nonhuman grid and a language structure within which to experience human interactive realities.

In Summary.

“The narrative and formal cannon [of the tectonic] transmits not only past subjective experience and affection but also the dynamics and power to govern contemporary reality”. (Quigley. 2007).

In this paper the tectonic is understood as an experience that enables a structured visual language to exist and be understood in the same moment as an engagement in subjective flux. The mechanics involved in the tectonic resonate deeply with the mechanics of videogame. This study has also presented evidence that there is potential in videogame to be a space for the purposes of art and illustration.

“A dictionary begins when it no longer gives the meanings of words, but their jobs.” (Bataille via Krauss. 2004).

“It is neither the ‘form’ nor the ‘content’ that interests Bataille, but the operation that displaces both of these terms. In this operation of slippage we see … what Bataille calls the informe”. (Bois. 1997).

Fredric Jameson had a theory where an art object was produced as a solution, but the problem to this solution was reasoned by the critic. The producer of the art text as problem solver, as creator of a solution, and the reader as problem maker, as the author of reason. Jameson reasoned that through the analysis of a creative text a collective political unconscious is uncovered. This unconscious rests between the signifier, the text, the solution and the signified, the critique, the problem. Therefore meaning exists somewhere in the space between the axiom and the idiom, the truth and its communication. Jameson was concerned with mapping the onset of postmodernism by basing it in history, tracing a horizontal story through the vertical logic and a-history of modernist ideology. Jameson’s book ‘The Political Unconscious’ was published in 1981. Two years earlier Rosalind Krauss’ paper, ‘Sculpture in the Expanded Field’, was published in the journal October. As a contemporary of Jameson, Krauss was also concerned with the mapping of postmodernism, she stated that it was “extremely important to map” the shift into postmodernism. (Krauss. 1978. 290). She did this by focussing on sculpture and used the Greimas or the Semiotic Square as her map. The square is a mechanism for expanding two opposite or binary ‘positive’ concepts by illustrating the relationship with their ‘negatives’. For instance:

For Sculpture in the Expanded Field Krauss set the positive concepts as landscape and architecture, and the negatives as not-landscape and not-architecture. In the spaces surrounding the square she found room between the terms to place site-construction, axiomatic structures, sculpture and marked sites. (Krauss. 1979. 284). Thereby ‘expanding’ the notion of sculpture beyond an object on a plinth. A contemporary of Krauss, Hal Foster, quotes Jameson’s opinion on the use of the semiotic square, he wrote that the diagram ‘constitutes a virtual map of conceptual closure, or better still of the closure of ideology itself, that is, a mechanism which, while seeming to generate a rich variety of possible concepts and positions, remains in fact locked into some initial aporia or double bind that it cannot transform from the inside by its own means.’ (Marshall. 2015). Jameson indicated that the semiotic square is a vertical mechanism which disrupts the continuity of his historical, horizontal and narrative mapping of postmodernism. Although both structuralist in approach, Krauss’ methodology is at odds with Jameson’s. If Jameson’s theory was put into a semiotic square, it might look something like this:

This exercise of squaring Jameson aims to illustrate the use value in Krauss’ method of mapping. Much criticism has been levied against her use of the square as reductionist and interior. Hal Foster stated that the binary and oppositional nature of the square, although useful for identifying space, refused linear change and development as it did not accept what was outside. (Marshall. 2015). By expanding Jameson’s art-text-as-solution (axiom) and critique-as-problem (idiom) it could be proposed that the square can be used as a location from which to explore beyond its boundaries the implication of which, I hope, validates the use of Krauss’ expansion map for the purposes of illuminating the spaces of videogame. 

In order for the semiotic square to explore beyond its boundaries, a temporal element is required. Such that both mapping devices, the horizontal narrative and the vertical logical, co-exist. A more elegant diagrammatic analysis than mine above has been performed by the architect Sandro Marpillero in his review of Retracing the Expanded Field. Retracing the Expanded Field was a roundtable seminar event involving Rosalind Krauss, Yve-Alain Bois, and Benjamin Buchloh, moderated by Hal Foster, held at the Kitchen, New York in 2014. A book containing the papers, the discussion and academic responses and reviews was published in 2015. The diagram used to illustrate the coexistence of vertical and horizontal is the Klein bottle.

Marpillero states “As a basis for producing conceptual figurations in space/time, the operational logic of the Klein Bottle not only describes the reciprocal exchanges between a participant in an aesthetic experience and the multiplicity of techniques structuring that experience. It also addresses the open-ended processes embedded in the production and realization of an architectural project.” (Marpillero. 2015). He asserts that combining horizontal and vertical into the bottle diagram releases the semiotic square from two dimensional boundaries by intersecting through, between, above and below.From Jameson we understand that development is horizontal, and from Krauss that it is vertical. We have also seen that horizontal and vertical transformation can be combined, in an expanded square or, as in Marpillero’s experiment, using the Klein bottle. This notion has echoes of Grids, a paper by Krauss published at the same time as Sculpture in the Expanded Field. Krauss wrote that ‘There are two ways the grid functions to declare modernity … One is spatial; the other is temporal” (Krauss. 1979. 9). She describes the grid in terms of its flatness, its geometricity, and its ordered rejection of nature, mimesis and of the real. In these terms the grid serves as the archetypal vehicle for modernist visual production, such as can be seen, she suggests, in Mondrian and Albers. The paper develops however, to demystify this statement, by describing the grid not only as spatial with vertical and horizontal axes, but also encompassing temporality. The temporal movement of the grid was characterised by Krauss as centripetal and centrifugal. This description imagines the movement of the grid as spiraling from the boundaries inwards, and in reverse spiralling from the centre beyond the frame into real space. She states that the grid in its bivalence  is ‘fully, even cheerfully, schizophrenic’. (Krauss. 1979). By this she meant that the grid (Videogame, by virtue of it pixelated form, is in fact a grid) undermines its own logic as static form that becomes movement, it is a reproduction of the real, of itself in real space, onto itself. It is its future and its past. If the schizophrenia of the grid is its use of temporality to undermine its own logic, then logically the grid is formless.

The suggestion that the grid is time-based, or four dimensional and not a fragment of a two dimensional plane, allows it to move forwards and backwards, through itself and back again, through time. A function that might be called formless. Formless, or l’informe was a conceptual mechanism of Georges Batialle. Using l’informe Bataille defended his critical position against the purity of aesthetics, academia and humanity seemingly inherent in art in the first part of the twentieth century. A formless, slippery, concept employed to disrail art and destroy its elevated positioning as truth. Terms often used by Bataille Documents:

A Bataillian outlook could be useful for critical analysis of certain videogames such as Ennuigi.

The image above is from indie game Ennuigi. Luigi, famous brother and plumber, wanders around an apocalyptic Mushroom Kingdom. The controls for Luigi are left and right, smoke and ‘speak’. Each phrase appears in type at the top of the frame and consists of an introspective, depressed, sometimes regretful, sometimes nihilistic sentiment. For example, “I look at a turtle, I think I have done you one better. You wear a shell. I have become one.” (‘Ennuigi’. 2015).  The player moves the character from one screen to next deciding whether to smoke or to speak. Josh Millard, Ennuigi creator, states, “this is one lens through which to look at … Luigi, the … complicit onlooker, wandering now through some fractured, rotting liminal place in this strange world, reflecting on it all in scattered fragments.” (Millard. Via Pescovitz. 2015).Could games like Ennuigi be described as formless? Krauss wrote that the opposite of form is chaos, not formlessness (l’informe). ‘Instead’ she writes, ‘let us think on informe as what form itself creates, as logic acting logically to act against itself.’ (Krauss. 1994).

In his review of the exhibition of collages involving, among others, Arp, Duchamp, Ernst, Mirò, Magritte, Man-Ray, and Picasso, at the Galarie Goemans in Paris in 1930, Carl Einstein posed the question “Painting as a language. Why not?” (Einstein via Rumold. 2004). A central theme of Einstein’s theories on modern art, on image making, and more specifically painting, was concerned with visual language. By understanding the cubist flatness of Picasso and the abstract forms of Mason as visual language, Einstein enabled a theory of distinction for modern painting from narrative painting and thereby secured a release from the ‘primordial image’. (Rumold. 2004). Clearly Einstein saw that contemporary paintings were different from what had gone before, but he did not view modernity as a pure or original form of art, as we might understand by the formalist theories of Greenberg. Neither did his view prefer one over the other. Moreover, he understood the limitations of modernist devices such as the grid and its antinarrative stasis, and is perhaps, ‘the earliest critic of the avant-garde cult of (and myth of) purity’. (Rumold. 2004).

By asking his question Einstein is encouraging the reader to perform a visual turn, to consider painting as a language separate from literature and from narrative. To consider an aesthetic visual language, which creates a space for painting, or image making, to become its own entity. Georges Bataille, Einstein’s collaborator and co-editor of the surrealist magazine Documents, was similarly ‘driven by the desire for a spontaneous opening of the image space as an alternative experience’. (Rumold. 2004). As a night time breeze that blows open a window, ‘un coup de vent nocturne qui ouvre une fenêtre’. (Joyce. 2002). Documents was written and published between 1929 and 1930, whilst modern art was in its infancy. Ninety years later, with videogame as a medium also in its infancy, could we ask the same question, Videogame as a language. Why not?

Ian Boggost is an academic and videogame designer that has suggested a need for a new rhetorical domain for videogame. He has stated that the visual expression of art, a ‘visual rhetoric’, can help formulate a ‘procedural rhetoric’ for videogame. ‘Unfortunately, many efforts to unite computers and rhetoric do not even make appeals to visual rhetoric, instead remaining firmly planted in the traditional frame of verbal and written rhetoric in support of vague notions of “the digital.”’ (Boggost. 2007.) What Boggost is suggesting is that videogame, a visual medium, has more involved procedures than other visual means of production and requires not just a visual rhetoric, but a procedural rhetoric. In videogame constructed images are selected and sequenced in order for the images to function as purposeful methods for player interaction. Boggost’s objective appears to be to reinvent or expand a theory of visual rhetoric for videogame or ‘a new medium’. (Boggost 2007). Perhaps Boggost is teetering on the edge of a turn. From verbal and written rhetoric, to an image led procedural rhetoric similar to Einstein’s visual turn from literature.

Professor of education James Paul Gee writes about videogaming in a linguistic framework, in the introduction to his book ‘What video games have to teach us about learning and literacy’ and references Wittgenstein in terms of the impossibility of existence outside language, or an agreed set of rules. Gee views videogame in terms of a multimodal literacy, with various semiotic domains each with an internal and external grammar. The internal grammar is the designers and producers, with story, process and patterns. The external is the players. Gee asserts that the external grammar manifests in the forms of critical discussion through the social media interactions of gamers. Critique is essential to the public establishment of an art text, however the I feel external grammar could be more closely aligned with Boggost’s implied turn. That the player/user/audience through their understanding of visual language and their individual contexts, read the work, the videogame, and create their own meaning. Here there are parallels with how an audience reads or experiences visual communication through art or design. Gee describes this as more complex in videogame, comprising of “six design elements in the meaning-making process: those of Linguistic Meaning, Visual Meaning, Audio Meaning, Gestural Meaning, Spatial Meaning and the Multimodal patterns of meaning.” (Gee. 2007. 65). These design elements form what Gee describes as a ‘multimodal space’ which is the videogame space where a player interacts with the creative work. His theory is based on the notion that the player is the receiver of design patterns and conventions which connote meaning and simultaneously designs the meaning through interaction, by playing the game. This player function of simultaneous connotation and denotation, I propose, is an alignment with Boggost’s ‘procedural rhetoric’, as his step beyond the visual. Perhaps then it is a turn to the multiliterate or multimodal and not a ‘visual turn’ that is in operation? “The challenge” Gee writes, “is to make space available so that different lifeworlds – spaces for community life where local and specific meanings can be made – can flourish. [A space to provide] members of subcultures with the opportunity to find their own voices.” (Gee. 2007. 70-71).

In answer to the question whether videogame has its own language and potential space for a distinct system of artistic communication, I believe that it does. In 1980, at the beginning of commercial gaming, videogame was being described as ‘schema’, that a player made sense of the game through interactive experience and referencing an individual ‘readymade store of similar occurrences and understandings’. (Douglas; Hargadon. 2001). Gee perceives this potential in terms of experiencing semiotic domains as ‘design spaces’ with which to engage, create, and promote problem solving and collaboration. Helping players learn ‘to experience (see and act on) the world in a new way’. (Gee. 2007. 37). Boggost asks us to recognise the procedurality of videogame in terms of persuasive expression and states, ‘As players of videogames … we should recognize procedural rhetoric as a new way to interrogate our world, to comment on it, to disrupt and challenge it’ (Boggost. 2007), and calls for the cultivation of responsibility and consciousness of procedural rhetoric.Visual communication is essential to the immersive nature of videogame. The game Gris is an example of the beauty available in the medium, with consideration of visual communication through form, colour and movement. This is true not only of standard console games, but also on mobile devices. For instance Lifelike available through Apple Arcade (2019) depends on the movement of the visual elements to create feeling through motion.

James Paul Gee wrote that gamers, through videogame are “Learning how to think about semiotic domains as design spaces that engage and manipulate people in certain ways and, in turn, help create certain relationships in society among people and groups of people, some of which have important implications for social justice.” (Gee. 2007. 37-38).

The language of videogame is unique, it is an inherently visual, interactive and immersive dialogue and within this is potential for creativity beyond a traditional notion of gaming. 

“The narrative and formal canon transmits not only past subjective experience and affection but also the dynamics and power to govern contemporary reality”. (Einstein via Quigley. 2007).

‘The Game FAVR’  FAVR: A Framework for the Analysis of Visual Representation in Video Games (2015) is a report researching the language of videogame. The aim was to review and formulate a language to enable an academic discourse for the medium. It was developed due to ‘a need to create a unified framework to analyze videogame images … that would account for the transformative and historical nature [of videogame].’ (Arsenault et.al. 2015). The term historical is used in relation to videogame development from an art historical context, in particular painting. The term transformative is referencing the potential of videogame to enable change. The paper suggests that currently videogame is discussed through an amalgam of borrowed terminology from art history, film and animation. The authors are concerned with creating and presenting a usable toolkit, a ‘unified framework of the ergodic animage, the rule-based and interaction-driven part of visual representation in video games.’  (Arsenault et.al. 2015). They have a structured methodology and aim to locate their research within the analytical game spaces first proposed by Micheal Nitsche in 2003. 

In Nitsche’s model ‘Players … engage in a constant dance between abstract problem-solving and partial suspension of disbelief by deciphering visual information, narrative propositions and game mechanics’. (Arsenault et.al. 2015).

The Game FAVR researchers used Nitche’s model as a foundation from which to define their own set of game spaces or planes. In this diagram FICTION is game narrative, RULES the game mechanics and VISUAL MEDIATION the decoding of visual information presented by the game. All three planes together form the plane of intelligibility, or how all parts combine to enable the player to play the game. The paper exposes a linguistic structure of videogame, categorising it into internal and external occularizations. In terms of composition the authors categorise the space of videogame as ‘1. tangible space, 2. intangible space and 3. negative space’. (Arsenault et.al. 2015). In short, the document makes the case for videogame as language and so develops a methodology that can be used for critical analysis of videogame in terms of image, space, constraints and potential.