The following blog post, unless otherwise noted, was written by a member of GamasutraÂs community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.
Copyright Â© 2018 Chris Solarski, unless otherwise stated / www.solarskistudio.com
This essay can be used as a worksheet for educational and training purposes as long as the contents remains intact and the author, Chris Solarski, is clearly credited. Please consider donating via Gumroad if you wish to use it in a classroom context.
By Chris Solarskiâ
Please note: thisÂ blog postÂ has been divided into 2 parts due the original essay’s 80+ pageÂ length, to make it less overwhelming forÂ theÂ format. You can read Part 1 here, or download theÂ complete essayÂ in PDF format via Gumroad:Â https://gumroad.com/l/TtrHnÂ Â Â Â
2. AESTHETICS AND MATH
2.1 The Origins of Shape Language
2.2 Secondary Shapes
2.3 Cultural Symbols
2.4 Duchamp, Objects and Meaning
2.5 Shape Language and Cultural Symbols in Composition
2.6 Overriding Cultural Symbols
2.7 Shape Language in Non-Western Art
2.8 Shape Language and Composition in Mainstream Culture
3. COMPOSITION FUNCTIONS
3.1 Visual Simplification
3.2 Visual Direction
3.3 Story structure
3.4 Holistic Design
3.5 Concealing the Composition
4. SHAPE LANGUAGE FOR GAME DESIGN
4.1 Shape Language for Game Art and Narrative
4.2 Shape Language in Support of Gameplay
5. THE ADAPTIVE COMPOSITION FRAMEWORK
5.1 The Magic Circle and Defining the User Experience
5.2 Shape Language and In-Game Music
5.3 Shape Language and Interactive Motion Lines
5.4 Interactive Motion Lines and Ludonarrative Dissonance
5.5 Formal Game Design: Single and Binary Control Schemes
5.6 Informal Game Design and Gameplay Vignettes
6. GAMEPLAY IS ART
The first version of this essay was published back in 2013 in the form of a Gamasutra article, after Iâd completed a book tour around North America (kindly supported by swissnex, AlpICT and the Swiss Arts Council, Pro Helvetia) to promote my first book, Drawing Basics and Video Game Art (DBaVGA). I had the privilege of speaking at institutions and events that included The Smithsonianâs The Art of Video Games exhibition, USC Interactive Media Division, and NYU Game Center. Although DBaVGA is aimed at game artists (it features artworks by the likes of Leonardo da Vinci and Nintendo side-by-side), this string of speaking events gave me an opportunity to present the core concepts of traditional art techniques adapted for game art to varied audiences that consisted of the general public, academics, students and professionals working in the games industry. Through this string of speaking events, I gradually developed a new perspective of my work, which enabled me to broaden its application to the entire game development team. What emerged was a game design framework that elevated the aesthetics of gameplay without sacrificing game mechanics.
From left: English (Watson Guptill 2012), Japanese (Born Digital Inc. 2014) and Korean (Freelec 2015) editions of Drawing Basics and Video Game Art (Watson Guptill 2012), and Interactive Stories and Video Game Art (CRC Press 2017) by Chris Solarski.
My second book, Interactive Stories and Video Game Art (ISaVGA), was the result of that retrospection. Iâm very proud that ISaVGA has received positive endorsements from distinguished creatives including Hollywood director, Marc Forster, and veteran game production consultant and Assassinâs Creed founding member, StĂ©phane Assadourian:
“The concepts presented in Interactive Stories and Video Game Art remind me of Joseph Campbell’s powerful demonstration of a universal storytelling structure. Coming from a technical game design background yet being artistically inclined, I never considered that atomic elements such as shapes, lines of movement and transitions contain a vocabulary strong enough to sculpt our mood, whether consciously or unconsciously. This book creates that wonderful feeling of learning new letters, which soon prove useful to form new sentences for bringing new poetry to the world.”
â StĂ©phane Assadourian
This 2-part essay (read Part 1 here)Â provides a disruptive game design framework based on Interactive Stories and Video Game Art, with original content and refinements that have arisen since the book was published. Readers will gain in-depth insights into shape language and composition, tracing the lineage of these long-standing techniques from antiquity to contemporary mainstream culture and games, where they are translated to adaptive composition. Adaptive composition is a high-level game design framework for heightening physical empathy and the sensory experience of players with a focus on art and narrative-driven games and transmedia. Shape language and composition areÂ framed in the form of Jesper Juulâs puzzle piece metaphor to provide a practical interpretation of Raph Koster’s theoretical work, with the aim of challenging game design conventions, and illustrating adaptive compositionâs relevance to the entire development team of game and narrative designers, artists, writers, programmers, musicians, game user researchers, etc. Practical exercises throughout the essay provide design tools for averting ludonarrative dissonance, and demonstrate the shortcomings of the magic circle metaphor, and common misconceptions regarding Marcel Duchamp’s teachings on objects and meaning. This essay is therefore sure to be of interest for those that are already familiar with Drawing Basics and Video Game Art, and Interactive Stories and Video Game Art.
Whether youâre a developer of video games, VR, AR or XR, a filmmaker, game user researcher, a gamification and transmedia expert, game theorist, psychologist, or simply interested in experiencing resonant and meaningful interactive storiesâyouâll find the following insights will awaken a new appreciation for art and craft, and provide you with creative tools that can be actioned immediately to influence the user experience.
Shape language and composition constitute visual storytelling techniques that have been refined for thousands of years dating back to antiquity. The artistic medium to which they are applied may be static (painting, comics and illustration), dynamic (film and animation) or involve audience agency (Buddhist sculpture and interior design) but their benefits remain consistent: to convey functionality, set the aesthetic tone of the subject and reduce visual complexity of the artwork. This sections explores these benefits in more depth, giving us a practical understanding of shape language in the context of composition that we can later apply to game development.
Massacre of the Innocents (c. 1611) by Peter Paul Rubens. (Image source: Wikipedia)
Take a moment to appreciate the complexity and details of the Rubens painting, above. Now imagine setting such a complex arrangement of visual elements in motion, as in a typical video game. We go from a semi-engaged âspectator modeâ with time to casually explore and contemplate cultural symbols, to a fully-engaged âagent modeâ where our attention becomes absorbed by interaction and the functional requirements of gameplay. Without an underlying composition framework for organization, visual information can quickly become overwhelming. The elegance of composition is that it enables artists and designers to reduce complex artworks to concise visual statements with the following benefits:
A composition based on a primary shape creates visual simplification.
To illustrate visual simplification, letâs remove everything from Rubensâ painting except the triangle at the heart of the composition. Youâll find other composition lines hidden within the paintingâsuch as the vignettes at the corners to keep the viewerâs eye contained with the picture frame (a technique similarly used in Monument Valley (2014) using subtle shadowing instead of lines)âbut our focus here is the central triangular concept, illustrated above. Stare at the middle of the triangle for a few seconds. Your eyes will be tempted to jump to the triangleâs three points because each point functions like a visual exclamation mark. However, keeping a fixed gaze on the triangleâs center is fairly easy. A circular composition would make it even easier to remain fixated upon because of its edge-free silhouette.
An image that doesnât follow a structure can become visually chaotic.
Now try the same exercise with the above compositionâthe result of randomized brush strokes in Photoshop. Youâll find that keeping your gaze fixed on a single element is near impossible because of the increased number of points to take in. The visual chaos is just too tempting for your attention to remain in one placeâillustrating one of the major benefits of shape language and composition.
The second function of composition is to serve as a visual guide that covertly influences viewers on a subliminal level. Elements that fall along composition lines (as illustrated below) will receive more attention than elements outside the framework, even if the audience is unaware of the embedded shape language.
Elements that fall along composition lines attract the viewerâs
attention more than elements outside the framework.
Conveying story or subtext is the third benefit of composition. Rubensâ Massacre of the Innocents is based on a skewed, unbalanced triangular composition that perfectly underscores the brutality of the paintingâs theme. The shape language of the compositionâwhich should ideally remain beyond the audienceâs perceptionâcan dramatically alter the tone of the cultural symbols being depicted.
The final benefit of composition concerns the development process at large, by encouraging a holistic design framework that prioritizes the unity of the entire artwork over details. A great example of this approach is illustrated by the HellboyÂ comic examples explored in the previous Shape Language and Composition in Mainstream Culture section (see Part 1).
In the following composition analysis of Raphaelâs Niccolini-Cowper Madonna we can see the techniques that classical artists used to imply primary shapes through composition using the deliberate placement of figures, gestures and silhouettes, drapery, shadow edges, and environment props. Every element has a function within the composition and has been manipulated to support the larger shape concept. Extraneous details have either been made to align with the composition, been subdued or removed altogether. The resulting composition lines serve to covertly direct viewers to key storytelling elements, such as Maryâs gaze towards Jesus.
Classical artists manipulated every element of a painting to covertly imply a primary shape, including figure gestures and silhouettes, drapery, shadow edges, and environment props. Niccolini-Cowper Madonna by Raphael.
Note that the triangle embedded in Raphaelâs painting is incomplete and âbroken.â This was a deliberate technique to conceal the composition framework so that viewers would not be distracted by the craft. A lay-person exploring a classical painting (or comic) would therefore believe that they’re absorbed solely in the cultural symbolism contained within the image, exploring the artwork at their own volition, while being covertly directed by the composition and the aesthetics of shape language. Using both affordances to affect viewerâs covertly on a primal level (universal shape language) and conscious level (subjective cultural symbols) is what leads a classically trained contemporary artist and teacher like Dorian Iten to say that âshape language is as valuable as visual grammar.”
Although weâve been analyzing shape language and composition in paintings up until now, itâs also important to reference composition in classical sculpture because of its relevance to 3D video games. Sculpture is where we find painting composition techniques projected into 3D space so that viewers can freely explore shape language in the round (360 degrees). The sculpture below of David by Gian Lorenzo Bernini (1598-1680) is among the finest examples of a three-dimensional composition with a shape concept that can be perceived from all sides.
[The application of shape language in Greco-Roman and Renaissance art goes deeper to an anatomical level, which you can learn more about in books like Robert Beverly Haleâs, A Master Class in Figure Drawing (Watson-Guptill 1991)].
Three views of David (c. 1624) by Gian Lorenzo Bernini.
We inherit all the above benefits of shape language and compositionâvisual simplification, visual direction, story structure and holistic designâwhen we adapt the techniques to game design. Adaption is straightforward if we consider the playerâs constantly shifting viewpoint and the dynamic properties of the medium. What is essential is that these techniques are appreciated by the entire development team if theyâre to be implemented effectively. The following sections tackle the answer that weâve been deferring for question (2), concerning the practical value of shape language for game design in art and narrative-driven gamesâdemonstrating why such design principles are not just a concern for visual artists and storytellers.
Shape language for game design relies on the same principles that we explored in the chair exampleÂ (see Part 1), and the previous section on the benefits of composition. Firstly, key symbolic elements of the gameplay experience are identified before an appropriate shape language and composition structure is considered that will evoke the desired aesthetic themes. Game artists regularly use these techniques to manipulate every visual aspect of a video game like character and environment designs however, the same techniques are also applied to game design to convey gameplay affordance. So letâs explore a few examples of shape language in game art and game design, in the context of Raph Kosterâs definition of games as âsystems built to help us learn patterns for survival.â We also wish to further challenge the notion that psych is not amenable to math.
From left: Mario in his original pixel form, and a contemporary,
high-resolution version. Artwork by Nintendo.
Letâs begin by taking the earlier chair and Tony Montana examples (see Part 1), and applying them to Mario from Nintendoâs Super Mario franchise. Marioâs base symbolic elements are his mustache, overalls, cap, protruding nose, and shoes. Color, of course, also plays an important part in identifying Mario but our focus here is on shape language. Even if the stylistic representation of Mario is altered, the character will remain identifiable as long as its base symbolic elements remain intactâillustrated by the blocky pixel version of Mario alongside the modern, high-resolution version, above.
Manipulating the shape language of Marioâs base symbolic elements results in characters with different temperaments. From left: Mario, Luigi, Waluigi and Wario by Nintendo.
Manipulating Marioâs shape language results in a character with different temperamentsâechoing the example of glass objects (see Part 1). In the line-up below we see that Marioâs base symbolic elements have been aligned to the square (Luigi) and the triangular (Waluigi and Wario). Luigiâs forms remain largely the same as Marioâs but his overall shape has been stretched upwardâvisually aligning him to Mario but with a pillar-like aesthetic. Ignoring proportion differences, Waluigi and Wario is where we see the strongest shape language changes to Marioâs base symbolic elementsâsuch as the ears, nose, mustache and shoesâwhich have been aligned to the triangle. The results of these shape shifts are aggressive, alter ego versions of Mario and Luigi, where the symbolic ingredients of one iconic character have been manipulated with shape language to convey different temperaments.
Every visual aspect of a gameâthe environment, props, user-interactive, etc.âcan all have their shape language manipulated while keeping symbolic elements intact to elicit varying aesthetic themes associated with primary shapes and their derivatives. Shape tuning need not be so overt, with subtle manipulations of shape language possible by varying a characterâs outfit to change its overall silhouette (see examples from Framed InkÂ inÂ Part 1), complemented by drapery and the arrangement of accessories.
Kirby by Nintendo.
Using a pencil and paper, first make a faint copy of Kirby, and then begin adjustments to the characterâs facial features and overall shape. Can you give Kirby an aggressive or angry appearance without introducing diagonal lines or shapes? If you do succeed, please share your artwork via the #AngryKirby hashtag. The result will likely shift Kirbyâs temperament closer to Meta KnightâKirbyâs archenemy.
A characterâs poses and gestures are also responsive to shape language. Note that Mario, Luigi, Waluigi and Wario have each been assigned a pose that echoes their shape language to underscore their respective temperaments. Marioâs jump aligns to a larger circle; Luigi stands steady and upright; Waluigiâs pose echoes depictions of the Vajrapani in Buddhist art (see Part 1); and Warioâs arms create a thunderous zigzagging line across his form. We could imagine adapting the Zen teachings of Suzuki ShĆsan, described earlier, inviting audiences to adopt the pose of each character, such as:
Luigi is Marioâs dependable, taller brother and sidekick. He resonates stability and confidence which you can absorb from him in times of uncertainty.
Character shapes and poses aligned to primary shapes help players distinguish characters more easily as well as prime them physically and emotionally for certain behavior. Additionally, shapes and poses that adapt dynamically to gameplay can imbue characters with a greater sense of life and emotional depth, and convey their intentions through nonverbal body languageâreducing the necessity for a user interface and character stats.
Note that adaptive character poses appear in games like the Resident Evil series, where the protagonist becomes physically impaired when injured. However, this has more to do with communicating the characterâs health stats (much like a user-interface icon), than an emotional change.
Shape language reference for character races in Destiny (2014) by Bungie. Created by Bungieâs former Art Director, now Game Director, Christopher Barrett.
A common criticism against shape language is that it leads to overly simple visual narratives and stylistic representations. However, classical paintings and comics like Hellboy (see Part 1)feature shape language on many levelsâfrom the overarching composition to the smallest anatomical detailâand nobody would argue that they are visually simplistic or present subtext in an overt, obvious manner. Shape language encompasses all stylizations, and can be used to convey complex narratives. In the context of gameplay this also means that we can use shape language to organize the fast-paced flow of visual information, and make it easier for players to read a gameplay environment for functional and aesthetic affordances. This is where shape language has a clear advantage over cultural symbols.
Shape language is used in games with idealized as well as realistic style. From left: Super Mario Galaxy 2 (2010) by Nintendo, and Uncharted 4: A Thief’s End (2016) by Naughty Dog.
For instance, the idealized style of Super Mario Galaxy 2 (2010), by Nintendo (above left), features characters and environment props that align to either the circle, square or triangle depending on their affordance: friendly non-player characters (NPCs) and bonus objects like warp pipes and mushrooms are generally round; jumping platforms are square; and enemies are triangular (albeit with softened tips to align with the family-friendly atmosphere of the series). The affordance of gameplay elements becomes easier to comprehend when pure shape language is used and players mustnât second-guess whether, say, an NPC is friendly or will inflict damage.
Shape language illustration conveying gameplay affordance, by Emilia Schatz for 80.lv.
Games with realistic stylizations, like Uncharted 4: A Thief’s End by Naughty Dog (above-right) use shape language for gameplay in much the same way as the Super Mario series, but their implementation is harder to discern at a glance. In an article for 80.lv titled, Defining Environment Language for Video Games, Emilia Schatz, lead level designer at Naughty Dog, summarizes the functional affordances of shape language for gameplay as follows:
Â âRound shapes evoke safety and well-being. Theyâre viewed as non-threatening due to their lack of points and edges. They blend in and disappear, lacking affordance. Rectangular shapes on the other hand are solid, stationary, and powerful. They evoke a feeling of strength, either as an obstacle to overcome, or something to be reliably used. Rectangular shapes are strong indicators of affordance. Finally, diagonal and pointy shapes evoke a sense of danger and imbalance.â
â Emilia Schatz
Shape language in Uncharted 4: A Thiefâs End, by Naughty Dog. Illustration by Emilia Schatz for 80.lv.
In the above illustration you can see that Emiliaâs overlays indicate how shape language was used in support of gameplay for Uncharted 4: A Thiefâs End. By keeping the environmentâs shape language as distinct as possible, Emilia ensured that the functionality and location of climbable objects remains easy to identify. Non-interactive objectsâthose that align to circles and trianglesâare visually subdued or avoided. The visual distinction of the circle, square and triangle is why shape sorting puzzles for kids likewise feature the primary shapes, because they help in the development of visual perception.
MORF (2011) by SOLARSKI STUDIO.
Shape language can also be used as an illusion for narrative purposes, without necessarily affecting gameplay. MORF (2011) is a simple browser-based game developed to explore the aesthetic links between character and environment shapes, which you can play here. Players must guide a round character through two functionally identical environments that have differing aesthetic themes. I had the opportunity of presenting MORF at an art exhibition that included an audience of experienced players and individuals that had never played a game before. Players would bump their way through the first, circular level without incident, but when they reached the second, edgy level (pictured above) and accidently landed their character on triangles, theyâd exclaim words like âouch!ââwords that we use when we hurt ourselves in reality. This suggests that interactions between playable characters and the environment need not always have gameplay value, when an aesthetic response is desired.
The shape spectrum below is a valuable comparative analysis tool to assess the aesthetic alignment of characters, and the gameplay affordances and emotional themes that players will likely attribute to their design. Characters that align to the circle tend to be friendly, caring, cheerful, dynamic and âjumpyâ; square characters tend to be diplomats, wise, serious or heavy âtanksâ; and triangular characters usually indicate a bad temperament, anguish or evil intentions however, triangles can also indicate a friendly character with a âzippyâ personality. These aesthetic associations reference the tactile senses that we explored via the glass sphere (marble), cube (paperweight) and pyramid (spearhead) example in the opening section of this essay (see Part 1). The shape to which a
character aligns may be presented explicitly, in the characterâs silhouette, or implied via discreet physical and costume details like the vertical stripes on Dr. Liara T’Soniâs uniform. Naturally, the shape spectrum can be used to assess a single playable character across a range of emotional states, a cast of characters and non-player characters.
The shape spectrum is used as a comparative analysis tool for assessing the perceived gameplay functionality and temperament of characters. Characters from left (incl. copyright owner): Sora (Square Enix), Elizabeth (Irrational Games), Isabelle (Nintendo), Dr. Liara T’Soni (BioWare), Mei (Blizzard), Adam Jensen (Eidos MontrĂ©al), Tracer (Blizzard), Bowser (Nintendo), Bubble Head Nurse (Konami).
Players can, of course, be conditioned to understand that a triangular character is calm and friendly within the confines of a gameâs system. Although such conditioning must be treated with care, since the game may feel unfair or suffer from usability issues if interaction outcomes donât match with player expectations. In the words of Emilia Schatz:
âIf you feel your visual language is not reading as clearly as you had hoped, adjusting your primitive [primary] shapes may be one way to help.â
â Emilia Schatz
Shape language for gameplay requires careful balancing, which is a task that is further complicated when cultural symbols must be considered. The shape of the playable character in Journey (2012), often referred to as the âtraveler,â supports gameplay because it references wind kitesâa symbol that is familiar to most cultures around the world. Familiarity with kites and how they function, means that the travelerâs visual references make the flying and gliding aspects of Journeyâs gameplay easier for players to understand. While the traveler has a pointy shape, its gentle temperament is predominantly conveyed through its movements, which weâll explore shortly in Shape Language for Interactive Motion lines. If Journey featured a larger cast of characters, then the development team may have encountered instances of players second-guessing whether the traveler is a friend or foe. In which case, the traveler could have been given rounder, Mario-esque forms while maintaining the kite reference but that would have had knock-on effects to all other visual elements of the gameânot to mention, aligning the characterâs visual theme with round forms, which would likely be at odds with the team’s artistic vision. These considerations reference the design techniques as we explored with the Trinity, and shape language and cultural symbols in Christian paintings (see Part 1).
The traveler in Journey (2012), by thatgamecompany (left), uses shape language and the cultural symbol of a kite (right) to convey character and gameplay functionality.
Our exploration of shape language started with the Ancient Greek philosophers and their tactile associations with the four classic elements: Earth, Air, Water and FireÂ (see Part 1). We found that shape language is a universal concept because it pervades Nature and is the result of physical laws. Itâs no consequence, therefore, that we find shape language used in everything from engineering to art and design, from East to West, including typography, vehicle design, figurative art, architecture, interior design, furniture, comics, film and animation. The examples in this section demonstrated that shape language is equally important for video games to convey the aesthetic and functional affordances of gameplay elements like characters and environment objects (see shape spectrum earlier), for stylizations that range from from abstract and idealized (Super Mario Galaxy series) to realistic (Uncharted series). We can therefore conclude that shape language does indeed support Raph Kosterâs definition of games as âsystems built to help us learn patterns for survival.â Perhaps, more importantly, the math that defines primary shapes has aesthetic and functional value for the user experience with significant implications on game design and game programming.
Now that we have some answers to question (2) with a preparatory understanding of shape language in support of gameplay, we can go ahead and explore composition in a wider gameplay contextâconsidering the users experience in its entirety via a disruptive game design framework inspired by traditional composition techniques.
Composition is nothing more than the act of combining parts or elements to form a whole. In the context of art, modern day master, Nathan Fowkes, describes composition as the âthoughtful arrangement of pictorial elements to best portray subject matter.â Composition offers several benefits that include visual simplification, visual direction, layering of story or subtext, and a holistic design framework (see Composition FunctionsÂ earlier).
Laying a classical painting down flat and imagining the composition framework as navigable pathways gives us an instant solution for translating visual concepts to interactive environments.
Translating composition techniques to games is fairly straightforward despite the unique way in which the medium is experienced. We need only lay a classical painting down flat and imagine the composition framework as navigable pathways to get a feeling for the possibilities. The lines that viewers would implicitly detect with their eyes when looking at a classical painting or comic, now become pathways along which players can traverse through a three-dimensional environment.
Adaptive composition is the subject of the first half of Interactive Stories and Video Game Art (where itâs referred to as dynamic composition). Adaptive composition is a high-level framework that implies that the composition can respond dynamically to gameplay and narrative contextâcreating heightened physical empathy and sensory experiences. As you will recall from previous sections, the functional elements used to embed a composition are little more than lines, shapes and volumes that informed the design and placement of cultural symbols. In the context of game design, you can think of adaptive composition as the fundamental elements needed for the whiteboxing stage of developmentâthe preliminary prototype stage where key gameplay elements are blocked-in with placeholders to test if the gameâs basic design supports the desired experience before final assets are created. Itâs upon this foundation of adaptive composition that layers of detail will be added (character design, light, color, voice acting, animation, etc.) by the various disciplines involved in the development process.
The visual elements that are used to embed shapes in compositions across disciplines like painting, photography, film and comics (the deliberate placement of figures, gestures and silhouettes, drapery, shadow edges, and environment props) are translated to the boldest elements we can find in a video game space. Elements must be bold to account for interactivity and multiple perspectives. This does not mean that the elements used in other media cannot be applied to games, but theyâre often too small and subtle to be noticed in the midst of gameplay, and should therefore be treated as secondary.
Visual elements used to indicate shapes in painting or comics compositions translate to bold elements (i.e. character and environment shapes, and pathways) in the context of games to account for interactivity and multiple perspectives. Â
Unlike traditional art forms, interactivity means that player actions are bound to the visual and artistic experience and must also be considered in the context of composition, even though they occur outside the picture frame. An important consideration for game design is that each element of adaptive composition is pervasiveâmeaning that it is consistently present during gameplayâto ensure that players are always aware of it despite a constantly shifting viewpoint within the game world. To get the most out of dynamic composition itâs also important to treat each element as adaptiveâtuning the elements along the shape spectrum to amplify or subvert the aesthetics in relation to gameplay and/or narrative. The eight elements of dynamic composition are as follows:
The goal of these eight elements, used in conjunction with shape language, is to arrive at the same level of visual clarity and covert storytelling as weâve explored in traditional art and contemporary media like film and comicsÂ (see Part 1).
From left: the static, vertical composition lines in The Baptism of Christ (c.1450) by Piero della Francesca are echoed in the virtual 3D forests of Firewatch (2016) by Campo Santo.
Using adaptive composition, the game screen can be crafted to resonate with round friendliness (i.e. Animal Crossing series), the poised stability of vertical straight and horizontal lines (i.e. Superbrothers: Sword & Sworcery EP), or vibrate with triangular aggression (i.e. Devil May Cry series), which must be experienced in motion through interactivity for the full aesthetic effect. Be sure to see the later references to ABZĂ in this context.
What makes identifying the eight elements of adaptive composition possible is the ubiquity of shape language, without which we would have to contend with an infinitely complex Duchamp-ist universe. Instead we rely on 3000+ years of humanityâs direct study of Nature, which evolved into steadfast methodologies that transcend engineering, art and design fields in real and virtual contexts. Shape language and cultural symbols are not mutually exclusive but prioritizing shape language over Duchamp’s teachings gives us a manageable framework that every player and individual on the development team can intuitively appreciate irrespective of cultural background and expertise.
Illustration of the magic circle metaphor that defines play in binary terms.
(Image source: Gaming Conceptz)
What may be of particular interest for game theorists is the adaptive composition frameworkâs value to help define play. The magic circle metaphor, popularized by Eric Zimmerman and Katie Salen, is the most commonly referenced definition of play. Despite its popularity, the definition has been widely criticized because it defines play in binary terms: youâre either acting within a space designated for play, or performing the same actions in a real-world, non-play context. For this reason, Jesper Juulâs puzzle piece metaphor is more apt at defining play because it imagines play as a system connected to different interfaces on its sidesâechoing the links that shape language establishes between games, the real world and transmedia. The puzzle piece metaphor significantly improves upon the magic circle by making it a) âeasier to identify how a game fits a contextâ and b) âhow players enter and leave a game.â
Because Jesper is a game theorist, point (a) explores the boundaries between games and their surroundings (what is and what isnât play). The adaptive composition framework, on the other hand, goes one level down to explore the boundary between the game and players with a focus on the physical aspects of interactionâthe essence of what makes games unique from other art forms. The aim here is to make it easier to identify how component parts of games (sensory inputs) influence the playerâs physical movements (physical outputs) and emotions.
The adaptive composition framework also serves to demonstrate that a traditional artistâs perspective of gameplay and interaction has richer, fluidly interconnecting boundaries across transmedia and reality than the magic circle suggestsâreferencing Jesperâs point (b) and how players enter and leave a game via the game boundary. As weâve found throughout this essay, games have indisputable connections with creative disciplines as varied as film, comics, board games, virtual reality (VR), augmented reality (AR), cross reality (XR), theatre, architecture, etc. The adaptive composition framework can be applied to all these art forms, requiring only minor conceptual adjustments to fit each medium. For instance, character shapes in location-based games are a combination of the player’s real world physique and attire, and their in-game avatar [see the following analysis of a location-based game titled, Crossover Strategies]. Similar to Jesperâs concept of âinterfaces,â not every game will utilize all eight elements of adaptive composition, with elements occurring in active, inactive, or absent states.
Weâve already encountered a similar observation in Raph Kosterâs quote, in which Raph states that the essential considerations for making long-lasting games (human psychology, etc.) are âelements that arise from outside the gameâs rules and from outside the âmagic circle.ââ [Italics added for emphasis]
The eight elements of the adaptive composition framework are visually presented below in the form of an octagonal puzzle, with each element serving as a sensory input to the player located at the center. The octagonal puzzle is set within a square that represents the game boundaryâallowing the high-level framework to be tessellated with reality and transmedia, like board games, location-based games, films and comics. Interactive motion lines have been emphasized because of the elementâs higher importance. The overarching concept highlights the shortcomings of Eric and Katieâs magic circle metaphor as well as suggesting a more tangible approach for defining play.
The adaptive composition framework illustrated with reference to Jesper Juulâs puzzle piece metaphor.
We’ve already explored two elements of adaptive compositionâ character shapes and poses, and environment shapesâin some depth throughout Shape Language for Game Development, earlier. We’ll briefly explore two additional elementsâaudio, and interactive motion linesâto demonstrate the versatility of shape language and the adaptive composition framework.
Audio in the context of adaptive composition focuses on sound that supports gameplayâwhen players are in control of a video game character. Audio is unquestionably a nonvisual medium, which makes it difficult to imagine how it fits with shape language. To solve this issue, we can look to the visual manifestation of music that we know as dancingâthe act of physically turning something that is intangible (sound) and making it tangible (physical movement). Music has such a primal influence on us that it can trigger and choreograph physical activity on a subconscious level. For instance, weâve all had the experience of listening to a song and realizing that our foot has been tapping the musicâs rhythm without our full awareness. When we dance our whole body becomes instinctively engaged in the act of translating sound into form and movement (although some people are more naturally gifted than others!).
From top: sine (circle), square, and triangle melodic contours in
music directly correlate to our shape spectrum of primary shapes.
There are several components of audio to consider in support of gameplay but weâll focus here on melodic contoursâthe shape that notes and pitches describe across a musical score. Melodic contours fall into three primary categories that echo our shape spectrum: curved (sinusoidal), square and triangular (illustrated above). As with character shapes and poses, melodic contours can pull on our kinesthetic strings when we play (or dance) to evoke corresponding physical gestures. Adaptive gameplay music, therefore, serves to cue players to an appropriate mode of interaction. The playerâs gestures represent an asynchronous interpretation of music that is strongly influenced by the aesthetics of melody (and tone, rhythm and tempo) but not coordinated with the same precision as in dance. To the benefit of game designers, music can suggest more nuanced movements than game design constraints may allow.
The work of game audio designers echoes that of silent film accompanists, whose job it is to musically portray the protagonistâs movements and amplify actions with sound effects.
While we can aptly relate player activity to that of a dancer responding to music, the role of game audio designers has a clear resemblance to silent film accompanistsâlike those that played to Buster Keaton (1895â1966) and Charlie Chaplin (1889â1977) films. Audio designers working in video games face the same challenges of portraying the playable characterâs movements through music and amplifying actions with sound effects. A composer must work with animators, game designers, and scriptwriters to ascertain in advance the playable characterâs varied emotional states, stride tempo, andÂ abilities to determine the music that will accompany players as they experience the gameâs narrativeâmuch like the accompanist deciding which song will best fit the action of a silent film.
Music clips from The Legend of Zelda: Ocarina of Time (1998) soundtrack by Koji Kondo. Copyright by Nintendo.
Listen to the above three excerpts from the The Legend of Zelda: Ocarina of Time (1998) soundtrack with a pencil and paper in hand. While listening, draw lines that correspond to the melodic contours that you perceive in-between the beat. You can go back and forth over the pencil lines, as the drawing itself is not importantâonly the physical experience of drawing, which may feel as if you are dancing across the page. Pay particular attention to the sensations in your drawing arm and how they change between music clips. Do the melodic contours evoke movements that you could imagine as analog stick inputs on a controller, relating to character movements in leisurely locations, spiritual sanctuaries and enemy dungeons in the Ocarina of Time? Would you use any of the following adjectives to describe your lines: cheerful, relaxed, respectful, tranquil, zippy, edgy or aggressive? How about button presses? How frequently and with what intensity would you imagine pressing action buttons? Are these sensations evoked by the musicâs beat or other musical elements?Â Â
The accompanistâs spontaneous accentuations and impromptu notes that underline special events are equivalent to inÂgame sound effects that are designed by audio teams but activated spontaneously by players during gameplay as their character âdancesâ across the screen. As David Kanaga, the composer behind Proteus (2013), and Panoramical (2015), puts it:
“There’s all these really detailed time structures happening in games. So when I’m doing music I try to think, ok, the game already has time structures in place, and the goal of the music oughta be to just hug that as tightly as possible.”
âÂ David Kanaga
Interactive motion lines in the context of adaptive composition are not concerned with âlocalâ animations, like the movements of a characterâs hand or facial expressions, because such animations are often too subtle to be noticed in the midst of action. Our primary concern is for broader animations that remain readable at every resolution. Motion lines are the dynamic, in-game equivalent of lines used to compose paintings like Niccolini-Cowper Madonna by Raphael, and Hellboy comics by Mike Mignola.
Frame from Ballet Rotoscope (2011), by Masahiko Sato and EUPHRATES.
The image above is a frame from an experimental video titled Ballet Rotoscope (2011) by Masahiko Sato and EUPHRATES, which investigates computer-generated geometry. The experiment nicely illustrates the concept of interactive motion lines using live footage. If thereâs only one exercise that you try, be sure to do the following Motion Lines Study, Part 1 exercise, which consists of drawing the interactive motion lines experienced by players of Journey, INSIDE (2016), and Vanquish (2010).
From left: Journey (2012) by thatgamecompanyÂ (original video by GadgetGirlKylie);Â âINSIDE (2016) by PlaydeadÂ (video clip via IGN);Â Vanquish (2010) by Platinum Games (original video by TheBlueDragon)
Using a pencil and paper, watch each of the above gameplay clips and draw the motion lines that you perceive for each character. You can think of motion lines as the trails created by Light Cycles in Tron (1982), even though such trails are generally not visualized. The focus of the exercise is on the physical sensations of drawing, which are directly linked to the physical movements of a player controlling the character with an analog stick. How do the signature motion lines for each character aesthetically differ between gameplay clips? Do the physical sensations you feel correspond to shape language? Naturally, the concept of motion lines also applies to NPCs.
Gameplay in INSIDE (2016), by Playdead, presented in a comic page format.
The exercise of drawing motion lines can be used as a method to visualize narrative as a continuous thread that binds the various gameplay sequences into a coherent whole with a focus on the playerâs physical gestures, as illustrated by INSIDE, above. The horizontal gameplay section (top row), evokes nervous back-and-fourth movement as the player evades the guards. In the vertical gameplay section (right
column), the playerâs physical outputs are linked to the submarineâs spotlight, which is responsive and âtwitchyâ compared to the flowing movements of the submarine itself (yellow line). Note that the overlays below have a strong conceptual link to the composition lines in Hellboy comics Â (see Part 1), and that tracking interactive motion lines is applicable to all game viewpoints, including 2D, 3D and 4D (see Miegakure by Marc Ten Bosch) views.
StarCraft II: Heart of the Swarm (2013) by Blizzard Entertainment. Original video by RedAlert2Havoc.
Real-time strategy (RTS) games like Blizzardâs StarCraft heavily rely on shape language to convey functionality of game units. In the above video clip from StarCraft II: Heart of the Swarm (2013) youâll see that resource gathering units move smoothly to and fro, while attack units move erratically with jagged motion lines. In all the visual noise, these distinct movements allow players to discern unit types peripherallyâwith jagged movements attracting more visual attentionâwithout having to focus directly on a given unit. This effect is achieved principally because âcollisionsâ are switched off for workers, and their animations speed up and slow down smoothly. In contrast, acceleration and deceleration of attack units is abrupt, and they have their collisions switched on, and must therefore make additional movements to avoid objects and each other.
ABZĂ (2016) by Giant Squid.
The masterfully orchestrated motion lines and shape language in ABZĂ (2016) also deserves in-depth analysisâsuch as the aesthetic differences between free exploration, current riding, and careful circumvention of exploding mines as well as the beautiful, natural shape language of fish shoals, as seen below.
Motion lines may seem like an obvious consideration for game developers but implementing them correctly to heighten the playerâs physical empathy towards a life-like playable character may fail for one of two reasons.
The first reason occurs when the development team loses sight of motion linesâprioritizing other, less essential aspects of game experience. This can easily happen, especially when huge development budgets and team sizes are involved. Take, for instance, the gameplay sequence below from Halo 4 (2012). Using a pencil and paper, draw the interactive motion lines of Master Chief, while the playable character pilots a Ghost vehicle. Youâll find that the sequence is very linear, evoking little lateral movement. The few turns that Master Chief performs are gentle because of the Ghostâs slow turning speed.
Halo 4 (2012) by 343 Industries. Original video by MythicTyrant.
The above gameplay sequence represents one of the dramatic highpoints of Halo 4âs narrative (the player has just unwittingly released the Didact and the universe is in peril), so the physical experience should be closer to Vanquish if players are to feel like theyâre empathizing with a life-like Master Chief. Instead thereâs a decided lack of tension that is at odds with the situation despite so much eye-candy and sound effects in the sequence. Academically this is known as ludonarrative dissonance, which is when the user experience is disconnected from the narrative.
The second reason that motion lines may not match the intended play experience is due to platform limitations, not developer oversight. For instance, ludonarrative dissonance can occur in Proteus if players come to the game in an agitated state of mind. Proteus is a meditative experience with free-form exploration and no specific goals for players to accomplish. The gently billowing plants and slow movement of the disembodied character sets the pace at which players can explore the environment. Nothing on-screen encourages players to make erratic, jagged gesturesâlike shaking the camera back and forthâbut the gameâs mouse and keyboard input means that the characterâs pace is developer-defined but the camera isnât. You can purchase Proteus here and try it yourself.
The mouse and keyboard controls of Proteus (2013) by David Kanaga allow players to make gestures that may not be inline with the authorâs intended experience.
It’s true that most individuals playing Proteus will come to the game with the right mindset but it’s nonetheless important for designers to carefully curate the aesthetics of each element of adaptive composition and assess platform limitations to ensure that the potential for ludonarrative dissonance is reduced. Otherwise players may get the sense that their virtual head belongs to a Quake III Arena (1999) contest, while their pace references a character from Animal Crossing (2001). In the case of mouse and keyboard interfaces, curating motion lines is, admittedly, more difficult than with console controllers.
Where the adaptive composition framework aspires to have the greatest disruptive effect on game design concerns interactive motion lines in formal gameplay. Formal gameplay is here defined as gameplay with explicit objectives and consistent rules that players must master to win. Such games are identified by their heavy UIs and end-of-level stat screens, which are hallmarks of pen-and-paper role-playing game (RPG) that enumerate the playerâs progress towards a gameplay objective. Playable characters in formal games are aesthetically constrained to a very limited range of motion lines because their control schemes are bound to an intricate system of interactions.
For example, Chess is a traditional game in the formal gameplay category. Maintaining the angular, V-shaped move permitted by knight pieces is essential for the development of gameplay strategies. Players are not interested in empathizing with the knight, so switching its motion lines to an aesthetically gentle arc mid-game would be an intrusive featureâdiluting the objective of Chess and making the game too complex to play.
New Super Mario Bros. Wii (2009) by Nintendo. Original video by Sirloin.
The same is true of formal games like New Super Mario Bros. Wii (2009) by Nintendo (above), where Marioâs jump arcsâthe characterâs signature abilityâalign to the circle shape concept. Marioâs character controls must remain consistent, so that designers can structure gameplay in a predictable manner and ensure the gameâs intricate system of interactive possibilities doesnât break.
Changing the aesthetics of a characterâs interactive motion lines can have detrimental consequences for the game system. Characters from left: Mario by Nintendo, and Ninja Gaidenâs Ryu Hayabusa by Tecmo.
Dynamically altering Marioâs friendly, up-beat jump arcs to an aggressive and vengeful aesthetic would exponentially increase the complexity of balancing the gameâs systemâaffecting Marioâs ability to successfully navigate the environment. As in Chess, the playerâs mastery of New Super Mario Bros. Wiiâs game system is the experiential goal, not emotional empathy, and Nintendo does a better job of creating exciting challenges for players to master than most other developers.
Super Smash Bros. Wii U (2014) by Nintendo. Original video by WiiUGamer12.
If players wish to experience Mario with more aggressive, Vanquish-type movements they must play a game like Super Smash Bros. Wii U (2014), featured above. Combing the motion lines of New Super Mario Bros. Wii with Super Smash Bros. Wii U into one game would be a surefire way to create a broken or unmanageable game system from a designerâs perspective.
The desert âgameplay vignetteâ in Uncharted 3: Drakeâs Deception (2011), by Naughty Dog, allowed the gameâs designers to insert a control scheme with a different aesthetic to the core gameplay.
Despite their narrative inclinations and a definite wish to elicit player-character empathy, series like Uncharted and God of War are fundamentally formal games abiding to the same creative constraints as Chess and the Mario series. To create a sense of aesthetic variety, such games are often designed with binary character control schemes: a fast and responsive control scheme for combat, and another, more ponderous control scheme for puzzle-exploration sequences. These two schemes reflect interaction modesânot the playable characterâs emotional statesâand are conveyed pictorially (battle axe at the ready = combat mode). Character development is largely passed on to non-interactive cutscenes, and gameplay vignettesâlike the desert sequence in Uncharted 3, above, which is set aside from core gameplay.
Characters in series like God of War (2018), by Santa Monica Studio, feature single or binary control schemesâespecially evident in the transitions between battle and post-battle aesthetics. Original video by MKIceAndFire.
The single or binary character controls of such games are most noticeable at the tail end of combat where we would expect to find motion lines that reflect the emotional ups and downs of a real person. First, try to imagine what it would feel like to engage in life-threatening combat and the range of emotionsâfrom anger and heavy regret to euphoria and buoyant reliefâyouâd experience during and after battle. Now replay a combat sequence in a game like God of War (2018), applying the earlier Motion Lines Study, Part 1 exercise. Ignore automated actions that players trigger but donât have direct control over, like local animations for spins or attack moves. Where weâd expect to experience nuanced motion lines that realistically model the dynamic physical tension of a real person on a heroic quest we, instead, tend to find only single or binary aesthetic control schemes. The same is true of the Uncharted series, where Nathan Drake will plough through countless enemies and then instantly revert to a puzzle-exploration mode replete with sarcastic wit the moment a bloody battle has concluded.
Itâs important to stress that this is not developer oversight. Single and binary character control schemes simply highlight that emotional empathy is not the main goal of gameplay in such titles. Sophisticated combat systems, environment puzzles and levelling-up are central to the experience, which the developers achieve exceptionally well.
Gameplay vignettes like Uncharted 3âs desert sequence are classified under informal gameplay, which prioritizes freeform play or pure aesthetic experience. An incredible amount of work goes into concealing the intersections between formal and informal gameplay sequences of such titles. However, the aesthetic constraints dictated by formal game design principles, derived from pen-and-paper RPGs, create a misleading hurdle that prevents the motion lines during combat and puzzle-exploration from being explored with the same emotional range as informal gameplay and cutscenes. This is the essence of what Raph Koster was referring to by writing that âpsych tends not to be that amenable to math.â What is needed is the introduction of adaptive composition to complement the stat-heavy gameplay inherent in formal game designâwith shape language taking the role of the traditional gamemaster who verbally weaves an aesthetic accent in-between each roll of the playerâs dice.
âThe Decentâ in Journey, by that thatgamecompany, is a masterpiece of adaptive game design aesthetics, evoking motion lines with various âtextures.â Original video by GadgetGirlKylie.
Games that entirely forgo formal game design principles (often described as âinteractive artâ or âart gamesâ) have greater freedom to modulate interactive motion lines to evoke a wide spectrum of physical experiences and sophisticated in-game narratives. There is a middle ground, however, between formal and informal game design, which is illustrated by groundbreaking games like Journey, which utilize all the elements of adaptive composition to evoke a poetic sequence of physical gestures, as illustrated by âThe Decentâ gameplay sequence above. Compare the interactive motion lines in The Decent to the earlier God of War gameplay sequence. Which characterâJourneyâs traveler or God of Warâs Kratosâhas a wider range of interactive âtexturesâ and the feeling of emotional depth? Naturally, short play times make it easier for designers to create games composed entirely of unique gameplay vignettes with aesthetically varied motion linesâa feature that makes games like The Beginnerâs Guide (2015) and That Dragon, Cancer (2016) all the richer.
That Dragon, Cancer by Numinous Games is based on Amy and Ryan Greenâs autobiographical experience of raising their son, Joel, who was diagnosed with terminal cancer at the age of twelve months. As discussed during an IGDA event on Death and Consequence in Game Design, the developerâs of That Dragon, Cancer wanted to convey that such tragic situations can also contain moments of positivityâmuch like a tragicomedy that offers humor to offset an emotionally oppressive situation. For instance, thinking of a loved one can trigger the memory of a funny situation. The gameâs designer, Josh Larson, also highlighted that people deal with death in different ways. Players of That Dragon, Cancer find that Amy remains doggedly optimistic while Ryanâs tendency is to confront the factsâoften withdrawing into an empty space of contemplation.
That Dragon, Cancer (2016), by Numinous Games, presents players with a sequence of varied gameplay vignettes that explore every facet of dealing with deathâfrom moments of optimism and joy (left) to withdrawal and sorrow (right).
Conveying such multifaceted experience of death would not be possible if Josh and the team had focused on one core gameplay mechanicâas is typical in most video games. That Dragon, Cancer is therefore composed of a sequence of gameplay vignettes that explore the emotionally complex nature of losing a loved one. Fleeting moments of happiness, triumph and optimism are contrasted with moments of sorrow and reflection using gameplay mechanics to convey each aesthetic. For example, a Mario Kart-style driving sequence (above left) elicits joyous activity from players but is contrasted by moments of relative stillness when players are tasked with exploring a desolate hospital or find Ryan adrift in an emotional abyss (above right). Each gameplay vignette works in combination with the whole to create a poignant reflection on death.
Since motion lines can be modulated along the shape spectrum, we can consider implementing increasingly nuanced adaptive character control schemes that respond aesthetically to narrative context and player actions. Established visual feedback techniques, used to pictorially convey interaction modes, can be used with new emphasis on the character’s emotional state.
Shape sensitivity is critical for developing a sophisticated command of interactive motion linesâgamingâs unique artistic ingredientâas well as bridging the gap between formal and informal game design. The adaptive composition framework gives us conceptual tools to translate traditional composition to the interactive medium of games, where the player is an integral component of the artwork. Conceptualizing the adaptive composition framework in the form of Jesper Juulâs puzzle piece metaphor highlights gamingâs aesthetic connections to reality and art forms across transmedia, with shape languageâs sensory inputs serving to define the user experience. We saw that adaptive in-game music can also be aligned to the shape spectrum (see Exercise: Drawing Melodic Contours), and that interactive motion lines (see Exercise: Motion Lines Study, Part 1) are the backbone of interactive experiences, which enables players to physically empathize with life-like virtual characters and embody the video game artwork.
The aesthetic and functional appreciation of shape language and composition that weâve accumulated over the course of this essay gives us a solid foundation to tackle the ever-persistent debate: are games art?
With so much historic material to support gamingâs connection to art, itâs a wonder that the âare games art?â debate still continues. To help settle the argument it is worthwhile likening gameplay and interaction to Abstract Expressionismâa Modern Art movement that developed in the 1940s to explore spontaneous and automatic creative methods. Abstract expressionist figures like Jackson Pollock (1912-1956), shifted emphasis away from the art object (i.e. a painting) towards the artist and the physical aspects of creation.
In the below-left photo by Hans Namuth (1915-1990) you can see Pollock using his iconic drip painting technique that allowed the artist to make spontaneous creative decisions using gestures that wouldnât look out of place in a Nintendo Wii commercial. The accompanying painting from 1950 titled, Autumn Rhythm (Number 30), illustrates the results of Pollockâs techniqueâconsisting of lines of varying aesthetic quality that were a direct response to a myriad of spontaneous decisions, including the variable amount of paint on the brush, the flow of dripping paint, existing lines on the canvas, tension in the body, random thoughts, etc.
From left: Jackson Pollock photographed in the act of painting (c. 1950) by Hans Namuth, and the result of his drip painting technique in a painting titled, Autumn Rhythm (Number 30). (Image source: Wikipedia)
Aside from performing arts, this method of working was unprecedented for its pure visual depiction of the artistâs movements and creative process. This unity of aesthetics between the artist, the act of painting and the artwork gives us vital conceptual tools to address the games-art question. In a philosophical treatise on game art titled, On simulation, aesthetics and play: Artifactual Playground (2012), artist-philosopher and game designer, Douglas Stanley, presents a common line of argument against games as art, which contends that visuals in art and games are the result of two irreconcilable intents. Visual artworks like paintings are created solely to serve as aesthetic objects, while art in games is a byproduct of simulated metaphors designed to mimic real-life experiences. Take, for instance, the simulated metaphor of war that underlies a game of Chess, and the visual byproduct of gameplay: a Chess board with pieces arranged in various combinations.
Douglas goes on to compare the artwork of Piet Mondrian (1872-1944) and the arcade classic, QIX (1981), pictured belowâhighlighting their similarity in visual aesthetics but differing creative intents. The argument is that, unlike Mondrianâs painting, QIX is designed to simulate fast-paced competition over territory against a computer-controlled enemy. The visual similarities between Mondrianâs art style and QIX are therefore incidental because QIXâs visuals are the byproduct of a simulated metaphorânot a singular consideration for shape language, composition, color and value. Such conclusions are bewildering in a traditional art context because, it should go without saying, that metaphors can be used to mimic every aspect of humanity.
From left: Broadway Boogie Woogie (c. 1942) by Mondrian, and QUIX by Taito.
The simulated metaphor of Tennis for Two (1958)âthe first video game ever createdâis a faithful representation of tennis, despite the gameâs primitive graphics. So why not consider the possibility of a game that simulates an artistâs creative processâseeing that games champion active participation? Arenât all art forms an abstraction of reality that the artist must simulate? In the case of Mondrian this is certainly the case, since the artist received a traditional art training before evolving his iconic style using a process described as follows:
Tennis for Two (1958) by William Higinbotham.
Â âI want to come as close as possible to the truth [Nature] and abstract everything from that, until I reach the foundation (still just an external foundation!) of things.â
â Piet Mondrian
Resolving the conflict of intent between art and gameplay is easy if we imagine changing the gameplay of QIX, so that its simulated metaphor mimics Mondrianâs artistic methods. To do so we must appreciate that the relationship between artist (author) and the act of painting (agency) changes in the context of games because the game development team authors the game system but players act as agents to activate and perform the artwork. Once weâve established this relationship between developer-author and player-agent, we can better perceive the possibilities of creating gameplay that simulates the gestures of an artist.
From left: Mondrian in the process of creating Broadway Boogie Woogie in 1942 (image via Counterlight’s Peculiars blog; original author of photo unknown), and Spring Sun (c. 1910). (Image source: Wikipedia)
We know that Mondrianâs creative process was to cut out and position squares of construction paper that were replaced with considered and precise paint strokes once a pleasing visual composition had been arranged (above left). The artistâs creative decisions were driven by an impulse to distill the essence from earlier traditional paintings that focused on landscapes in a combination of naturalistic and impressionistic art styles (above right). Using the adaptive composition framework, we have the conceptual tools, and intent, to create a âMondrian simulationâ that will evoke Mondrian-like decision making and gestures from playersâthe foremost intention being to simulate Mondrian the artist, not a Mondrian painting. Such a simulation could reference the gentle territory gameplay of Flower (2009), by thatgamecompany, transferred to a linear aesthetic.
SPOILER ALERT: the following paragraph contains a potential spoiler for Gone Home (2013) by Fullbright.
Gone Home (2013) by Fullbright features gameplay that is transformed by the gameâs suspense-horror aesthetic to elicit emotions that the simulated metaphor alone could not elicit.
Is a Mondrian simulation more deserving of an art label than QIX because it simulates a real-life artist? Must simulated metaphors model the subject to have artistic value? For instance, Gone Home simulates a detective case but the gameâs use of suspense-horror cultural symbols evokes feelings of vulnerability that help players empathize with the social issues and teenage angst of the narrative. Can Gone Homeâs gameplay be labelled art even though the visuals and narrative donât match the simulated metaphor? Is a Mondrian simulation true to the artist only if the visual aesthetics accurately reproduce the artistâs paintings, or is evoking Mondrian-like physical gestures sufficient?
A really exciting discovery occurs when we compare the work of abstract expressionists to visualizations of player movement in video games, like the one above for Super Mario Bros. (1985). Movement tracking is generally used in game development for user testing purposes but we can appropriate the tools to study interactive motion lines in the context of simulated metaphors. What the examples below illustrate is the art of interactionâthe visual link between player gestures and the playerâs interaction with a game system that references Abstract Expressionism.
From left: visualizations of player movement in the âearlyâ game, and the âlateâ game, in League of Legends (2009) by Riot Games. (Images sourced via The New York Times)
The images above visualize the movements of 100,000 players in 10,000 games of League of Legends (2009), showing over six years of cumulative player time. The visualizations look similar to abstract expressionist art, wouldnât you agree? As the authors of Watch 10,000 League of Legends Games in 30 Seconds (The New York Times 2014) note: âwhile each match is complex, the same patterns emerge over time.â This is an extraordinary observation because it demonstrates the level of aesthetic control and consistency that developers have on the interactive experience of playersâvisually proving the value of our theoretical Mondrian simulation. It seems that, when viewed up-close, the playerâs movements are similar to Jackson Pollockâs line-work. When viewed afar, the playerâs collective movements reveal a concealed composition, which is not the result of a single author, but the collaborative effort of the gameâs developers and players.
Images from Ubisoftâs DNA tracking system used to visualize multiple players moving through Assassinâs Creed III (2012) by Ubisoft. (Images sourced via Gamasutra, Game Telemetry with DNA Tracking on Assassin’s Creed by Jonathan Dankoff)
Ubisoft has a custom movement tracking tool called DNA, which it uses to analyze player movement during play-testing prior to release as well as post-launch, as described in Game Telemetry with DNA Tracking on Assassin’s CreedÂ (Gamasutra 2014) by Jonathan Dankoff. In the interests of creating game systems that reference the subtler movements of an artistâs creative process, it would be an interesting challenge to iterate on such tools by developing ways to visualize the aesthetic qualities of movement and player gestures in finer detailâreferencing the Ballet Rotoscope video from Shape Language and Interactive Motion Lines and synaesthetic experiences like Panoramical by Fernando Ramallo and David Kanaga.
Columns from left: Helen Frankenthaler working in the studio (photograph sourced via Widewalls) and Around the Clock with Red (1983); Norman W. Lewis at work (photograph sourced via Pennsylvania Academy of the Fine Arts) and Composition 1 (1945); Kazuo Shiraga painting in the artistâs studio in 1960 (photograph via The Greenstein Lab, copyright Amagasaki Cultural Center, courtesy of Fergus McCaffrey) and Work (c. 1990).
Study the above images to appreciate the aesthetic connections between artist and artwork that abstract expressionists sought to establish. The mid-20th century onwards saw an increase in diversity, represented here by Helen Frankenthaler (1928-2011), Norman W. Lewis (1909-1979), and Kazuo Shiraga (1924-2008). For additional context, we know that Frankenthaler often painted with a technique called soak stain (paint that was heavily diluted with turpentine), which allowed the artist to achieve the soft, watercolor-like edges. Lewis painted in a fairly conventional mannerâsitting or standing at the easel with a paint brush in hand. Shiraga gained recognition creating mud paintings by using the entire body to make impressions in wet mudâa technique that was later adapted for oil paint.
Following on from the Motion Lines Study, Part 1 exercise, revisit some memorable gameplay narrative moments. Before playing, use the abstract expressionist analogy above to consider the emotional and physical state of your playable character and draw a corresponding line with an aesthetic quality that illustrates your concept. Once thatâs done, play the game sequence and consider whether your physical gestures echo what youâd expect the character to be feeling. Additionally, is there an aesthetic change to the motion lines across the gameplay sequence?
Player movement tracking in Rubberband Racing
by Max Knoblich. (Image sourced via Gamasutra)
It is safe to conclude that a simulated metaphor (gameplay) is indeed artâart that is modelled on the creative process and authored by a development team before being activated by players. That the lines visualized by tracking tools are not generally visualized in-game (aside from games like QIX, the characterâs trail in ABZĂ or the trailing scarf in Journey) is not important, since classical compositions were also purposefully concealed (see Concealing the Composition). The question is only whether game developers care more about formal gameplay (rules, objectives and stats) or informal gameplay experiences and interactive motion lines that they evoke, which will determine whether a simulation’s output is treated as an “incidental byproduct” or the deliberate aesthetic result of a carefully orchestrated adaptive composition.
By using shape language and composition principles, combined with abstract expressionist concepts, it is clear that an appreciation of games as art requires going beyond surface representation. Instead, we must take-on the classic role of the artist-craftsperson to appreciate the creative process itself.
The more we understand about shape language and composition, and the sensory connections between visuals, music and and physical gestures, the greater the harmony will emerge between the component parts of our games.
The adaptive composition framework.
The above sections on Shape Language for Game Design and The Adaptive Composition Framework, offered just a brief overview to four of the eight elements of adaptive composition, illustrated above: character shapes and poses, environment shapes, audio and interactive movements lines. There is a lot more to learn about each element thatâtogether with shape language and the shape spectrumâgives us greater control of the game development process and orchestration of aesthetics that respond dynamically to gameplay and narrative context. These principles are timeless, applicable to VR, AR, XR and all future expressive media. The high-level framework nicely complements or enhances other popular game design methodologies like Jesse Schell’s “lenses” [see The Art of Game Design: A Book of Lenses (CRC Press 2008)]. Equally, designers will find that existing game mechanics and system designs remain valid and mostly unchanged. What the framework does is give these mechanics and systems an aesthetics purposeâchampioning the big picture over details.
The adaptive composition framework also helps to demonstrate game designâs artistic value. Not the conventional notion of game art as a visual âbyproductâ of gameplay but of gameplay as artâthe rules, mechanics, etc.âand its connections to traditional art principles.
Roman statue of two girls playing Knucklebones (3rd century BC)
framed within a circular composition inspired by ancient Greek sculpture.
The ancient Greek philosophers and Islamic artists were among the first to begin ascribing aesthetic value to primary shapes, influenced by their advanced mathematical knowledge that resulted in the Platonic solids (see Part 1), which form the conceptual basis of Greco-Roman and Islamic art. This pioneering work linking math and aesthetics went on to influence ancient Buddhist art, which arguably holds the most relevance to game design through the concept of embodiment. These concepts, inherited from ancient art, allowed us to conclude that âpsychâ is indeed amenable to math as experienced through exercises like, Motion Lines Study, Part 1). The fundamental animation concepts of hard vs. curved, and opposing curvesÂ (see Part 1) used in contemporary animation are also a legacy of the ancient Greek philosophers. Even âludoâ (play) has its origins in antiquity. With shape language and composition being such a central topic across all art forms, why arenât we talking about it in the context of game design?
Duchampâs teachings hold some answers, since the artist-philosopher has had a tremendous influence on contemporary art thinking. Youâll now find it hard to receive a formal art education, in a classic âacademyâ sense, outside of a handful of traditional academies and specialist art courses that focus on production techniques for illustration, film, animation, and game art, etc., where visual storytelling is still the focus. Itâs here that the importance of shape language and composition endures, attended by celebrated artists like Nathan Fowkes, whose clients include DreamWorks, Blue Sky and Disney. In a Schoolism.com interview with Bobby Chiu, Nathan uses Mondrian as an example to stress the importance of composition as follows:
Â âWhatâs the job of our brain? Itâs to take all of the noise that we see and experience, and draw out from it the meaningful relationships [âŠ], wash away everything else. Thatâs being a human being. Thatâs basic survival. And thatâs exactly what [Mondrian] did in his work. He distilled all of the noise away until the only thing left were pure, simple, meaningful relationships. [âŠ] No matter what we do, that is our job, to find the meaningful relationships that makes those pictures really hold together with the greater sense of purpose.â
â Nathan Fowkes
Bicycle Wheel by Marcel Duchamp, turned upside-down, as a commentary on the persistence of shape language: the dynamic circle, stable square and slicing triangle.
The truth is that the games industry at large would benefit if everybody along the development pipeline had an understanding and appreciation for traditional art principles. Shape languageâs basis in Nature means that it is understood across culturesânot to mention, every individual on the development team. Through shape language, which weâve found embedded across engineering, art and design fields, weâve seen that primary shapesâthe circle, square and triangleâsupport Raph Kosterâs definition of games as âsystems built to help us learn patterns for survival.â
Judging from the international examples of shape language and composition in creative disciplines as diverse as film, comics, photography, and interior design, it appears that coded survival lessons are a crucial component of all art and storytelling mediums as well as play. In fact, its scientifically believed that storytelling could be an evolutionary mechanism that helped keep our ancestors alive. The existential purpose of stories evokes a comforting sense of control over the worldâteaching audiences to find patterns in chaos and meaning in randomness. This is perhaps the best definition for shape language and compositionâaligning perfectly with Raphâs definition of games.
Take a moment to look back through your drawings for each exercise in this essay to appreciate the aesthetic differences that can be achieved despite each pencil stroke aligning to one of the primary shapes. This is the kind of line quality an artist expresses in a life drawing class. Through shape language and adaptive composition, we can re-visit traditional dramatic structures with a fresh focus on physical empathy and embodiment, which is the topic of the second half of Interactive Stories and Video Game Art.
The benefits of using shape language include:
The utilitarian basis of shape language takes on high-level emotional themes when placed in an artistic context. Irrespective of an objectâs cultural meaning, aligning its broader shape to the circle, square or triangle will consequently align it to one of the following aesthetic themes:
Circle: movement, energy, grace, friendliness, continuity
Square: balance, strength, restraint, tranquility, rest
Triangle: force, instability, aggression, pain, agitation, tension
The benefits of composition include visual simplification, visual direction, story structure and a holistic design framework. We inherit all these benefits when we adapt composition to game designâa straightforward process that requires the acknowledgement of players within the composition structure, and that each supporting element is pervasive, dynamic (adapting to the narrative or gameplay context) and visually broad (so as to remain visible at all zoom levels and from every perspective). These are not superficial considerations exclusive to game art. Each element of adaptive composition involves close collaboration between all the various disciplines involved in game developmentâsuggesting new possibilities for advanced adaptive and procedural storytelling techniques that shape the playerâs aesthetic experience on the fly.
By conceptualizing the adaptive composition framework in the form of Jesper Juulâs puzzle piece metaphor, we can also reflect on the definition of games, their boundaries, and how they interface with reality and across transmedia experiencesâsomething that would not be possible without the elegantly simple affordances of shape language versus an infinitely complex Duchamp-ist universe. However, shape language and culturally learned symbols are not mutually exclusive because the former works best when used to covertly influence how audiences experience the latter.
Using these techniques doesn’t ensure a masterpieceâin the same way that understanding music notation doesn’t turn a novice into Mozartâbut it does make the game development process feel more like tuning a familiar instrument with subtle controls that affect the player-audience. These collective insights enable game designers to generate subtler forms of dramatic tension that donât rely on escalation of gameplay difficulty, which is the wider industryâs go-to for orchestrating in-game narrative experiences. Leading design by physical gestures will no doubt blend the distinctions between formal and informal game design and dissolve the predominance of single and binary character control schemes, which lead to an interactive form of uncanny valley in high-end AAA games.
It just so happens that the first ever video game, Tennis for Two, simulated a sports activity. This does not mean that all games must be decedents of sports and competition. We can equally think of games as decedents of the artistâs creative process in sculpture and painting without denying the value of objective-driven gameplay. Animation went through a similar path of self-discovery by overcoming supposed limitations set by the mediumâs early pioneers and critics that couldnât imagine the medium useful for anything other than comic gags, as explained by Walt Disney (1901-1966):
âAt first the cartoon medium was just a novelty, but it never really began to hit until we had more than tricks. You have to have pathos [empathy] in the thing.â
â Walt Disney
Replace âcartoonâ with âvideo game,â and âtricksâ with âincidental byproductsâ and you have a fairly good idea of the untapped possibilities that weâve yet to discover using art-driven frameworks and the exponential increase of subject matter that they afford.
Peanuts comic strip (1960) by Charles M. Schulz.
The above Peanuts comic strip perfectly summarizes what makes video games such a unique art form. Charlie Brown explains that an appropriate stance is vital to getting the most out of feeling depressed: avoid straightening up and holding your head high because that will inadvertently make you feel better. By combining game design with shape language and adaptive composition techniques, we have the design tools to heighten sensory inputs to influence player gestures and, as a result, player emotions in increasingly subtle ways that surpass any art form that has come before.
Interactive Stories and Video Game Art: A Storytelling Framework for Game Design is available direct from CRC Press or via Amazon and Barnes & Noble. For updates about future essays and work, youâre welcome to follow me on Gumroad, Twitter, Facebook and Instagram or via my homepage.
A special thank-you to Julia Ayumi Bopp (MMI Human-Machine Interaction at the University of Basel, Switzerland), Martina Hugentobler, Samuel Smith (ProSapien), Corrinne Lewis (University of Utah), Kayleigh MacLeod, Stefano Maccarinelli (Stelex Software), Edith Solarski-Meier, Henri Suhonen, friends, family, and #swissgames organizations like Pro Helvetia and SAE Zurich that kindly supported my work throughout the years. Iâd also like to thank Alastair Ingason, Lalima Singh, David Javet (University of Lausanne GameLab and Tchagata Games), David Canela (MODSORK), Robert Zubek (SomaSim), and Amit Goyal (SuperSike Games) for giving me valuable feedback on sections of this essay, and Benjamin Poole for directing me to Raph Koster’s work.
Chris Solarski is an artist-game designer and author living in Zurich, Switzerland. His two books are widely considered essential reading for anybody looking for holistic design tools for orchestrating the emotional and physical experience of players in games and transmedia. Chris’ first bookâDrawing Basics and Video Game Artâhas been translated into Japanese and Korean. His second bookâInteractive Stories and Video Game Artâhas been described as gaming’s equivalent to the screenwriting classic, Story, by Robert McKee and is endorsed by film director, Marc Forster. With kind support from the Swiss Arts Council, Pro Helvetia, Chris has given talks on interactive storytelling and game art internationally, including Disney Research, the Smithsonian’s The Art of Video Games exhibition, SXSW, GDC Europe, and FMX. Chris is currently collaborating with internationally renowned artist, Phil Hale, to develop an indie game based on the Johnny Badhair series of paintings. He also lectures at SAE Institute in Zurich and manages the IGDA Switzerland Chapter and Co-Chairs the IGDA Game Art SIG, which he cofounded with Natasha Bulatovic Trygg.