Auto-skinning videogames

An idea: a system to automatically reskin videogames; that is, to remap the sprites and real-world references of a videogame so that it's "about" something else.

One approach is to think of an author defining spaces of games in terms of constraints on their real-world references. A space of games is, for example, "a game where an attacker chases a target and the target tries to get away". The author specifies the nouns and verbs in the game, and defines constraints over real-world referents using the ConceptNet and WordNet databases of common-sense knowledge. An individual game in that space is made by choosing specific nouns and verbs in the space that fit the constraints.

We demonstrate some examples of sensible game spaces that can be mapped on to micro-games in the style of Nintendo's WarioWare series. These are small games, typically lasting a few seconds, that involve one simple element of gameplay such as "dodge the car". We choose this domain as an example because its gameplay is simple enough to allow a focus specifically on the real-world references rather than the game mechanics, yet the space of possibilities is rich enough for the series to be interesting to game designers.

Game spaces and stock mechanics

The three main components of our system are game spaces, which define a space of possible games such as a game where one entity tries to avoid another entity; stock mechanics, which are literal game code with templated slots; and mapping rules, which specify how a game space should map onto a stock mechanic to render a specific game within the space playable.

The game spaces are primarily thematic, but also include some assumptions about how the mechanics would operate. For example, a game with one entity avoiding another has two entities (the avoider and the attacker), which must be chosen so that it "makes sense", in terms of common-sense knowledge, that one would be avoiding the other. This thematic content comes with an assumption that the game itself, however implemented, will involve a mechanic where the avoider tries to get away, and wins if successful, or loses if caught.

The stock mechanics are bundles of game mechanics that specify a fully playable game, but with templated slots for some entities in the game. For example, one of our WarioWare-style mechanics (which we call Dodger) has one object moving across the screen, while another object that the player controls moves up or down quickly to get out of its way.

For example, here are two mappings of a game from the attacker-avoidance game space in which a duck avoids a bullet:

The mapping rules ensure that the mechanic assumptions that are implicitly part of a game space are respected, by mapping the game space onto stock mechanics in such a way that the common-sense constraints are preserved. In the above example, a game about a duck avoiding a bullet is mapped onto two different stock mechanics. In the game shown on the left, the avoider (a duck) is mapped onto an avatar that the player moves up and down, while the attacker (a bullet) is mapped onto a computer-controlled sprite that moves left across the screen towards the player. In the game shown on the right, the avoider (a duck again) is mapped onto a computer-controlled sprite that moves randomly, and the player controls crosshairs to play the role of the attacker (a bullet again), which isn't explicitly shown. These examples of mappings are straightforward one-to-one mappings, but more complex mappings can also be constructed, where for example animate and inanimate objects map differently, and other parts of the game are changed based on properties of the specific terms being filled into the slots.

In our prototype we've so far defined three game spaces that map onto five sets of stock mechanics. The three game spaces are an attacker-avoidance game, a reservoir- or meter-filling game, and an acquisition game. The five stock mechanics are Dodger, which shows a side view in which a player dodges an incoming object; Shooter, which shows a target crosshair the player uses to shoot an object; Pick-Up, in which the player runs through a maze to pick up an item; Pump, in which the player button-mashes to raise a meter; and Move, in which a player button-mashes to move across the screen.

Constraint-based game skinning

The author in our system defines a game space by specifying variables, which are marked as nouns or verbs, and constraints on the variables. The constraints are either on the values that individual variables can take, or on how the values of multiple variables must relate to each other. The author may specify the constraints using relations in the ConceptNet and WordNet databases, as well as some combinations of the two, and logical operators like and and or.

ConceptNet is a graph-structured common-sense knowledge base extracted from OpenMind, a collection of semi-structured English sentences expressing common-sense facts gathered from online volunteers. ConceptNet's nodes are English words or phrases, and its links express semantic relationships such as (CapableOf "person" "play video game"). For our purposes, CapableOf, CapableOfReceivingAction, PropertyOf, and UsedFor are the most useful relations.

WordNet is a hierarchical dictionary of English words. A word below another one in the hierarchy is a specialization of the higher-up one (the higher word is a "hypernym" and the lower one is a "hyponym" if a noun, or "troponym" if a verb). This can be used to constrain variables in a game space to specific types of words; for example, constraining a noun to be a hyponym of "animate object" makes sure that inanimate objects aren't put into the slot of a stock mechanic where they wouldn't make sense. More generally, WordNet allows us to perform taxonomic generalizations over relations defined in ConceptNet.

A serious ConceptNet drawback is weak coverage: it knows that a duck can be shot, but not that a pheasant can be shot, for example. Fortunately, combining queries to ConceptNet with hierarchical information from WordNet mitigates this problem to a large extent. In specifying a particular ConceptNet constraint, the author can specify whether WordNet "inheritance" should be performed on any of the terms or variables in either direction (towards more general or towards more specific words). For example, the query about whether a pheasant can be shot should have hypernym (towards more general terms) inheritance enabled on "pheasant" and would therefore return true, because from WordNet we find that a pheasant is a type of animal, and ConceptNet knows that animals can be shot.

The attacker-avoidance game space

Here is the set of variables and constraints specifying the space of "avoid" games, where one noun avoids another noun:

noun Avoider
noun Attacker
verb Attack_verb: shoot, attack, damage, chase, injure, hit
constraint: (ConceptNet CapableOfReceivingAction ?Avoider ?Attack_verb)
constraint: (WordNet hyponym ?Avoider "animate thing")
constraint: (or (and (WordNet hyponym ?Attacker "projectile")
                     (ConceptNet CapableOfReceivingAction ?Attacker ?Attack_verb))
                (ConceptNet CapableOf ?Attacker ?Attack_verb))

A graphical view of the same space:

The game space is defined by three variables—two nouns and a verb—and several constraints between them. The nouns can implicitly range over any noun for which the system has graphics, subject to the constraints. The verb range is specified by the author, in the form of five verbs chosen as representative of the type of action to take place in the game. The constraints specify how to bind these variables to specific terms from ConceptNet so as to maintain the common-sense semantics of "attack" and "avoid".

The first constraint specifies that the Avoider has to be capable of serving as the direct object of the Attack_verb (represented by the CapableOfReceivingAction relation in ConceptNet); hypernym inheritance is done on the Avoider (not shown in the figure for simplicity). The second constrains the Avoider to being an "animate thing" since it must move to avoid the Attacker. Some inanimate things could sensibly function as Avoiders, especially humorously (a piece of bread trying to avoid a toaster, say), but specifying which of them makes sense gets trickier, so in this example we as the authors have simply decided to limit the game space to consider only animate things as Avoiders.

The third constraint is somewhat more complex. The most obvious constraint to add is that the Attacker must be CapableOf the Attack_verb (with hypernym inheritance on the Attacker). After trying this, however, it turns out to preclude many games that we think of as canonical in the avoider game design space. Something trying to avoid a bullet, for example, is excluded because a bullet isn't itself CapableOf "shoot": A bullet doesn't shoot, but is shot, and therefore isn't CapableOf, but rather CapableOfReceivingAction shoot. We informally expect it to serve in the role of attacker and perhaps anthropomorphize it as "chasing" the Avoider, but those notions are too subtle for the common-sense databases that currently exist to capture. To take these cases into account, we've added an alternate possibility for fulfilling the last constraint: If an Attacker is a projectile (according to WordNet), then we check whether the Attack_verb can sensibly act on it, rather than whether it can sensibly act out the Attack_verb.

* * *

This just scratches the surface of the problem. Given the vast size yet lack of completeness of common-sense knowledge bases, a game space defined by a given set of constraints will often produce counterintuitive results, including concrete games that the author doesn't expect, while excluding games the author might think are canonical in the game space. Thus we've also built an interactive editor to help authors understand and modify these game spaces (described in the much longer research-paper version of this note).

Future research can go in many directions: removing the "stock" in the stock mechanics by making them modifiable as well (rather than a pure reskinning system), including a database of common game-design tropes that can be filtered for applicability to the current partial design and suggested to the author, thus helping more directly with the how-to-represent problem, and pushing dynamic generation into the deployed game so that it can be customized by or react intelligently to a specific player.