--- title: "Getting started with replayr" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Getting started with replayr} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 4) ``` Emulators, game engines and research data sets record the input of a game as it is played: tool-assisted speedruns are written as movie files, *Doom* demos store every tic of a speedrun, and data sets for learning from demonstration keep the actions of human players. Each recorder has its own format. replayr reads them into the same R objects, so that the input of an NES movie, a *Doom* demo and a Wii recording can be inspected and compared in the same way. This guide reads a recording, looks at the input of one player, identifies its controls, and finds the time of every observation; it ends with numeric channels such as sticks and mice, recordings of several players, and the formats that replayr reads. ```{r setup} library("replayr") ``` ## Reading a recording `read_movie()` reads a recording and chooses the reader from the file's extension. replayr comes with small original recordings; this one holds a minute of artificial NES input for two controllers: ```{r read} path <- system.file("extdata", "sample.fm2", package = "replayr") movie <- read_movie(path) movie ``` A recording read this way is an object of class `replayr_movie`. It holds the input of every player as a logical array of observations, players and controls, together with the header of the file and the layout of its controls. `n_observations()`, `n_players()` and `n_buttons()` give its dimensions, and `summary()` describes the input of each player: ```{r summary} summary(movie) ``` The summary counts, for each control, the observations in which it is held and the times it is pressed, so that one long hold and many short presses can be told apart. ## One player's input `player()` returns the input of one player as a logical matrix, one row per observation and one column per control. The columns are named by the control's key, so `input[, "a"]` is the A button: ```{r player} input <- player(movie, 1) dim(input) head(input[rowSums(input) > 0, ], 3) ``` `press_counts()` counts the observations in which each control is held. Divided by the number of observations, the counts give the share of the recording for which each control is held: ```{r press-rate} shares <- press_counts(movie, 1) / n_observations(movie) barplot(shares, las = 2, ylab = "share of observations held", main = "Player 1") ``` A recording can also be cut: `movie[1:600]` keeps the first 600 observations and is again a recording, on which every function works, and `as.data.frame()` gives one row per observation with the controls and numeric channels of a player. ## Control identities The columns of a recording are compact: they hold the controls of its device and no others. `button_layout()` says which control each column is, with a key, a label as on the device, and an ID from a shared registry: ```{r layout} button_layout(movie) ``` The IDs make controls comparable across recorders. The same rules decide in every reader which controls share an ID, so a console's buttons have the same IDs whichever emulator recorded them. A button that carries a letter or word of the registry, such as A, B, Start or Select, has that ID, wherever it sits on the pad; PlayStation buttons, which carry shapes, have the IDs of the SNES buttons in the same places; numbered buttons follow a fixed order. The compatibility guide, `vignette("compatibility", package = "replayr")`, lists every rule and every device. Controls of other devices take the ID of the registry control in the same role. A light gun's trigger, for instance, has the ID of the A button and its offscreen shot that of Y, so that a study can compare it with a pad; `aliases = FALSE` keeps the device's own controls with qualified IDs instead. When several controls of a recording share an ID, their column is active when any of them is pressed. ```{r aliases} gun <- system.file("extdata", "paper", "light-gun.bk2", package = "replayr") button_layout(read_movie(gun))[, c("id", "key", "name")] button_layout(read_movie(gun, aliases = FALSE))[, c("id", "key", "name")] ``` Recordings of different devices have different columns. `align_buttons()` puts a list of recordings into matrices with the same columns, filling the controls that a recording lacks with `FALSE`. A shared ID says that two buttons are the same control, not that they do the same thing in a game; what a button means remains a question for the analysis. Some recordings store what the game was asked to do rather than which button was pressed: a *Doom* demo records attacking and using, whatever keys the player had bound. Such controls have the kind `action` and IDs of their own, such as `action:attack`. ## Time An observation is a video frame in most console recordings, a tic of the game in a *Doom* demo, an event in some computer recordings, and a poll of the controller in others. `observation_unit()` names it, and `recording_info()` reports the platform, the video standard, the frame rate and the number of rerecords under one name each, whatever the format calls them: ```{r info} observation_unit(movie) recording_info(movie) range(observation_time(movie)) ``` `observation_time()` gives the time of every observation in seconds. It takes the times stored in the recording where there are any, as in libTAS, JPC-RR and openMSX recordings, and otherwise the frame rate of the console and its video standard; `frame_rate_source` says which. Where neither is known, no rate is assumed: `observation_time(movie, fallback_rate = 60)` names one, and controller polls, whose number per frame varies, have no time. ## Sticks, mice and other numeric channels Not every input is a button. `analog_channels()` returns the numeric channels of a player, such as the axes of sticks, the movement of a mouse, touch positions or motion sensors, in the units of the recording. Their columns are named by key, so the X axis of a left stick is `left_x` in every format; `channel_layout()` lists them beside the recording's own labels. In this artificial recording of a Saturn mouse, the left button is held while the mouse moves: ```{r mouse} mouse <- read_movie(system.file("extdata", "paper", "mouse-drag.bk2", package = "replayr")) channel_layout(mouse) held <- player(mouse)[, "a"] head(analog_channels(mouse)[held, ], 3) ``` The common keys are: | Key | Measurement | |---|---| | `left_x`, `left_y`, `right_x`, `right_y` | Axes of the analogue sticks; the N64 stick, the 3DS Circle Pad and the Nunchuk stick are the left stick | | `left_trigger`, `right_trigger` | Analogue triggers | | `mouse_dx`, `mouse_dy` | Movement of a mouse since the previous observation (`mouse_x`, `mouse_y` are positions) | | `touch_x`, `touch_y` | Position on a touch surface | | `aim_x`, `aim_y` | Where a light gun points | | `accel_x`, `accel_y`, `accel_z`; `gyro_x`, `gyro_y`, `gyro_z` | Motion sensors | A stick can also become directions. `read_movie(analog = "cardinal8")` adds eight direction columns per stick to the buttons, `"cardinal4"` four, and `"angle"` and `"angle+magnitude"` add the stick's direction in degrees and its distance from the centre to `stick_channels()`. A pair such as `c(left = "angle+magnitude", right = "cardinal8")` treats the two sticks differently, and `deadzone` sets how far a stick must move before it counts: ```{r sticks} dtm <- system.file("extdata", "sample.dtm", package = "replayr") gc <- read_movie(dtm, analog = c(left = "angle+magnitude", right = "cardinal8")) head(stick_channels(gc), 3) grep("^right_", colnames(player(gc)), value = TRUE) ``` Numeric values keep the range and unit of their source; converting the counts of a Wii Remote's motion sensors into physical units, for instance, needs the device's calibration. The help page of `read_movie()` gives the defaults and their reasons. ## Several players and unassigned polls A recording can hold several players, and `player(movie, 2)` returns the second. Some formats, such as GameCube recordings with several controllers, store a sequence of controller polls without saying which port each poll came from. replayr then returns an object of class `replayr_poll_stream` instead of a movie, and `polls()` lists the polls with their port left unknown: ```{r polls} polls_path <- system.file("extdata", "paper", "unassigned.dtm", package = "replayr") head(polls(read_movie(polls_path)), 3) ``` When the order in which the controllers were polled is known, the argument `controller_order` assigns the polls to players. ## Formats and files replayr reads 33 recording formats under 37 file extensions. `supported_formats()` lists them; with `details = TRUE` it names the reader and the emulator of each, and the platform where a format serves one console: ```{r formats} head(supported_formats(details = TRUE)) ``` The extension of a file decides its reader. A recording under another extension is recognised by its content where its format has a signature, as most binary formats do, and `recording_format()` says which files of a folder are recordings. A format without a signature, such as a *Doom* demo, is named with `read_movie(path, format = "lmp")`. A download that bundles several recordings in a ZIP archive is listed with `movie_members()`, and one of its recordings read with `read_movie(path, member = )`. Very long BizHawk recordings can be processed in parts with `read_bk2_chunks()`, which hands each part to a function of the analyst's. ## Further guides - `vignette("compatibility", package = "replayr")` lists the platforms, formats, versions and devices that replayr reads, the rules for the IDs of controls, and the recordings used to check them. - The pisa package analyses the input that replayr reads: it maps controls to the actions of a game and compares players with the methods of stylometry.