Package 'multimolang'

dfMaker Function

Description

dfMaker() processes and organizes keypoints data generated by 'OpenPose' (Cao et al., 2019), compiling multiple JSON files into a structured data frame. It applies linear transformations to align and scale the keypoints within a custom coordinate system defined by the user. The function supports datasets containing pose keypoints alone or combined with face and hand keypoints.

Usage

dfMaker(
  input.folder,
  config.path,
  output.file = NULL,
  output.path = NULL,
  no_save = FALSE,
  fast_scaling = TRUE,
  transformation_coords = c(1, 1, 5, 5)
)

Arguments

input.folder	Path to the folder containing 'OpenPose' JSON files. The folder should contain only the JSON files to be processed; including other files may lead to unexpected behavior.
config.path	Path to the configuration file used for extracting metadata from filenames, particularly when processing data from the UCLA NewsScape archive. If not provided, default settings are used, which may not extract NewsScape-specific metadata.
output.file	Name of the output file. If NULL and there is only one unique id in the data, an auto-generated name with a .parquet extension is used. If multiple unique id values are present and output.file is NULL, the function will return an error prompting you to provide an explicit output file name. If the specified output.file ends with .csv, the output will be saved in CSV format; otherwise, the default format is .parquet.
output.path	Path to save the output file. If NULL, the file is saved in a default directory called df_outputs in the current working directory.
no_save	Logical. If TRUE, the output is not saved to a file.
fast_scaling	Logical. If TRUE, uses a simplified scaling transformation that utilizes only the primary base vector v.i, ignoring the secondary base vector v.j. Warning: When fast_scaling is TRUE, scaling is performed only using pose keypoints (t_typepoint = 1), and the secondary vector v.j is not utilized.
transformation_coords	Numeric vector of length 4 specifying the transformation parameters: t_typepoint Type of keypoints to use for transformation. Possible values: 1: Body keypoints (pose). 2: Face keypoints. 3: Left hand keypoints. 4: Right hand keypoints. o_point Index of the keypoint used as the origin in the new coordinate system. i_point Index of the keypoint that defines the primary base vector v.i. j_point Index of the keypoint that defines the secondary base vector v.j. If i_point == j_point, v.j is calculated as a perpendicular vector to v.i (orthonormalization). If fast_scaling = TRUE, v.j is not utilized and should be set to NA.

Details

The dfMaker() function processes keypoints data generated by 'OpenPose', applying linear transformations to align and scale the keypoints within a custom coordinate system defined by the user. The transformation is specified by the transformation_coords parameter, which allows you to select specific keypoints to define the origin and the base vectors for the new coordinate system.

When fast_scaling = FALSE, the function uses two base vectors (v.i and v.j) defined by the keypoints specified in i_point and j_point to perform an affine transformation. If i_point == j_point, v.j is calculated as a perpendicular vector to v.i, resulting in an orthonormal basis.

When fast_scaling = TRUE, the transformation uses only the primary base vector v.i, and the scaling is simplified. The secondary vector v.j is not utilized, and j_point should be set to NA.

This function depends on the arrow package for efficient reading and writing of JSON and Parquet files. Ensure that the arrow package is installed.

The function expects JSON files generated by 'OpenPose' (tested with version 1.7.0) with a specific structure. Variations in the 'OpenPose' configuration or version may affect the format of these files. Ensure that the JSON files conform to the expected structure for accurate processing.

Each row in the output data frame represents a single keypoint detected in a specific frame and associated with a specific person. The columns nx and ny provide the transformed coordinates based on the selected origin and linear transformation parameters. The id column links the keypoints to the corresponding input file from which they were extracted.

The data frame may contain missing values (NA) for keypoints that could not be reliably detected.

Value

A data frame containing the processed keypoints data with the following columns:

id

Character: Identifier derived from the name of the file processed.

frame

Numeric: Frame number from which the data is extracted.

people_id

Integer: Identifier for each detected person in the frame.

type_points

Character: Type of keypoints (e.g., "pose_keypoints", "face_keypoints", "hand_left_keypoints", "hand_right_keypoints").

points

Integer: Index of the keypoints sequence.

x

Numeric: X-coordinate of the keypoint.

y

Numeric: Y-coordinate of the keypoint.

c

Numeric: Confidence score for the detected keypoint, ranging from 0 to 1.

nx

Numeric: Transformed X-coordinate in the new coordinate system.

ny

Numeric: Transformed Y-coordinate in the new coordinate system.

Additional metadata columns may be included if extracted based on the configuration, such as datetime, exp_search, country_code, network_code, program_name, and time_range.

R Version Requirements

This function uses the native pipe operator ⁠|>⁠ introduced in R version 4.1.0. Therefore, R version 4.1.0 or higher is required.

Error Handling

If the JSON files do not have the expected format or are empty, the function will skip these files and print a message indicating the issue. If output.file is NULL and multiple unique id values are found, the function will stop and return an error, prompting you to provide an explicit output.file name.

Note

When processing data from the UCLA NewsScape archive, the config.path parameter allows you to specify a configuration file that defines how to extract metadata from the filenames of the videos. The filenames in this archive have specific structures containing metadata such as date, time, country code, network code, program name, and time range.

Example of Configuration File (config.json):

{
    "extract_datetime": true,
    "extract_time": true,
    "extract_exp_search": true,
    "extract_country_code": true,
    "extract_network_code": true,
    "extract_program_name": true,
    "extract_time_range": true,
    "timezone": "America/Los_Angeles"
}

Ensure that your data filenames follow the standard NewsScape naming convention for accurate metadata extraction. If your data does not conform to this naming convention, you may need to adjust your filenames or modify the configuration accordingly.

The dfMaker() function processes all keypoints provided by 'OpenPose', including pose, face, and hand keypoints. For the specific indices and descriptions of these keypoints, please refer to the 'OpenPose' documentation.

References

For more information about the UCLA NewsScape archive, visit: https://bigdatasocialscience.ucla.edu/newsscape/

For a detailed description of the NewsScape infrastructure and its applications, refer to: Uhrig, P. (2018). "NewsScape and the Distributed Little Red Hen Lab: A Digital Infrastructure for the Large-Scale Analysis of TV Broadcasts." In Anglistentag 2017 in Regensburg: Proceedings, pp. 99–114.

The arrow R package is used for efficient reading and writing of JSON and Parquet files. For more information, visit: https://cran.r-project.org/package=arrow

'OpenPose' GitHub Repository: https://github.com/CMU-Perceptual-Computing-Lab/openpose

Examples

# Example 1: Define paths to example data included with the package
input.folder <- system.file("extdata/eg/o1", package = "multimolang")
output.file <- file.path(tempdir(), "processed_data.csv")
output.path <- tempdir()  # Use a temporary directory for writing output

# Run dfMaker() with example data
df <- dfMaker(
  input.folder = input.folder,
  output.file = output.file,
  output.path = output.path,
  no_save = FALSE,
  fast_scaling = TRUE,
  transformation_coords = c(1, 1, 5, 5)
)

# View the first few rows of the resulting data frame
head(df)

# Example 2: Using NewsScape data with a custom configuration file
# Define the configuration file path
config.path <- system.file("extdata/config_all_true.json", package = "multimolang")

# Run dfMaker with custom configuration
df <- dfMaker(
  input.folder = input.folder,
  config.path = config.path,
  output.file = output.file,
  output.path = output.path,
  no_save = FALSE,
  fast_scaling = TRUE,
  transformation_coords = c(1, 1, 5, 5)
)

# View the first few rows
head(df)

---