GitHub - daedalusLAB/multimolang: Multimodal Language Analysis Tools (original) (raw)

multimolang: Multimodal Language Analysis Tools

multimolang is a package designed to support multimodal data analysis for linguistic research. It serves as the main repository for the MULTIFLOW project and provides tools to process, analyze, and derive insights from multimodal datasets, starting with gesture trajectories extracted from video data.

The first implemented tool, dfMaker, processes raw OpenPose data into structured dataframes, optimized for large-scale, multimodal data science workflows. Future versions of multimolang will include additional tools for prosody analysis and multimodal language integration.


Installation

Install from CRAN

install.packages("multimolang")

Or install using devtools

devtools::install_github("daedalusLAB/multimolang")

For detailed installation instructions and usage examples, please refer to the package vignettes.


Functions

Currently, the main function in the package is dfMaker, which provides:


Example Usage

Process OpenPose Data

Define paths to example data included with the package

input.folder <- system.file("extdata", "ex_videos", "o1", package = "multimolang") output.file <- file.path(tempdir(), "processed_data.csv") output.path <- tempdir()

Run dfMaker with example data

df <- dfMaker( input.folder = input.folder, output.file = output.file, output.path = output.path, no_save = FALSE, fast_scaling = TRUE, transformation_coords = c(1, 1, 5, 5) )

View the resulting data

head(df)


Development Status

Below is the current progress and pending tasks for multimolang:

Element Status
Package Name Defined and unique in CRAN.
DESCRIPTION File Complete with title, description, authors, maintainer, version, and dependencies.
Documentation Complete for the dfMaker function, with examples.
NEWS File Not included; optional for the first version.
Tests Implemented to ensure proper functionality.
README Included here.
Cross-Platform Compatibility Tested on Linux and Windows operating systems.
Included Functions Includes dfMaker; additional tools planned for multimodal analysis in future updates.

Note: Future updates will incorporate tools for prosody and voice analysis, as well as co-speech gesture processing.


About MULTIFLOW

The MULTIFLOW project investigates the interplay between gesture, prosody, and language to uncover multimodal signatures in communication. Learn more about our research and goals.

Publications in Preparation


Contributing

We welcome contributions to multimolang! For bug reports, feature requests, or collaboration inquiries, open an issue on GitHub.


See Also


Recent Changes

Version 0.1.1 (December 12, 2024)

For a complete list of changes, refer to the CHANGELOG in the repository.