mexca.postprocessing

Post-process extracted emotion expression features.

Attributes

AU_REF

Names of facial action units.

LANDMARKS_REF

Indices of facial landmarks.

Functions

split_list_columns(→ Union[polars.LazyFrame, ...)

Split (nested) list columns into separate columns.

get_face_speaker_mapping(→ Dict[str, str])

Get optimal mapping between face and speaker labels by counting overlapping frames.

sub_labels(→ Union[polars.LazyFrame, polars.DataFrame])

Replace label column with labels from a mapping.

Module Contents

mexca.postprocessing.AU_REF = [1, 2, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 32,...[source]

Names of facial action units.

mexca.postprocessing.LANDMARKS_REF[source]

Indices of facial landmarks.

mexca.postprocessing.split_list_columns(df: polars.LazyFrame | polars.DataFrame, au_columns: Iterable, landmark_columns: Iterable) → polars.LazyFrame | polars.DataFrame[source]

Split (nested) list columns into separate columns.

Parameters:

  • df (polars.LazyFrame or polars.DataFrame) – Data frame with extracted emotion expression features as stored in Multimodal.features.

  • au_columns (Iterable) – Names for new facial action unit columns.

  • landmark_columns (Iterable) – Names for new landmark columns.

Notes

For example, Pipeline.apply() returns a polars.LazyFrame with (nested) list columns face_box, face_au, and face_landmarks containing multiple coordinates or predictions for multiple facial action units per row. These can be split into separate columns which only contain a single value per row.

mexca.postprocessing.get_face_speaker_mapping(df: polars.DataFrame, face_label_column_name: str = 'face_label', speaker_label_column_name: str = 'segment_speaker_label') → Dict[str, str][source]

Get optimal mapping between face and speaker labels by counting overlapping frames.

Uses the Hungarian algorithm to find an optimal mapping between face and speaker labels.

Parameters:

  • df (polars.DataFrame) – Data frame with columns face_label_column_name and speaker_label_column_name`.

  • face_label_column_name (str, default="face_label") – Name of the face label column.

  • speaker_label_column_name (str, default="segment_speaker_label") – Name of the speaker label column.

mexca.postprocessing.sub_labels(df: polars.LazyFrame | polars.DataFrame, mapping: Dict, column: str) → polars.LazyFrame | polars.DataFrame[source]

Replace label column with labels from a mapping.

dfpolars.LazyFrame or polars.DataFrame

Data frame with label column.

mappingdict

Dictionary with mapping.

columnstr

Name of label column in data frame.