mexca.video.mefl

Multi-dimensional edge feature learning (MEFL).

Implementation of the MEFL module from the paper:

Luo, C., Song, S., Xie, W., Shen, L., Gunes, H. (2022). Learning multi-dimentionsal edge feature-based AU relation graph for facial action unit recognition. arXiv. https://arxiv.org/pdf/2205.01782.pdf

Code adapted from the OpenGraphAU code base (licensed under Apache 2.0).

Module Contents

Classes

CrossAttention	Apply a cross-attention layer.
GraphEdgeModel	Learn the relationships between nodes in a graph.
GatedGNNLayer	Apply a gated graph neural network (GNN) layer.
GatedGNN	Apply multiple gated graph neural network (GNN) layers.
MEFL	Apply multi-dimentional edge feature learning.

class mexca.video.mefl.CrossAttention(in_features: int)

Apply a cross-attention layer.

Parameters:

in_features (int) – Size of each input sample.

Notes

Performs cross-attention between two inputs x and y as defined in eq. 4 of the corresponding paper. Linear layer weights are initialized with \(N(0, \sqrt{\frac{2}{out\_features}})\).

class mexca.video.mefl.GraphEdgeModel(in_features: int, n_nodes: int)

Learn the relationships between nodes in a graph.

Graph edge model: This class combines facial display-specific action unit representation modeling (FAM; i.e., cross-attention) with AU relationship modeling (ARM).

Parameters:

• in_features (int) – Size of each input sample.

• n_nodes (int) – Number of nodes in the graph.

Notes

Linear layer weights are initialized with \(N(0, \sqrt{\frac{2}{out\_features}})\). Batch norm weights are initialized as 1 and biases as 0.

class mexca.video.mefl.GatedGNNLayer(in_features: int, n_nodes: int, dropout_rate: float = 0.1)

Apply a gated graph neural network (GNN) layer.

Parameters:

• in_features (int) – Size of each input sample.

• n_nodes (int) – Number of nodes in the graph.

• dropout_rate (float, default=0.1) – Rate parameter of the dropout layer.

Notes

Performs gated graph convolution according to Bresson and Laurent (2018, eq. 11). Linear layer weights are initialized with \(N(0, \sqrt{\frac{2}{out\_features}})\). Batch norm weights are initialized as 1 and biases as 0.

class mexca.video.mefl.GatedGNN(in_features: int, n_nodes: int, n_layers: int = 2)

Apply multiple gated graph neural network (GNN) layers.

Parameters:

• in_features (int) – Size of each input sample.

• n_nodes (int) – Number of nodes in the graph.

• n_layers (int, default=2) – Number of gated GNN layers.

Notes

Performs gated graph convolution according to Bresson and Laurent (2018, eq. 11) for multiple layers.

class mexca.video.mefl.MEFL(in_features: int, n_main_nodes: int = 27, n_sub_nodes: int = 14)

Apply multi-dimentional edge feature learning.

Parameters:

• in_features (int) – Size of each input sample.

• n_main_nodes (int, default=27) – Number of main nodes in the facial graph.

• n_sub_nodes (int, default=14) – Number of sub nodes in the facial graph.

Notes

First learns node features via a series of linear layers for each main graph node. It then learns node relationships between graph nodes via graph edge modeling (GEM; cross-attention) from inputs and node features. Node features are transformed via global average pooling (GAP) and forwarded to multiple gated GNN layers together with the node relationship weights. Finally, a similarity calculation (SC) layer (cosine similarity) is applied to predict node activations. Sub node activations are calulated based on matching main node features. SC layer weights are initialized using Glorot initialization (see torch.nn.init.xavier_uniform()).