mexca.video.mefl
================

.. py:module:: mexca.video.mefl

.. autoapi-nested-parse::

   Multi-dimensional edge feature learning (MEFL).

   Implementation of the MEFL module from the paper:

       Luo, C., Song, S., Xie, W., Shen, L., Gunes, H. (2022). Learning multi-dimentionsal edge
       feature-based AU relation graph for facial action unit recognition. *arXiv*.
       `<https://arxiv.org/pdf/2205.01782.pdf>`_

   Code adapted from the `OpenGraphAU <https://github.com/lingjivoo/OpenGraphAU/tree/main>`_ code base
   (licensed under Apache 2.0).



Classes
-------

.. autoapisummary::

   mexca.video.mefl.CrossAttention
   mexca.video.mefl.GraphEdgeModel
   mexca.video.mefl.GatedGNNLayer
   mexca.video.mefl.GatedGNN
   mexca.video.mefl.MEFL


Module Contents
---------------

.. py:class:: CrossAttention(in_features: int)



   Apply a cross-attention layer.

   :param in_features: Size of each input sample.
   :type in_features: int

   .. rubric:: Notes

   Performs cross-attention between two inputs *x* and *y* as defined in eq. 4 of
   the corresponding `paper <https://arxiv.org/abs/2205.01782>`_.
   Linear layer weights are initialized with :math:`N(0, \sqrt{\frac{2}{out\_features}})`.


.. py:class:: GraphEdgeModel(in_features: int, n_nodes: int)



   Learn the relationships between nodes in a graph.

   Graph edge model: This class combines facial display-specific action unit representation modeling (FAM; i.e., cross-attention)
   with AU relationship modeling (ARM).

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_nodes: Number of nodes in the graph.
   :type n_nodes: int

   .. rubric:: Notes

   Linear layer weights are initialized with :math:`N(0, \sqrt{\frac{2}{out\_features}})`.
   Batch norm weights are initialized as 1 and biases as 0.


.. py:class:: GatedGNNLayer(in_features: int, n_nodes: int, dropout_rate: float = 0.1)



   Apply a gated graph neural network (GNN) layer.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_nodes: Number of nodes in the graph.
   :type n_nodes: int
   :param dropout_rate: Rate parameter of the dropout layer.
   :type dropout_rate: float, default=0.1

   .. rubric:: Notes

   Performs gated graph convolution according to `Bresson and Laurent (2018, eq. 11) <https://arxiv.org/pdf/1711.07553.pdf>`_.
   Linear layer weights are initialized with :math:`N(0, \sqrt{\frac{2}{out\_features}})`.
   Batch norm weights are initialized as 1 and biases as 0.


.. py:class:: GatedGNN(in_features: int, n_nodes: int, n_layers: int = 2)



   Apply multiple gated graph neural network (GNN) layers.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_nodes: Number of nodes in the graph.
   :type n_nodes: int
   :param n_layers: Number of gated GNN layers.
   :type n_layers: int, default=2

   .. rubric:: Notes

   Performs gated graph convolution according to Bresson and Laurent (2018, eq. 11) for multiple layers.


.. py:class:: MEFL(in_features: int, n_main_nodes: int = 27, n_sub_nodes: int = 14)



   Apply multi-dimentional edge feature learning.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_main_nodes: Number of main nodes in the facial graph.
   :type n_main_nodes: int, default=27
   :param n_sub_nodes: Number of sub nodes in the facial graph.
   :type n_sub_nodes: int, default=14

   .. rubric:: Notes

   First learns node features via a series of linear layers for each main graph node.
   It then learns node relationships between graph nodes via graph edge modeling (GEM; cross-attention)
   from inputs and node features. Node features are transformed via global average pooling (GAP) and forwarded
   to multiple gated GNN layers together with the node relationship weights. Finally,
   a similarity calculation (SC) layer (cosine similarity) is applied to predict node activations.
   Sub node activations are calulated based on matching main node features.
   SC layer weights are initialized using Glorot initialization (see :func:`torch.nn.init.xavier_uniform`).