mexca.video.anfl
================

.. py:module:: mexca.video.anfl

.. autoapi-nested-parse::

   Action unit (AU) relationship-aware node feature learning (ANFL).

   Implementation of the ANFL module from the paper:

       Luo, C., Song, S., Xie, W., Shen, L., Gunes, H. (2022). Learning multi-dimentionsal edge
       feature-based AU relation graph for facial action unit recognition. *arXiv*.
       `<https://arxiv.org/pdf/2205.01782.pdf>`_

   Code adapted from the `OpenGraphAU <https://github.com/lingjivoo/OpenGraphAU/tree/main>`_ code base
   (licensed under Apache 2.0).



Classes
-------

.. autoapisummary::

   mexca.video.anfl.GNN
   mexca.video.anfl.AUFeatureGenerator
   mexca.video.anfl.FacialGraphGenerator
   mexca.video.anfl.ANFL


Module Contents
---------------

.. py:class:: GNN(in_features: int, n_nodes: int, n_neighbors: int = 4)



   Apply a graph neural network (GNN) layer.

   Transform action unit (AU) features using digraph connectivity.
   Inputs and outputs correspond to AU features.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_nodes: Number of nodes in the digraph.
   :type n_nodes: int
   :param n_neighbors: Number of top K similar neighbors for computing graph connectivity.
   :type n_neighbors: int, default=4

   .. rubric:: Notes

   See eq. 1 in the corresponding `paper <https://arxiv.org/abs/2205.01782>`_.
   Functions :math:`{g, r}` are linear and the nonlinear activation function :math:`\sigma` is ReLU.
   Linear layer weights are initialized with :math:`N(0, \sqrt{\frac{2}{out\_features}})`.
   Batch norm weights are initialized as 1 and biases as 0.


.. py:class:: AUFeatureGenerator(in_features: int, out_features: int = 27)



   Generate action unit (AU) features.

   Inputs correspond to face representations (embeddings) and outputs to AU features.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param out_features: Size of each output sample.
   :type out_features: int, default=27

   .. rubric:: Notes

   AU specific features are generated by individual linear and global average pooling transformations.


.. py:class:: FacialGraphGenerator(in_features: int, n_main_nodes: int = 27, n_sub_nodes: int = 14, n_neighbors: int = 4)



   Generate action unit (AU) activations from AU features using a facial graph.

   Inputs correspond to AU features and outputs to AU activations. Main plus sub nodes represent facial AUs.
   Sub nodes represent left and right activations of AUs 1, 2, 4, 6, 10, 12, and 14.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_main_nodes: Number of main nodes in the facial graph.
   :type n_main_nodes: int, default=27
   :param n_sub_nodes: Number of sub nodes in the facial graph.
   :type n_sub_nodes: int, default=14
   :param n_neighbors: Number of top K similar neighbors for computing graph connectivity.
   :type n_neighbors: int, default=4

   .. rubric:: Notes

   First applies a graph neural network (:func:`GNN`) transformation to AU features.
   Transformed features are fed into similarity calculating (SC) layers for main and sub
   nodes as in eq. 2 of the corresponding `paper <https://arxiv.org/abs/2205.01782>`_.
   Sub node activations are calulated based on matching main node features.
   SC layer weights are initialized using Glorot initialization (see :func:`torch.nn.init.xavier_uniform`).


.. py:class:: ANFL(in_features: int, n_main_aus: int = 27, n_sub_aus: int = 14, n_neighbors: int = 4)



   Apply AU relationship-aware node feature learning (ANFL).

   Transform face representations into facial action unit (AU) activations.
   Inputs correspond to facial representations (embeddings) and outputs to AU activations.

   :param in_features: Size of each input sample.
   :type in_features: int
   :param n_main_aus: Number of main AUs.
   :type n_main_aus: int, default=27
   :param n_sub_aus: Number of sub AUs.
   :type n_sub_aus: int, default=14
   :param n_neighbors: Number of top K similar neighbors for computing graph connectivity.
   :type n_neighbors: int, default=4

   .. rubric:: Notes

   First generates AU features from face representations (see :func:`AUFeatureGenerator`)
   and then transforms them into activations using a facial graph (see :func:`FacialGraphGenerator`).