r"""Applies an affine transformation to the input. Concretely: .. math:: y = x W^\top + b where: where :math:`W` has shape

r"""Applies layer normalization [1] on the inputs. Computes .. math:: y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta,

r"""Randomly zero a portion of the elements during training. The remaining elements are multiplied with :math:`\frac{1}{1-p}` where :math:`p`

Implements the scaled dot product attention with multiple heads. Given inputs for queries, keys and values the ``MultiHeadAttention`` produce

* Small wrapper over a TCP socket to simplify initiating connections. */

* Small wrapper over a TCP socket to simplify initiating connections. */

Wrappers of CUDA resources.

A simple MLP.

A simple MLP.