GELUFunction Class Reference

The GELU function, defined by. More...

Static Public Member Functions

static double Deriv (const double y)
 Computes the first derivative of the GELU function. More...

 
template
static void Deriv (const InputVecType &y, OutputVecType &x)
 Computes the first derivatives of the GELU function. More...

 
static double Fn (const double x)
 Computes the GELU function. More...

 
template
static void Fn (const InputVecType &x, OutputVecType &y)
 Computes the GELU function. More...

 

Detailed Description

The GELU function, defined by.

\begin{eqnarray*} f(x) = 0.5 * x * {1 + tanh[(2/pi)^(1/2) * (x + 0.044715 * x^3)]} \\ f'(x) = 0.5 * tanh(0.0356774 * x^3) + 0.797885 * x) + (0.0535161x^3 + 0.398942 * x) * sech^2(0.0356774 * x^3+0.797885 * x) + 0.5\\ \end{eqnarray*}

Definition at line 31 of file gelu_function.hpp.

Member Function Documentation

◆ Deriv() [1/2]

static double Deriv ( const double  y)
inlinestatic

Computes the first derivative of the GELU function.

Parameters
yInput data.
Returns
f'(x)

Definition at line 65 of file gelu_function.hpp.

◆ Deriv() [2/2]

static void Deriv ( const InputVecType &  y,
OutputVecType &  x 
)
inlinestatic

Computes the first derivatives of the GELU function.

Parameters
yInput data.
xThe resulting derivatives.

Definition at line 80 of file gelu_function.hpp.

◆ Fn() [1/2]

static double Fn ( const double  x)
inlinestatic

Computes the GELU function.

Parameters
xInput data.
Returns
f(x).

Definition at line 40 of file gelu_function.hpp.

References M_PI.

◆ Fn() [2/2]

static void Fn ( const InputVecType &  x,
OutputVecType &  y 
)
inlinestatic

Computes the GELU function.

Parameters
xInput data.
yThe resulting output activation.

Definition at line 53 of file gelu_function.hpp.

References M_PI.


The documentation for this class was generated from the following file:
  • /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.4.2/src/mlpack/methods/ann/activation_functions/gelu_function.hpp