AIFH Volume 3, Chapter 1: Activation Functions

Sigmoid Activation Function

$$\sigma(x) = \frac{1}{1 + e^{-x}}.$$

Tanh Activation Function

$$\sigma(x) = tanh(x).$$

Linear Activation Function

$$\sigma(x) = x.$$