Taking the Derivative of the Sigmoid Function

First, apply the reciprocal rule.

$$
s’(x) = \frac{d}{dx} \left( \frac{1}{e^{-x}+1} \right) = \frac{-\frac{d}{dx}(e^{-x}+1)}{(e^{-x}+1)^2}
$$

Second, this is just a linear equation, so we can take the derivative of each part. Constant 1 goes away.

$$
= \frac{-\frac{d}{dx}(e^{-x})}{(e^{-x}+1)^2}
$$

Next, exponential function rule with chain rule.

$$
= \frac{-( e^{-x} \cdot \frac{d}{dx}({-x}))}{(e^{-x}+1)^2}
$$

Next, this is just a linear equation, so we can take the derivative of each part.

$$
= \frac{-(-1)\cdot{e}^{-x}}{(e^{-x}+1)^2} =
$$

Eliminate the double negative.

$$
= \frac{e^{-x}}{(e^{-x}+1)^2}
$$

We could stop with the above derivative. However, most texts do not display the above derivative as the final form of the sigmoid functions derivative. For “computational efficiency” we algebraically transform this to use the original sigmoid function twice.

$$
= \left( \frac{1}{e^{-x}+1} \right) \left( \frac{-e^{-x}}{e^{-x}+1} \right)
$$

This now reduces to the commonly used form of the sigmoid’s derivative.

$$
s’(x)= s(x)(1-s(x))
$$