~ Gradient descent

Gradient descent describes the process of finding a local minimum of a function by following the negative value of the gradient at each point stepwise. Notationally, this is described in the following way:

$$a_{i+1} = a_i - \eta \nabla f(a_i).$$

Here, $a_i$ refers to the $i$’th step, $\eta$ is the step size and $f$ is the function we want the minimum of.

Page made with mdzk