Descent direction

In optimization, a descent direction is a vector $\mathbf {p} \in \mathbb {R} ^{n}$ that points towards a local minimum $\mathbf {x} ^{*}$ of an objective function $f:\mathbb {R} ^{n}\to \mathbb {R}$.

Computing $\mathbf {x} ^{*}$ by an iterative method, such as line search defines a descent direction $\mathbf {p} _{k}\in \mathbb {R} ^{n}$ at the $k$th iterate to be any $\mathbf {p} _{k}$ such that $\langle \mathbf {p} _{k},\nabla f(\mathbf {x} _{k})\rangle <0$, where $\langle ,\rangle$ denotes the inner product. The motivation for such an approach is that small steps along $\mathbf {p} _{k}$ guarantee that $\displaystyle f$ is reduced, by Taylor's theorem.

Using this definition, the negative of a non-zero gradient is always a descent direction, as $\langle -\nabla f(\mathbf {x} _{k}),\nabla f(\mathbf {x} _{k})\rangle =-\langle \nabla f(\mathbf {x} _{k}),\nabla f(\mathbf {x} _{k})\rangle <0$.

Numerous methods exist to compute descent directions, all with differing merits, such as gradient descent or the conjugate gradient method.

More generally, if $P$ is a positive definite matrix, then $p_{k}=-P\nabla f(x_{k})$ is a descent direction at $x_{k}$. This generality is used in preconditioned gradient descent methods.

See also

• Directional derivative