Lecture 13 - Unconstrained Optimization

Resources:

• Nocedal-Wright2006_Book_NumericalOptimization.pdf Ch 2.1-2.2

---

Just look at the optimization problem without the constraints $\mathcal{I}$ and $\mathcal{E}$. Has use cases in machine learning (gradient descent etc).

Formula

${\min }_{x\in {\mathbb{R}}^n}f(x)$ Where $f:{\mathbb{R}}^n\to \mathbb{R}$ is a smooth function

---

Sub-Topics and Term

Taylors Theorem General Algorithm for Solving Unconstrained Optimization Termination Criteria Descent Directions Scaling

Lecture notes

if $x^{\ast }$ is a global solution, then $f(x^{\ast })\le f(x),\forall x$ If $x^{\ast }$ is a local solution, then $f(x^{\ast })\le f(x),\forall x\in \mathcal{N}(x^{\ast })$