Introduction

Mathematical Optimization

Optimization Problem

$minimize \space f_0(x)$ $subject \space to \space f_i(x) \le b_i, i = 1,...,m$

• $x = (x_1, x_2 ... x_n)$ : Optimization variables

• $f_0 : \mathbb{R}^n \to \mathbb{R}$ : Objective function

• $f_i : \mathbb{R}^n \to \mathbb{R}, i = 1,...,m$ : Constraint functions

Optimal Solution

$x^*$ is the smallest value of $f_0$ among all vectors that satisfy the constraints

Solving optimization problems

General optimization problem

• Very difficult to solve

• Not always finding the solution

Exception

• Least-Squares problems

• Linear programming problems

• Convex optimization problems

Least Squares

$minimize \space \lVert Ax - b \rVert_2^2$

• Analytical Solution : $x^* = (A^TA)^{-1}A^Tb$

Linear Programming

$minimize \space c^Tx$ $subject \space to \space a_i^Tx \le b_i, i = 1,...,m$

Convex Optimizatiom

Objective and Constraint functions should be convex