# SVM Remark

November 15, 2018 - 2 minute read -
• The original Vision for SVM: Maximize the distance of points to the separation hyper plane and put them in different sides. We can formulate the problem in the following form:

, where $d_\beta ( \mathbf{x} )$ is distance of feature $\mathbf{x}$ to the separation plane with paramter $\beta$.

• But it is sometimes harder to solve the min-max style optimization problems. Hence we construct a proxy problem to find the solution for $\beta$.

, where $d_\beta ( \mathbf{x} )$ is computed based on $1/ \mid \beta \mid$. Thus, we can translate the problem into

• It is sometimes inveitable that some error will occur. Thus we introduce the idea of soft margin and modify the loss function for this problem. We want to optimize the loss function:

when $C$ is huge, the tolerance for error classificaitonis low. Alternatively, we can also consider loss function with the following form:

Under this scenario, if $\lambda$ is huge, then the marin should be big and the tolerance for error classification is high.

• For the loss function, we can consider a so-called hinge loss, which is

. It get the name because the graph of the function looks like a hinge. Finally, the output loss function is

. Because $y_i(\beta\mathbf{x}-\alpha)$ should be larger than $1$, this equation will penalize points that wrongly calssified.