In the regularization,why we use θ^2 rather than θ？
C.J
The regularization is lambda*sum(θ^2)
bakkal
I've already answered this in your previous question (see last paragraph), but I'll try again.
The problem regularizing with sum(θ) is that you may have θ parameters that cancel each other
Example:
θ_1 = +1000000
θ_2 = -1000001
The sum(θ) here is +1000000 -1000001 = -1 which is small
The sum(θ²) is 1000000² + (-1000001)² which is very big.
If you use sum(θ) you may end up without regularization (which was the goal) because of large θ values that escaped the regularization because the terms cancel each other out.
You may use sum(|θ|) depending on your search/optimisation algorithm. But I know θ² (L2 norm) to be popular and works well with gradient descent.
Collected from the Internet
Please contact [email protected] to delete if infringement.
edited at
Related
Related Related
- 1
Why should we use classes rather than records, or vice versa?
- 2
Why do some CLIs use ` and ' rather than ‛ and ’?
- 3
Why use TimeSpan.CompareTo() rather than < > or =
- 4
Why use a function rather than a reference to member?
- 5
Why use ImageIcon rather than Image?
- 6
Why to use ByteArrayInputStream rather than byte[] in Java
- 7
Why use "nohup &" rather than "exec &"
- 8
Why do some CLIs use ` and ' rather than ‛ and ’?
- 9
Why use Python format rather than slicing?
- 10
Why use regex finditer() rather than findall()
- 11
Angular 6 - Why use @ngrx/store rather than service injection
- 12
Why do STL numeric algorithms use 'op' rather than 'op='?
- 13
Why use address of first element of struct, rather than struct itself?
- 14
Why does Clojure Spels use quoted symbols, rather than keywords?
- 15
Why do CDNs use different domain names rather than subdomains?
- 16
Why should I use a pointer rather than the object itself?
- 17
Why does Apple use assign rather than weak to store a delegate?
- 18
Why does `libpq` use polling rather than notification for data fetch?
- 19
Why do Microsoft use 14.667 rather than 14 as the ControlContentThemeFontSize?
- 20
Why do *nix folk like to use ``...'' rather than the " character?
- 21
Why is 'use strict' usually after an IIFE (rather than at the top of a script)?
- 22
Why does Prolog use =< Rather than <= like most languages?
- 23
Why does Go use ^ rather than ~ for unary bitwise-not?
- 24
Why should I use simple arrays rather than container classes?
- 25
Why does GWT use code generators rather than annotation processors?
- 26
Why use Spring Boot rather than Spring Boot-less?
- 27
why use invoke helper rather than just call functor?
- 28
Why use ContextImpl to implement Context rather than ContextWrapper in Android?
- 29
Why use a Messenger rather than passing the reference to a Handler?
Comments