Electron microscopy
  
PythonML
Jensen's Inequality
- Python Automation and Machine Learning for ICs -
- An Online Book -
Python Automation and Machine Learning for ICs                                                           http://www.globalsino.com/ICs/        


Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix

=================================================================================

Jensen's Inequality is a mathematical concept that is often used in the field of machine learning, particularly in expectations and convex functions. The inequality provides a relationship between the expectation of a convex function applied to a random variable and the convex function of the expectation of that random variable.

For a convex function , and a random variable , Jensen's Inequality states that:

          Jensen's Inequality ------------------------------------ [3694a]

Here, represents the expectation operator. For convex function , we have > 0.

In simpler terms, the expected value of the convex function applied to a random variable is greater than or equal to the convex function of the expected value of that random variable.

This inequality has important implications in machine learning, especially when dealing with risk, optimization, and the analysis of algorithms. It is commonly used in convex optimization problems, where convexity is a desirable property for ensuring the convergence and efficiency of optimization algorithms.

Assuming f(x) = x2, the probability of is 1/2 and the probability of is also 1/2, we then can calculate the expected value using the formula for the expected value of a function of a random variable,

          Jensen's Inequality ------------------------------------ [3694b]

In this case, where and , we can then calculate:         

          Jensen's Inequality ------------------------------------ [3694c]

Therefore, under the given probabilities, the expected value for the convex function = 6.5 as shown in Figure 3694. In this figure, f(E[x]) = 0.24. Therefore, Jensen's Inequality 3694a is true.

Jensen's Inequality

Figure 3694. Jensen's inequality for the convex function = x2 (code).

If the function is a concave function, the direction of Jensen's Inequality is reversed compared to the convex case. Then, we have,

          Jensen's Inequality ------------------------------------ [3694d]

And, < 0.

============================================

         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         

 

 

 

 

 



















































 

 

 

 

 
=================================================================================