Electron microscopy
  
Independent and Identically Distributed (i.i.d./IID)
- Python for Integrated Circuits -
- An Online Book -
Python for Integrated Circuits                                                                                   http://www.globalsino.com/ICs/        


Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix

=================================================================================

Independent and Identically Distributed (i.i.d.) is a term commonly used in statistics and probability theory to describe a set of random variables or data points that share two important characteristics:

  1. Independence: This means that each random variable or data point is not influenced by or dependent on the others in the set. In other words, the occurrence or value of one variable does not provide any information about the occurrence or value of another variable in the set. Mathematically, if X1, X2 , ..., Xn are a set of random variables, they are independent if:

    P(Xi | X1, X2, ..., Xi-1, Xi+1, ..., Xn) = P(Xi) --------------------------------------- [3878a]

    This means that the conditional probability of Xi given all the other variables is equal to the marginal probability of Xi.

  2. Identically Distributed: This means that all random variables or data points in the set have the same probability distribution. In other words, they have the same range of possible values and follow the same probability density function (PDF) or probability mass function (PMF). Mathematically, if X1, X2, X3, ..., Xn are identically distributed, it means:

    P(Xi) = P(Xj) for all i and j in the set.

The formula for the variance of the sample mean () of independent and identically distributed (i.i.d.) random variables with variance can be given by,

          formula for the variance --------------------------------- [3878b]

where,

         δ2 is the variance of each individual random variable

is the sample size.

Figure 3878a shows the comparison between IID (Independent and Identically Distributed) and DID (Dependent Identically Distributed). The blue histograms represent the distributions of the variables when the variables are Independent and Identically Distributed. The orange histograms represent the distributions of the same variables, but now taking into account the correlation with other variables. The presence of the orange histograms indicate how the variable's distributions are influenced by the correlations with other variables.

formula for the variance

(a)

formula for the variance

(b)

Figure 3878a. Comparison between IID and DID: (a) with "medium" correlation strength of correlation_matrix = np.array([[1.0, 0.5, 0.3], [0.5, 1.0, 0.4], [0.3, 0.4, 1.0]]), and (b) with "low" correlation strength of correlation_matrix = np.array([[0.2, 0.5, 0.3], [0.2, 0.2, 0.2], [0.3, 0.2, 0.3]]) (Code).

============================================

         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         
         

 

 

 

 

 



















































 

 

 

 

 
=================================================================================