=================================================================================

Naive Bayes is a probabilistic machine learning algorithm that is commonly used for text classification tasks. It's based on Bayes' theorem and makes the assumption of independence among features, which is why it's called "naive." In text classification, Naive Bayes is often applied to categorize documents into predefined classes or categories. It's widely used for tasks like spam detection, sentiment analysis, and topic categorization. The algorithm calculates the probability of a document belonging to each class based on the occurrence of words or features within the document. There are different variants of Naive Bayes classifiers, such as Multinomial Naive Bayes for text classification where the features are word frequencies.  The class (c), which maximizes the posterior probability given the input document (d), is represented by argmax_c∈CP(c∣d),

              ------------------------------- [3601a]

where,

           P(c) is the prior probability of class c, representing the probability of encountering class c without considering any specific features from the document. P(c) is equal to the ratio between the number of documents of class c and the total number of documents.  

          P(d∣c) is the likelihood, representing the probability of observing the document d given that it belongs to class c. In text classification, this is often calculated based on the occurrence of words or features in the document. 

          P(d) is the evidence probability, representing the probability of observing the document d regardless of its class. It acts as a normalizing factor. 

In text classification, the class in Equation 3601a is predicted for a given document.  Then, c^ is the most probable class, given by,

             ---------------------------- [3601b]

P(d|c) can be given by,

            P(d|c) = P([w₁, w₂, ..., w_n]|c)  ---------------------------------------- [3601c]

In the text classification using Naive Bayes, two simplifying assumptions are commonly made: 

1. Bag of Words (BOW) representation:

   This assumption implies that the order of words in the document is not considered. The text is treated as an unordered set of words, and only the frequency or presence of words matters. This simplification helps in reducing the complexity of the model and makes it computationally more efficient.

2. Conditional Independence of Word Appearances: 

   The "naive" assumption in Naive Bayes is that the occurrences or frequencies of individual words in the document are independent of each other, given the class label. In reality, words in a document might be dependent on each other, but this assumption simplifies the calculation of probabilities. Despite its simplicity, Naive Bayes often performs well in practice for text classification tasks.

With the assumptions above, then we can have,

             P(d|c) = P(w₁|c) * P(w₂|c) * ... * P(w_n|c)  ---------------------------------------- [3601d]

Equation 3601d is much easier to calculate than Equation 3601c. However, Equation 3601d multiplies a lot of small numbers, which consumes a large computer memories. To make the computation further easier, we use the logarithmic form, instead, as shown below,

             ---------------------------------------- [3601e]

The advantage of using the logarithmic form of the Naive Bayes equation lies in numerical stability and computational efficiency:

1. Numerical Stability: 

   When dealing with small probabilities, multiplying many probabilities together can lead to underflow issues (resulting in numerical instability). Taking the logarithm helps in converting multiplication operations into addition operations, making the calculations more stable.

2.  Computational Efficiency: 

   Computing the sum of logarithms is computationally more efficient than multiplying a large number of probabilities, especially when working with real-world datasets that can have a large number of features (words in the case of text classification). 

3. Avoiding Underflow: 

   In the product form, if we have many small probabilities, the product can become extremely small, leading to numerical precision issues. The logarithmic form mitigates this problem by transforming the multiplication into addition, preventing underflow. 

4. Simplifying Optimization: 

   When training a Naive Bayes model, the logarithmic form simplifies the optimization process. Maximizing the log-likelihood is equivalent to maximizing the likelihood, but it is often more numerically stable and simplifies the mathematics involved in optimization. 

Table 3601a shows an example of texts for text classification.   

 Table 3601a. An example of texts for text classification.

In the example in Table 3601a, we have two classes (Spam and Not Spam) and 6 documents in total. Then, the priors, which are the probabilities of the classes, are:

         P (Spam) = 4/6 = 2/3  ---------------------------------- [3601f]

         P(Not Spam) = 2/6 = 1/3  ---------------------------------- [3601g]

We need to calculate the likelihood of each word given each class, P(w_i|c), given by,

           ---------------------------------- [3601h]

For instance,  the likelihood of the word "Phishing" given the documents "Spam" is given by,

          P(Phishing|Spam) = 3/11 ---------------------------------- [3601i]

And, then we do the same calculations for every words. 

However, when we calculate the likelihood of the word "payment", then we get,

          P(help|Spam) = 0/11  ---------------------------------- [3601j]

We get this likelihood as zero since the "payment" word doesn't occur in the "Spam" class.  In this case, a problem happens: when we take its log, then we have computation error. To fix this error, we add a smoothing parameter (α), which is often set to 1, and another smoothing parameter |V|, which is the size of the total vocabulary:   

           ---------------------------------- [3601k]

Then, Equation 3601j becomes,  

          P(help|Spam) = (0+1)/(11+17)  ---------------------------------- [3601l]

Now, our model has been trained.   

Assuming we want to  calculate the likelihood probability (Spam ∣ Gain Phishing) in log scale, we need to use Equation 3601a. First, we calculate that the document "Gain Phishing" belongs to the class "Spam," so that the likelihood probability in log scale would be:   

         logP(Spam∣Gain Phishing) = logP(Spam) + logP(Gain|Spam) + logP(Phishing|Spam) -------------------------------- [3601m]

From Table 3601a, we can have:

         logP(Spam) = −2.0  

         logP(Not Spam) = −1.5 

         logP(Tech|Spam) = −3.0 

         logP(Support|Spam) = −2.5 

         logP(Phishing|Spam) = −1.8 

         logP(Gain|Spam) =  −3.3322

        logP(Phishing|Spam) =  −0.6931

        logP(Gain|Not Spam) = −3.1352

        logP(Phishing|Not Spam)  = −3.1352

Then, we have,

        logP(Spam∣Gain Phishing)  = (−2.0) + (−3.3322) + ( −0.6931) = −6.0253 ------------------------ [3601n]

Secondly, we calculate that the document "Gain Phishing" belongs to the class "Not  Spam," so that the likelihood probability in log scale would be:  

         logP(Not Spam∣Gain Phishing) = logP(Not Spam) + logP(Gain|Not Spam) + logP(Phishing|Not Spam) ------------- [3601o]

                                                            = (−1.5 ) + (−3.1352) + (−3.1352) = −7.7704 -------------------- [3601o]

Then, the probability of P(Spam|Gain Phishing) in percentile is given by,

         P(Spam|Gain Phishing)  = 100 x  exp(−6.0253 )/(exp(−6.0253 ) + exp(−7.7704)) = 74.074 %

============================================