=================================================================================

Performance metrics in machine learning are quantitative measures used to evaluate the effectiveness and efficiency of a machine learning model. These metrics provide insights into how well a model is performing on a given task and help assess its generalization capabilities. The choice of performance metrics depends on the specific nature of the problem being solved, as different tasks may require different evaluation criteria. 

Some common performance metrics in machine learning are: '

1. Accuracy: 

   The ratio of correctly predicted instances to the total instances. It is a simple and widely used metric, but it may not be suitable for imbalanced datasets. 

2. Precision: The ratio of true positive predictions to the total predicted positive instances. It focuses on the accuracy of positive predictions. 

3. Recall (Sensitivity or True Positive Rate): The ratio of true positive predictions to the total actual positive instances. It measures the ability of the model to capture all the relevant instances. 

4. F1 Score: The harmonic mean of precision and recall. It provides a balance between precision and recall, especially when there is an imbalance between classes. 

5. Specificity (True Negative Rate): The ratio of true negative predictions to the total actual negative instances. It measures the ability of the model to correctly identify negative instances. 

6. Area Under the Receiver Operating Characteristic curve (AUC-ROC): A metric used for binary classification problems. It measures the model's ability to distinguish between positive and negative instances across different probability thresholds. 

7. Mean Squared Error (MSE): Commonly used for regression problems, it calculates the average of the squared differences between predicted and actual values. 

8. R-squared (Coefficient of Determination): Another metric for regression tasks, it indicates the proportion of the variance in the dependent variable that is predictable from the independent variables. 

9. Confusion Matrix: A table that summarizes the number of true positive, true negative, false positive, and false negative predictions. It is often used to derive other metrics such as precision, recall, and accuracy. 

The choice of which metric to use depends on the specific goals and characteristics of the machine learning task. For example, in a medical diagnosis task, high recall might be more important to minimize false negatives, even at the cost of increased false positives. In fraud detection, precision might be more critical to minimize false positives. 

Performance metrics are easier to understand and are directly connected to business goals, which are benefits of Performance metrics over loss functions.    

Note that bias, see page3352, can exist in performance metrics, e.g. accuracy versus balanced accuracy:

Using simple accuracy as a performance metric might be misleading in imbalanced datasets. For instance, if one class significantly outnumbers another, a model might appear to perform well by simply predicting the majority class more frequently.

Mitigation of bias: 

• Use balanced accuracy, which calculates the average of recall obtained on each class, giving equal weight to the performance on each class.
• Implement other metrics like F1-score, precision-recall curves, or AUC-ROC for a more nuanced evaluation.

============================================