Python Automation and Machine Learning for EM and ICs

An Online Book, First Edition by Dr. Yougui Liao (2019)

Practical Electron Microscopy and Database - An Online Book

Table of Contents/Index

Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix

   
Four/five V of big data Introduction
Evaluating a ML model with BigQuery ML Introduction
Building an effective machine learning team Introduction
Principles of ethical and responsible ML (selection bias, confirmation bias, automation bias, model fairness) Introduction
Best practices for implementing ML in semiconductor manufacturing Introduction
Identifying the business value of using ML Introduction
Big data lifecycle Introduction
XGBoost (Extreme Gradient Boosting) Introduction
BigQuery ML and Python-based ML frameworks Introduction
Python libraries for Bayesian ML techniques Introduction
BigQuery ML Introduction
Comparison between Naive Bayes algorithms and Bayesian machine learning techniques Introduction
Biological (human brain) and AI neural networks Introduction
Bisection search (binary search) Introduction
Branching Introduction
Code blocks Introduction
Mistakes that beginner machine learning (ML) students often make Introduction
Backtracking search Introduction
Binary constraint Introduction
Bayesian ML techniques Introduction
Knowledge base (repository) Introduction
Uninformed search/blind search algorithms Introduction
Breadth-First Search (BFS) Introduction
Bellman expectation and Bellman optimality equations Introduction
Reinforcement learning       
Introduction
Electroencephalogram cap (EEG cap) for brain Introduction
Bayesian networks Introduction
(Forward and backward) propagation equations Introduction
Batch Gradient Descent (BGD), Stochastic Gradient Descent (SGD), Mini-Batch Gradient Descent, Batch Stochastic Gradient Descent, Momentum, (Adagrad, Adadelta, RMSprop), and Adam (Adaptive Moment Estimation) Introduction
Neural network vs. end-to-end learning vs. black box model
Introduction
AdaBoost (Adaptive Boosting) Model Introduction
Boosting versus Bagging Introduction
Boosting in ML Introduction
Regular decision trees and decision trees with bagging Introduction
Bagging in decision trees Introduction
Bagging (Bootstrap Aggregating) Introduction
Gradient Boosting Introduction
Blackbox optimization algorithms Introduction
Comparison among Grid Search, Bayesian Optimization, Random Search and Manual Search Introduction
Bayesian optimization Introduction
Batch sizes Introduction
Bayes error/Bayes risk/Bayes rate/irreducible error Introduction
Frequentist approach versus Bayesian approach Introduction
Bias and variance, and bias-variance trade-off in ML Introduction
Comparison among classifier, hyperplane and decision boundary Introduction
Hyperplane/Decision Boundary in ML Introduction
Multivariate Bernoulli learning model Introduction
Bernoulli distribution Introduction
Exponential Family: Parameter, Sufficient Statistic, Natural Parameter, Base Measure and Log-Partition Function (Bernoulli distribution and Gaussian distribution) Introduction
Bayesian Probability, Bayesian Statistics (Distribution Over a Distribution), versus Bayesian Inference Introduction
Bandwidth parameter (τ) in LWR and KDE Introduction
Batch gradient descent Introduction
Practice ML projects for beginners Introduction
Binary trees Introduction
Big O notation Introduction
Point-Biserial Correlation Introduction
Brute force discretization Introduction
Union Bound/Boole's Inequality Introduction
Deviation Probability (Hoeffding Bound) Introduction
Sample Size versus Bounds Introduction
Probability bounds analysis (PBA) Introduction
Bound in math/ML Introduction
Similarity-based clustering method (SCM) Introduction
Send a variable from one script (back) to another script with a function Introduction
BERTScore/BERT (Bidirectional Encoder Representations from Transformer) Introduction
Built-ins/Builtins Commands in Python Introduction
Build own/customized keyword candidates Introduction
Overfitting and underfitting Introduction
Create a Batch File to Run a Python Script Introduction
pyodbc for bridging SQL to Python Introduction
.bat (batch) files for Command Prompt Windows Introduction
Automatically restart script execution after it breaks/fails/error Introduction
Plot graph/figure/image from CSV file/DataFrame by removing/hiding blank/empty cells with axis range (plt.xlim()) Introduction
Plot confidence bands Introduction
Compare dates (x days after or before a date), and difference between two dates in days Introduction
BaseException Introduction
Extract substrings between brackets (including brackets) Introduction
Get username and encoded password with getpass or or base64 Introduction
(Single) Naive Bayes/Gaussian Naive Bayes Introduction
Bayes' theorem (Bayes rule or Bayes law) in machine learning Introduction
Multinomial Naive Bayes algorithm Introduction
Random (Bootstrap) Forests Introduction
Box and Whisker plots Introduction
Built-in functions in Python Introduction
Add letter/commas/numbers/characters to the end/beginning of strings in a list Introduction
Replace the lines between two lines “xx” and “yy” in a text file with new lines Introduction
Remove string 0s from the back/end of a list until non-zero values Introduction
borderwidth= (code)
Automated Conversion of Text-Based Labels into Chrome-Compatible Bookmarks   
break a functions to exit a loop Stop the loop, e.g. while loop immediately. Introduction. Normally do not use it, but a break statement is needed if it’s possible to end a loop w/o the condition being false. code1, code2
.bold (code)
bool() Applies the standard truth testing procedure to the passed object/value and returns a boolean value, that is, converting a value to Boolean (True or False). The object will always return True, unless: the object is empty, like [], (), {}; the object is False; the object is 0; the object is None. code. (code).
__bool__() If for any object x, the __bool__() method is not defined, we get True as a result.
watchdog.utils.bricks.SkipRepeatsQueue Thread-safe event queue based on a special queue that skips adding the same event (FileSystemEvent) multiple times consecutively. Thus avoiding dispatching multiple event handling calls when multiple identical events are produced quicker than an observer can consume them.
watchdog.utils.BaseThread Producer thread base class subclassed by event emitters that generate events and populate a queue with them.
Parameters: event_queue (watchdog.events.EventQueue) – The event queue to populate with generated events.
watch (ObservedWatch) – The watch to observe and produce events for.
timeout (float) – Timeout (in seconds) between successive attempts at reading events.
watchdog.utils.BaseThread Consumer thread base class subclassed by event observer threads that dispatch events from an event queue to appropriate event handlers.
Parameters: timeout (float) – Event queue blocking timeout (in seconds).
class watchdog.observers.api.BaseObserver(emitter_class, timeout=1)
watchdog.observers.api.BaseObserver Platform-independent observer that polls a directory to detect file system changes.
watchdog.observers.api.BaseObserver File system independent observer that polls a directory to detect changes.
class watchdog.utils.BaseThread
__bool__  
bit_length  
Button() Introduction
scipy.linalg.block_diag Create a block diagonal matrix from the provided arrays.
Binary I/O, io.BytesIO f = open("myfile.jpg", "rb"), f = io.BytesIO(b"some initial binary data: \x00\x01"). Code.
skimage.measure.block_reduce(image, block_size) Down-sample image by applying function to local blocks.
BeautifulSoup Introduction. Has an excellent XML- and HTML- parsing library for beginners, but it may be a bit slow.
Bob Is developed at Idiap Research Institute in Switzerland, Bob is a free signal processing and machine learning toolbox. The toolbox is written in a mix of Python and C++. From image recognition to image and video processing using machine learning algorithms, a large number of packages are available in Bob to make all of this happen with great efficiency in a short time.
Bokeh A Data Visualisation library for Python, Bokeh allows interactive visualisation. It makes use of HTML and Javascript to provide graphics, making it reliable for contributing web-based applications. It is highly flexible and allows you to convert visualisation written in other libraries such as ggplot or matplotlib. Bokeh makes use of straight-forward commands to create composite statistical scenarios.
bokeh.plotting.figure.circle()  
v2.rectangle(image, start_point, end_point, color of border line, border thickness) border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code.
.backward() (code)
t.begin_fill() To be called just before drawing a shape to be filled. (code)
from pynput.mouse import Button (code)
.press(Button.left) (code)
.release(Button.left) (code)
.click(Button.left, x) x clicks of mouse. (code)
.press(Button.right) (code)
.release(Button.right) (code)
.bottomright (code)
.bind() Introduction
'<Button-1>' Single left click. (code)
highlightbackground= (code)
background= (code)
.buttons=  
.button_pressed()  
.top/.bottom/.left/.right  
basicConfig  
background_color=  
driver.navigate().back() Navigate backward in browser history
os.path.basename()  
Color in table obtained by matplotlib.pyplot/change background color of cells in table  
Mirror/reflect image from left to right/from top to bottom  
Plot horizontal stacked bar/histogram  
Training process in ML (with "best"-option table)  
Predictive/predict model (with "best"-option table)  
Median blurring and cv2.medianBlur()  
.pack(side=LEFT)/.pack(side=RIGHT)/.place(x=, y=) --- position of the buttons  
Locate/find the center/coordinates of a bright (maximum/highest intensity) spot in an image & find nearest white pixel to a given/specifical pixel location on an binary image  
Automated Detection and Clustering of Bright Spots in Grayscale Images Using Configurable Binning and 2D Connectivity Analysis  
find_element(By.XPATH, "")  
Add padding/black/colored edge to images  
Reasons (benefits) of automation and how to start  
Browser-based visualization tool  
Dummy variables/binary variables  
Categorical bins  
tf.feature_column.bucketized_column  
"Bridges" in Python coding  
Make selected text bold  
Open the page of the downloaded list of webpage browser  
Calculation in an Excel Sheet, Style, Bold, and Color  
Bring/activate an application/window to most front/foreground  
Bind/link multiple commands to buttons  
Bind Python functions and methods to events (similar to if loops)  
Turn on and off with mouse press or a process/button switch  
FLAT, RAISED, SUNKEN, GROOVE and RIDGE in Tkinter button relief styles  
Bag-of-words model  
Find the best word/text similarity  
(Single and multiple enter/input) box for pop-up window  
Calibrate and put a scale bar, and draw a line segment on an image  
Merge/combine two text files into a new text file, add a new line to the beginning of a text file  
Build databases with different/uncertain number of members  
Wheatstone bridge and its simulation Introduction.
Open webpages in internet browsers (Chrome, Microsoft edge, IE browser) Introduction
Set the output image to zero everywhere except my mask (color filter), and display red, green, and blue (RGB) channels of an image. code, code.
Extract three blue, green, red images from a color image or grey image, or convert RGB (color) image into three blue, green, red images code

Create images with global, adaptive mean, adaptive Gaussian, binary, trunc, Tozero, and tozero thresholds.

code
Bail out/terminate of a loop code
Find/search birthyear by name code
Dropdown box/option/selection/choice Introduction
Check if both files are the same file, e.g. symbolic link, shortcut Introduction
Invert the contrasts of black and white images (Code)
watchdog with conditioning break Introduction
Top (ranking, best, must know) Python libraries/modules Introduction
Measure length/distance on an image w/o calibrated bar Introduction
Break/exit/skip a function/code line after a certain time Introduction
Binary classifiers Introduction
Generate text file with the bank of collecting all words, characters and strings from news Introduction
Comparison between Python, Blue Prism, UiPath, Automation Anywhere Introduction
Binary images Introduction
Wafer bin map (WBM) Introduction
Detection procedures/processes of spatial defect patterns (bins) in wafers Introduction
Smooth images (make image blurry) Introduction