Python Automation and Machine Learning for EM and ICs

An Online Book, First Edition by Dr. Yougui Liao (2019)

Practical Electron Microscopy and Database - An Online Book

Table of Contents/Index

Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix

   
Overcoming automation challenges and forward-looking suggestions  
LASSO (Least Absolute Shrinkage and Selection Operator)  
Epochs and overfitting  
YOLOv8 (You Only Look Once, version 8)  
Leveraging precision, speed, and automation: Integrating Mask R-CNN and YOLOv8  
OLS (Ordinary Least Squares) regression model  
Facets Overview  
First-order logic (FOL)  
POMDP (Partially Observable Markov Decision Process)  
Optimal value function in Markov Decision Process (MDP)  
Bellman expectation and Bellman optimality equations  
Reinforcement learning  
Open datasets, and open-source tools and libraries for Python and ML practice  
Finding a correct loss (risk, objective) function for a specific problem  
Leave-One-Out Cross-Validation (LOOCV)  
Optimal margin classifier/maximum margin separator  
Laplace smoothing/Laplace correction/add-one smoothing  
Algorithms for directly finding the global optimum  
Hypothesis (predicted output (h(x)))  
Big O notation  
Output (target variable, y, Y)  
Output the web links obtained by Google Search  
Extract the least/most frequency/duplicate/occurrence element in a list  
One hot encoding  
Outlier of feature  
except OSError  
.OneHotEncoder()  
Optimizer/optimization algorithms  
Blackbox optimization algorithms  
Comparison among Grid Search, Bayesian Optimization, Random Search and Manual Search  
Bayesian optimization  
Convergence and Optimization  
Adam optimization algorithm  
tf.keras.optimizers.Adam  
Optimization of energy efficiency in machine learning systems  
Global optimization  
Direct optimization  
scipy.optimize.linprog function  
OpenCV  
Comparison between TensorFlow, PyTorch, Theano and OpenCV  
Call other functions from main()  
Call and then run your own functions and modules in different/other Python files  
Open an application window through search at start  
Open a new tab in an application window  
Launch the existing opened application if there is or start a new one if there is not  
Open and close any type of files with default programs/apps (e.g. word, excel, dm3, dm4, Digital Micrograph, powerpoint, internet explorer, chrome, and so on) in windows  
Move the cursor/mouse to the found, similar spots one-by-one  
Selection between choices or options  
Turn on and off with mouse press or a process  
Copy text into clipboard and then use it immediately (one time clipboard)  
Image matching with cross correlation and overlap of template edge. In this matching process, Normalized cross-correlation with those edge images is performed.  
Get a specific output for every input  
Option/selection/choice methods ("pop-up windows of Yes and No ")  
Swap the order of arguments in a function  
Access variable inside and outside a function  
Prevent other applications to modify the content until other Python script runs  
Open webpages in internet browsers (Chrome, Microsoft edge, IE browser)  
Open folders and explorer  
Open any files, e.g. text (.txt), image files, and so on. .txt file will be opened in Python program, e.g. IDLE shell.  
Open and close specific files  
Load/launch/open images and ColorMixing in DigitalMicrograph  
dropdown box/option/selection/choice  
Randomly open an image/file in a specific folder  
Work with (e.g. open) all files and subfolders/subdirectory in a folder  
Infinite loops (e.g. stop infinite cycling of opening the same images)  
Get/list immediate subdirectories/subfolders; get only the last part of a path  
Methods to open google chrome (problems: Google chrome closes immediately after being launched with selenium)  
Find a specific word in a webpage and count occurrences  
Run multiple Python files/scripts one after another  
Move(remove) all files from original folder in a directory to a new directory  
Subtract/minus one image from another image  
Global access to a variable inside a function from outside of the function  
Resize and then sum/mix/overlap two images (w/o transparency)  
Send emails through outlook  
Merge/combine two pptx files into one  
Count occurrence/nubmer of words/phrase in a text file  
Pass variables between functions/from one to another  
Stack/overlap wafer bin map  
One-dimensional neural network  
Loss (risk, cost, objective) function  
"True label" ("observed label") in machine learning  
Print the files and keyword occurrence which have been searched from a ppt file  
Calculating the area fraction of each circle overlapped by a square grid and build wafer map  
Check if rectangles overlap  
Check if two circles intersect or overlap  
Select/skip columns by index in DataFrame without changing the DataFrame itself, and change the order of of the selected columns  
Get the frequency of occurrence of a string in a column DataFrame  
Overfitting in machine learning  
Read outlook messages in .msg format  
Inside/outside edges/margins of plotted images  
y axis values are not ordered (disordered)  
Sort dates/year/month by order  
Avoid two or multiple plots being wrongly/incorrectly/unnecessarily mixed/overlap  
OverflowError (too large to store)  
Build own/customized keyword candidates  
ODBC (Open Database Connectivity)  
Overfitting and underfitting  
Send a variable from one script (back) to another script with a function  
Input a sentence and then output a sentence based on a dictionary obtained from csv  
Operators  
Python modules to interact with the operating system (os, platform, subprocess, shutils, glob and sys)  
OSError  
sort_values(by=... ascending/descending order)  
PermissionError (E.g. file cannot be written when it is open)  
Merge/append rows/columns of a csv file into an old csv file if the rows/columns are not in the old csv file  
Selecting only numeric/number columns, and then select two specific columns for plot  
Sort DataFrame by dates/year/month order  
Hide x-axis tick labels (only show some labels) where x values are under certain conditions  
Generating heatmaps for grouped data occurrences  
.onScreen()  
Observer()  
os.path.splitext()  
observer.start()  
os.path.exists()  
os.rename(source, dest)  
os.startfile()  
File name, folder name. {}{}....format. Manipulation of file and folder names (rename file name and folder name): i) Create a new folder and then copy all files from a folder to the new folder and rename the file, and then open the file. If the folder exists, then no file will be copied, but the file will still be opened. ii) Print and export the folder names and file names (with or without extensions) from a folder into a text file. iii) csv2image filename.  
Open an image to file from URL (webpage), then it can be saved in PC code, code. open with color changed.
Set the output image to zero everywhere except my mask (color filter), and display red, green, and blue (RGB) channels of an image. code, code.
Show/open images in any image viewer code, code.
DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order
3 types of logical operators/booleans
Represent one of two values: True or False. code. code. code.
or    /  |    (or yliao) The output is 'true', if any of the conditions are 'true'. Any condition may be true for the statement to be true. "or" is binary, but not is unary. Introduction. Example code. code.
on_any_event(event) Catch-all event handler. event (FileSystemEvent) – The event object representing the file system event. (code)
on_created(event)

Called when a file or directory is created. event (DirCreatedEvent or FileCreatedEvent) – Event representing file/directory creation. (code)

on_deleted(event)

Called when a file or directory is deleted. event (DirDeletedEvent or FileDeletedEvent) – Event representing file/directory deletion. (code)

on_modified(event)

Called when a file or directory is modified. event (DirModifiedEvent or FileModifiedEvent) – Event representing file/directory modification. (code)

on_moved(event) Called when a file or a directory is moved or renamed. event (DirMovedEvent or FileMovedEvent) – Event representing file/directory movement. (code)
watchdog.observers.api: Immutables  
class watchdog.observers.api.ObservedWatch(path, recursive)

Parameters: path – Path string. recursive – True if watch is recursive; False otherwise.

class watchdog.observers.api.ObservedWatch(path, recursive)

Parameters: path – Path string. recursive – True if watch is recursive; False otherwise.

class watchdog.observers.api.EventQueue(maxsize=0)
class watchdog.observers.api.EventEmitter(event_queue, watch, timeout=1)
class watchdog.observers.api.EventDispatcher(timeout=1)
class watchdog.observers.api.BaseObserver(emitter_class, timeout=1)

watchdog.observers.api.EventDispatcher

Base observer.
watchdog.observers.Observer alias of InotifyObserver
class watchdog.observers.polling.PollingObserver(timeout=1)
watchdog.observers.api.BaseObserver Platform-independent observer that polls a directory to detect file system changes.
class watchdog.observers.polling.PollingObserverVFS(stat, listdir, polling_interval=1)
watchdog.observers.api.BaseObserver File system independent observer that polls a directory to detect changes.
__or__  
os.path.join(path Get a file path. Introduction. code. code. (code)
ord() Returns an integer representing Unicode code point for the given Unicode character. code. code
oct() Convert an integer number (of any size) to an octal string prefixed with “0o”
Objects
Introduction.
open() Filehandler = open(fileString[, mode]). FileString is a literal or variable, containing the file (and a relative/absolute path, if needed). mode is a string indicating how to open the file. By default, opens in read mode (if no specify).
'r' ---read-only mode. Introduction.
'w' ---write mode. E.g. append a new row or a new line to an existing csv file by deleting an original row/line. Introduction.
'a' ---append mode. E.g. append a new row or a new line to an existing csv file without deleting the original rows/lines. Introduction.
'r+' ---special read/write mode Introduction.
Adds text to the end
code, code. .txt. Create a file with such as name if there is not a such file, or add a line in the file if there is a file with such a file name: .txt.
with open() Introduction. code.
OS module OS module in Python provides functions for interacting with the operating system. OS comes under Python’s standard utility modules. This module provides a portable way of using operating system dependent functionality. os.path module is submodule of OS module in Python used for common pathname manipulation.
os.listdir() Introduction. Output the file list of images in a folder, but only returns the names. code. code. (code).
os.path.splitext() method Is used to split the path name into a pair root and ext. Here, ext stands for extension and has the extension portion of the specified path while root is everything except ext part. ext is empty if specified path does not have any extension. If the specified path has leading period (‘.’), it then will be ignored. Introduction. Examples are:
path name root ext
/Desktop/file.txt /home/User/Desktop/file .txt /Desktop /home/User/Desktop {empty}
file.py file .py
.txt .txt {empty}
os  Miscellaneous operating system interfaces. code1, code2.
os.chdir code
os.getcwd() Get current working directory/path.
win32clipboard.OpenClipboard() code.
outer(a, b[, out]) Compute the outer product of two vectors.
Image.open PIL.Image.open() Opens and identifies the given image file. code. image. code. code.
numpy.ones(shape, dtype=None, order='C', *, like=None) Return a new array of given shape and type, filled with ones. General, code.
Count duplicates/occurrence and show unique values in csv files CSV: Introduction
Search/print/output the rows CSV: Print the rows if their cell values are greater than a specific value, in the csv file with numbers only; output the rows if the cell value is in a specific range. Introduction. code.
hotkey('o') Introduction
.Popen() Opens a pipe from a command. This pipe allows the command to send its output to another command. Introduction
os.walk() (code).
cv2.imread(/path/to/image, flag) A method loads an image from the specified file. If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format) then this method returns an empty matrix. imread() decodes the image into a matrix with the color channels stored in the order of Blue, Green, Red and A (Transparency) respectively: (:, :, 0) represents Blue channel; (:, :, 1) represents Green channel; (:, :, 2) represents Red channel; (:, :, 3) represents Transparency channel. The flag is optional. code. code. code. code.
v2.rectangle(image, start_point, end_point, color of border line, border thickness) border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code.
Get dimensions (sizes) of image:
dimensions = img.shape
Get height, width, number of channels in image
height = img.shape[0]
width = img.shape[1]
channels = img.shape[2]
General, code. code. code. code. code. code.
cv2.circle() cv2.circle(image, center_coordinates, radius, color, thickness). code. code.
cv2.EVENT_FLAG_LBUTTON Mouse callback function with a single left click. code.
cv2.EVENT_LBUTTONDOWN Mouse callback function with a single left click. code. (code)
cv2.EVENT_FLAG_MBUTTON Mouse callback function with a single left click. code.
cv2.EVENT_LBUTTONUP Mouse callback function with a single left click. code.
cv2.EVENT_LBUTTONDBLCLK Mouse callback function with double left clicks. code.
cv2.EVENT_RBUTTONDOWN Mouse callback function with a single right click. code.
cv2.EVENT_RBUTTONUP Mouse callback function with a single right click. code.
cv2.EVENT_FLAG_RBUTTON Mouse callback function with a single right click. code.
cv2.EVENT_FLAG_CTRLKEY Mouse callback function with double right clicks. code.
cv2.EVENT_MBUTTONDOWN Mouse callback function with the single middle mouse click. code.
cv2.EVENT_MBUTTONUP Mouse callback function with the single middle mouse click. code.
cv2.EVENT_MOUSEMOVE Mouse move. code.
zip() Is a container of all such points and it will iterate to all such points and draw rectangle around this closed entity i.e. coins: code.
cv2.matchTemplate Returns a correlation map, essentially a grayscale image. Other than contour filtering, matching keypoints, contour detection and processing (with thresholding, edge detection, etc. to generate a binary image), template matching is arguably one of the most simple forms of object detection (only 2-3 lines of code), which can detect multiple instances of the same/similar object in an input image. This method quickly fails when there are unknown changes of rotation, scale, viewing angle, etc. In those cases, you should use dedicated object detectors including HOG + Linear SVM, Faster R-CNN, SSDs, YOLO, etc. code. code. code. code. code.
Limitations: The matching can fail (if there is no special treatments in the script) if the size of the template is substantially smaller than the feature in the image being searched.
cv2.Canny() Canny Edge Detection is a popular edge detection algorithm. This detection is susceptible to noise in the image, so that first step is to remove the noise in the image with a 5x5 Gaussian filter. Code. code. code.
(min_val, max_val, min_loc, max_loc) = cv2.minMaxLoc() Returns the max and min intensity values as an array
that includes the location of these intensities. Takes the correlation result and returns a 4-tuple which includes the minimum correlation value, the maximum correlation value, the (x, y)-coordinate of the minimum value, and the (x, y)-coordinate of the maximum value, respectively. Max_Val is the location with the highest intensity in the image, corresponding to the best matching input image with regard to the defined template.. code. code. code.
cv2.TM_SQDIFF() method=CV_TM_SQDIFF
cv2.TM_SQDIFF_NORMED method=CV_TM_SQDIFF_NORMED
cv2.TM_CCORR method=CV_TM_CCORR
cv2.TM_CCORR_NORMED method=CV_TM_CCORR_NORMED
cv2.TM_CCOEFF_NORMED() method=CV_TM_CCOEFF_NORMED
The third parameter here is the method used for matching. code, code. code.
cv2.TM_CCOEFF method=CV_TM_CCOEFF
cv2.add code.
cv2.imwrite Save images. code. code.
cv2.resize Resizing does only change the width and height of the image. code.
cv2.waitKey(0) Will display the window infinitely until any keypress (it is suitable for image display). Therefore, if you use waitKey(0) you see a still image until you actually press something. code. code. code. code.
cv2.waitKey(1)

Will display a frame for 1 ms, after which display will be automatically closed. Since the OS has a minimum time between switching threads, the function will not wait exactly 1 ms, it will wait at least 1 ms, depending on what else is running on your computer at that time.

0xFF == ord('q') = 11111111 (a 8 bit binary), meaning "taking keyboard input here as 'q'". code
27 code
cv2.setMouseCallback Mouse clicks: introduction. code. code.
cv2.destroyAllWindows() Simply destroys all the windows we created. code. code.
cv2.destroyWindow() To destroy any specific window with the exact window name.
cv2.imshow A method is used to display an image in a window. The window automatically fits to the image size. First argument is a window name which is a string. Second argument is our image. code. code. Code.
cv2.cvtColor() Is used to convert an image from one color space to another. code.
cv2.COLOR_BGR2GRAY code.
.rgb2gray() code.
"gray" Convert a GRB image to a grey image. code.
cv2.THRESH_BINARY If pixel intensity is greater than the set threshold, value set to 255, else set to 0 (black). code. code
cv2.THRESH_BINARY_INV Inverted or Opposite case of cv2.THRESH_BINARY. code. code
cv2.line() Draw a line in a image. code
cv2.arrowedLine() cv2.arrowedLine(image, start_point, end_point, color[, thickness[, line_type[, shift[, tipLength]]]]) is used to draw arrow segment pointing from the start point to the end point. The parameters of the cv2.arrowedLine function are the same as those for cv2.line. code.
cv2.putText()

cv2.putText(image, text, org, font, fontScale, color[, thickness[, lineType[, bottomLeftOrigin]]]) is used to draw a text string on any image. The font types are FONT_HERSHEY_SIMPLEX = 0, FONT_HERSHEY_PLAIN = 1, FONT_HERSHEY_DUPLEX = 2, FONT_HERSHEY_COMPLEX = 3, FONT_HERSHEY_TRIPLEX = 4, FONT_HERSHEY_COMPLEX_SMALL = 5, FONT_HERSHEY_SCRIPT_SIMPLEX = 6, FONT_HERSHEY_SCRIPT_COMPLEX = 7, and FONT_ITALIC = 16. The thickness of the line is in pixel. lineType: This is an optional parameter.It gives the type of the line to be used. bottomLeftOrigin: This is an optional parameter. When it is true, the image data origin is at the bottom-left corner. Otherwise, it is at the top-left corner. code. (code)

cv2.namedWindow code.
cv2.moveWindow Set the position (coordinates) of the opened window. code.
cv2.IMREAD_UNCHANGED = -1. reads the image as is from the source. If the source image is an RGB, it loads the image into array with Red, Green and Blue channels. If the source image is ARGB, it loads the image with three color components along with the alpha or transparency channel.
cv2.IMREAD_GRAYSCALE Reads the image as grey image. If the source image is color image, grey value of each pixel is calculated by taking the average of color channels. = 0, code.
cv2.IMREAD_COLOR Reads the image with RGB colors but no transparency channel. = 1
cv.THRESH_TRUNC If pixel intensity value is greater than threshold, it is truncated to the threshold. The pixel values are set to be the same as the threshold. All other values remain the same. code. code
cv.THRESH_TOZERO Pixel intensity is set to 0, for all the pixels intensity, less than the threshold value. code. code
cv.THRESH_TOZERO_INV Inverted or Opposite case of cv2.THRESH_TOZERO. code. code
cv::IMREAD_ANYDEPTH = 2  
cv::IMREAD_ANYCOLOR = 4  
cv::IMREAD_LOAD_GDAL = 8  
cv::IMREAD_REDUCED_GRAYSCALE_2 = 16  
cv::IMREAD_REDUCED_COLOR_2 = 17  
cv::IMREAD_REDUCED_GRAYSCALE_4 = 32  
cv::IMREAD_REDUCED_COLOR_4 = 33  
cv::IMREAD_REDUCED_GRAYSCALE_8 = 64  
cv::IMREAD_REDUCED_COLOR_8 = 65  
cv::IMREAD_IGNORE_ORIENTATION = 128  
cv::IMWRITE_EXR_TYPE_HALF = 1  
cv::IMWRITE_EXR_TYPE_FLOAT = 2  
cv::IMWRITE_JPEG_QUALITY = 1  
cv::IMWRITE_JPEG_PROGRESSIVE = 2  
cv::IMWRITE_JPEG_OPTIMIZE = 3  
cv::IMWRITE_JPEG_RST_INTERVAL = 4  
cv::IMWRITE_JPEG_LUMA_QUALITY = 5  
cv::IMWRITE_JPEG_CHROMA_QUALITY = 6  
cv::IMWRITE_PNG_COMPRESSION = 16  
cv::IMWRITE_PNG_STRATEGY = 17  
cv::IMWRITE_PNG_BILEVEL = 18  
cv::IMWRITE_PXM_BINARY = 32  
cv::IMWRITE_EXR_TYPE = (3 << 4) + 0  
cv::IMWRITE_WEBP_QUALITY = 64  
cv::IMWRITE_PAM_TUPLETYPE = 128
 
cv::IMWRITE_TIFF_RESUNIT = 256
 
cv::IMWRITE_TIFF_XDPI = 257
 
cv::IMWRITE_TIFF_YDPI = 258
 
cv::IMWRITE_TIFF_COMPRESSION = 259  
cv::IMWRITE_PAM_FORMAT_NULL = 0  
cv::IMWRITE_PAM_FORMAT_BLACKANDWHITE = 1  
cv::IMWRITE_PAM_FORMAT_GRAYSCALE = 2  
cv::IMWRITE_PAM_FORMAT_GRAYSCALE_ALPHA = 3  
cv::IMWRITE_PAM_FORMAT_RGB = 4
 
cv::IMWRITE_PAM_FORMAT_RGB_ALPHA = 5  
cv::IMWRITE_PNG_STRATEGY_DEFAULT = 0  
cv::IMWRITE_PNG_STRATEGY_FILTERED = 1  
cv::IMWRITE_PNG_STRATEGY_HUFFMAN_ONLY = 2  
cv::IMWRITE_PNG_STRATEGY_RLE = 3  
   
   
cv::IMWRITE_PNG_STRATEGY_FIXED = 4