Electron microscopy
Python Automation and Machine Learning for ICs: Chapter D
- Python Automation and Machine Learning for ICs -
- An Online Book: Python Automation and Machine Learning for ICs by Yougui Liao -
Python Automation and Machine Learning for ICs                                                               http://www.globalsino.com/ICs/        

Table of Contents/Index 
Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix



Dropout in ML  Introduction
ML model complexity versus dataset size Introduction
Four/five V of big data Introduction
Data scaling and startup Introduction
Characteristics of semi-structured data Introduction
Feature Selection: Chi Square to select dependent and independent variables Introduction
Data Ingestion in Apache Spark  Introduction
Deploy modes for driver process in Apache Spark: client mode and cluster mode Introduction
Spark driver program Introduction
Cheatsheet of PySpark (for SparkSQL) and DataFrames Introduction
Apache Data Ingestion Frameworks (ADIF) for CSV to DataFrame Conversion  Introduction
User-Defined Schema (UDS) for Domain-Specific Languages (DSL) and Structured Query Language (SQL) Introduction
Coarsening data Introduction
Comparison btween data lake and data warehouse  Introduction
Understanding and extracting insights from unstructured data with ML Introduction
Decision threshold in ML  Introduction
Data selection in ML for semiconductor manufacturing processes Introduction
Data warehousing Introduction
Drawbacks of Coursera classes Introduction
Dataset in Apache Spark Introduction
Directed Acyclic Graph (DAG) Introduction
RDD (Resilient Distributed Dataset) Introduction
Parallel computing and distributed computing Introduction
Comparison between RDBMS (Relational Database Management Systems) and Apache Hive Introduction
HDFS (Hadoop Distributed File System) Introduction
Data Storage in Hadoop (HDFS, HBase and YARN) Introduction
Machine learning versus data science Introduction
Martin Zinkevich's "Rule of Machine Learning": dataset quality Introduction
Exploratory data analysis (EDA) Introduction
Data quality tools Introduction
Troubleshooting/debugging and problem solving in Python programming Introduction
Default mutable argument Introduction
Decompositions in Python programming Introduction
Labor cost of data analysis with and without automation and ML techniques Introduction
Pipelines in Data Science Introduction
Joint probability Introduction
Independence (independent events) versus dependence (dependent events) in ML Introduction
Distributive Property Introduction
Manhattan distance Introduction
Depth-first search (DFS) Introduction
POMDP (Partially Observable Markov Decision Process) Introduction
Optimal value function in Markov Decision Process (MDP) Introduction
Formats of datasets for classification Introduction
 Stationary and Non-Stationary State Transitions in Markov Decision Process (MDP) Introduction
Finite-horizon MDP (Markov Decision Process) Introduction
Q-Learning with Function Approximation (Deep Q-Network - DQN) Introduction
Discretization in reinforcement learning Introduction
Markov Decision Process (MDP) Introduction
Cumulative Distribution Function (CDF) Introduction
Maximum Likelihood Estimation (MLE) of single Gaussian (normal) distribution Introduction
Anomaly detection Introduction
Density estimation algorithms in ML
Experiences of developing machine learning algorithms Introduction
Edge detection of images in neural network Introduction
Batch Gradient Descent (BGD), Stochastic Gradient Descent (SGD), Mini-Batch Gradient Descent, Batch Stochastic Gradient Descent, Momentum, (Adagrad, Adadelta, RMSprop), and Adam (Adaptive Moment Estimation) Introduction
Decorrelating models Introduction
Regular decision trees and decision trees with bagging Introduction
Bagging in decision trees Introduction
Dependently Identically Distributed/Correlated Identically Distributed Introduction
Ensemble of decision trees Introduction
Regularization techniques for decision trees Introduction
Linear correlation between two variables with Pearson Correlation Coefficient, Spearman Rank Correlation Coefficient, Kendall's Tau, Linear Regression, Coefficient of Determination and Correlation Ratio
Plot diagram/graph Introduction
kfp.dsl package versus pipelines and components Introduction
Data parallelism in distributed training Introduction
Misclassification loss in decision trees Introduction
Difference between estimation and approximation errors Introduction
Deterministic function Introduction
Assumptions related to distribution of data in ML Introduction
Example of ML debugging/diagnostic: Anti-Spam Introduction
Check (difference) whether or not a cell value in a column of a CSV file Matchs a value in a column of another CSV file, then do something: e.g. add a value to another column of a csv file  Introduction
Open datasets, and open-source tools and libraries for ML practice Introduction
  True Distribution Introduction
  Population Distribution Introduction
  Sample Distribution Introduction
  Distribution of θ (parameter distribution) Introduction
  Posterior distribution Introduction
Distribution of θ (parameter distribution) in ML Introduction
Tricks for learning three dimensional geometry Introduction
Train-dev-test split (training-validation-testing split: Ratio for splitting dataset into training, validation and test sets Introduction
Splitting a training dataset into different subsets Introduction
Representer theorem and its derivation
Comparison among classifier, hyperplane and decision boundary Introduction
Hyperplane/Decision Boundary in ML Introduction
Indicator function/Kronecker delta function Introduction
Hidden layer in deep learning neural network Introduction
DRAM applications and challenges in machine learning Introduction
Convolutional Layers (CONV) in Deep Learning Introduction
Fully Connected Layers (FC) in Deep Learning Introduction
Categorical distribution Introduction
Comparison between Poisson distribution, Gaussian (normal) distribution and logistic regression Introduction
Logistic regression versus Gaussian discriminant analysis Introduction
Bernoulli distribution Introduction
Discriminative algorithms versus generative models Introduction
Gaussian Discriminant Analysis (GDA) Introduction
Discriminative algorithms/discriminative models Introduction
Bayesian Probability, Bayesian Statistics (Distribution Over a Distribution), versus Bayesian Inference Introduction
Gaussian distribution and standard gaussian distribution (multivariate normal distribution) Introduction
Probability density function (PDF): comparisons between (normal (gaussian) distribution, uniform distribution, exponential distribution and poisson distribution) Introduction
Newton's method versus gradient descent Introduction
Independent and identically distributed (i.i.d./IID) Introduction
Kernel density estimation (KDE) Introduction
Algorithms for directly finding the global optimum Introduction
Direct optimization Introduction
Discretization error Introduction
Gradient descent algorithm (for updating θ) Introduction
Brute force discretization Introduction
Deviation Threshold in ML Introduction
Core Steps/Procedure/Designing of Machine Learning Introduction
Deviation Probability (Hoeffding Bound) Introduction
Actual Probability of Deviation Introduction
Joint probability distribution Introduction
ODBC (Open Database Connectivity) Introduction
Python drivers for SQL server (pyodbc, pymssql, PyMySQL, cx_Oracle) Introduction
Access and use SQL Database on SSMS (Microsoft SQL Server Management Studio Express) with pyodbc: localhost, insert rows, update, count updated, delete rows, comparision between extract data by Python and SQL itself Introduction
Insert data/row into SQL Database on SSMS with pyodbc Introduction
Check if a key exists in a dictionary Introduction
Get the latest/newest/most recent file in a folder within certain time/days Introduction
Plot data into the same graph/figure/image from different csv files Introduction
Plot data into the same graph/figure/image from different csv files Introduction
Linear Discriminant Analysis Introduction
Correlations/similarity/dissimilarity of two columns in csv data Introduction
Plot from dictionary Introduction
Plot multiple datasets on the same scatter graph with different x- and y-axis values Introduction
Plot multiple datasets on the same scatter graph with different x- and y-axis values Introduction
Sort dates/year/month by order Introduction
Lock a file to prevent deleting, and then release the file once job is done Introduction
Two-dimensional neural network Introduction
Regression tree/decision tree for regression Introduction
Classification tree/decision tree for classification Introduction
Batch gradient descent Introduction
Large and small datasets in ML Introduction
DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order
Add an item to a dictionary Introduction
Creates a dictionary from a csv file Introduction
Input a sentence and then output a sentence based on a dictionary obtained from csv Introduction
IDLE (integrated development and learning environment) and integrated development environment (IDE) Introduction
Check if a variable does exist/is assigned/defined Introduction
Find common/different elements/items between two lists/sets Introduction
Trick: Get coordinate difference between mouse positions Introduction
Check if all the (and how many, length of a string) characters in the text are digits/numbers Introduction
Create dictionary from nested (sublist) list and get the values with keys Introduction
Compute the similarity between two text documents/files Introduction
Clustering versus Classification of texts and documents Introduction
Sentence, text and document embeddings Introduction
Get directory/path/file name partially Introduction
Call and run another script in a different/any (parent or children) directory/path/subfolder from a script Introduction
Get the current directory/folder path Introduction
Change the current directory to any directory/path (e.g. with os.chdir) Introduction
Loop through a Python dictionary Introduction
Avoid duplicates when creating text file Introduction
Copy a file or all files (with os.mkdir) to save to somewhere (create a directory first if it does not exist) Introduction
Delete the column/row in a CSV file if they are empty or less than a number (or header/index only) Introduction
sort_values(by=... ascending/descending order) Introduction
Data Augmentation Introduction
One-dimensional neural network Introduction
Compare dates (x days after or before a date), and difference between two dates in days Introduction
Merge dictionaries (update(), **, chain(), ChainMap(), |, |=) Introduction
Check if a file exists again (double check) Introduction
Create a temporary file or directory/folder Introduction
Download Youtube vedio Introduction
RegEx (Regular Expression) (characters to check if a string contains a specified search pattern, remove double spaces, and clean texts) Introduction
Extract any substrings with any pattern (e.g. dot (.)) Introduction
Difference/comparison between real mouse click and click from script/program, e.g. Pyautogui
Different behavior of automation execution (e.g. pyautogui) locally or remotely through internet
Fréchet Inception Distance (FID) coefficient Introduction
year, month, date, hour, minute, and second: "datefmt='%Y-%m-%d %H:%M:%S')": year, month, date, hour, minute, and second Instruction
Plot figures with date/month/year Introduction
Change date/month/year format Introduction
Find and convert the file time/date Introduction
Last n days/weeks/months (.to_datetime(x), .set_index(y), .last(z), .reset_index(), and .max() in pandas) Introduction
Time and date used as a file/folder name stamp (e.g. duplicate a file in the same folder) Introduction
datetime.date: An idealized naive date, assuming the current Gregorian calendar always was, and always will be, in effect (code)
datetime.datetime: A combination of a date and a time. Attributes: year, month, day, hour, minute, second, microsecond, and tzinfo (code)
datetime.time: An idealized time, independent of any particular day, assuming that every day has exactly 24*60*60 seconds (code)
datetime.timedelta: A duration expressing the difference between two date, time, or datetime instances to microsecond resolution (code)
datetime.tzinfo: An abstract base class for time zone information objects (code)
datetime.timezone: A class that implements the tzinfo abstract base class as a fixed offset from the UTC (code)
datetime.strftime(format): Return a string representing the date and time, controlled by an explicit format string (code)
.date.today(): Get current date: code (in different formats) and code. (code)
date_parser: function to use to parse strings into datetime objects. If parse_dates is True, it defaults to the very robust dateutil.parser. Specifying this implicitly sets parse_dates as True. You can also use functions from community supported date converters from date_converters.py  
date_format: Format string for datetime objects  
dayfirst: if True then uses the DD/MM international/European date format (This is False by default)  
datetime.__format__(format): Same as datetime.strftime() (code)
Get the date and time (a past date) of N days ago Introduction
DataFrame cheatsheet/basics: number of rows and columns, value of specific single cell, scatter plots,   Introduction
DataFrame and DataFrame templates Introduction
DataFrame.from_csv(): from_csv(path, header=0, sep=', ', index_col=0, parse_dates=True, encoding=None, tupleize_cols=False, infer_datetime_format=False): Read CSV file. It is preferable to use the more powerful pandas.read_csv() for most general purposes, but from_csv makes for an easy roundtrip to and from a file, especially with a DataFrame of time series data.  
Remove duplicate cell values from a csv file/dataframe (keeping the first/top one) Introduction
Get header/column names from DataFrame Introduction
Convert dataframe row/column into a comma separated string Introduction
Iterate over rows in a DataFrame/read and print row-by-row (number of columns and rows, df.shape[0]/df.shape[1]) Introduction
Cheatsheet of PySpark (for SparkSQL) Introduction
Convert all elements of specific column or in entire dataframe into strings Introduction
Compare string entries/cells/elements of columns in different dataframes
Create table on pptx with certain rows and columns in DataFrame Introduction
Check and drop negative from dataframe pandas Introduction
DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order
csv workflow: Read into dataframeSelect a specific column from DataFrame, Select several specific columns to plot 
Plot workflow: Create new empty column/row in DataFrameMove the cells in a column to another column under certain conditionSelect specific columns for scatter plot
Skip/remove empty rows (row-by-row) in DataFrame/csv Introduction
Remove/delete the duplicated/same rows in dataframe::>> df_unique = df.drop_duplicates()  
Extract a subset of DataFrame from a DataFrame Introduction
Select specific columns from a DataFrame to form a new DataFrame Introduction
Output the row into dataframe if the value of the cell in a column contains a specific substring in a csv file (with headers) Introduction
Extract a SubDataFrame from A DataFrame Introduction
Merge columns with character/symbol Separation Introduction
Extract the last column as subdataframe Introduction
Plot graph/figure/image from CSV file/DataFrame Introduction
Plot graph/figure/image from CSV file/DataFrame by removing/hiding blank/empty cells with axis range (plt.xlim()) Introduction
Skip/replace empty cells from DataFrame/CSV file Introduction
Separately plot data into the same graph/figure/image from different csv files for each category (import multiple CSV files and concatenate into one DataFrame): append row-by-row or column-by-column Introduction
Convert csv/dataframe column to a list or vice versa Introduction
Remove unwanted/unnecessary parts from strings in a column of dataframe Introduction
Select/skip columns by index in DataFrame without changing the DataFrame itself, and change the order of of the selected columns Introduction
Find the same elements in columns in two separate dataframes and then merge them Introduction
Remove rows if (multiple) NaN is more than a number in DataFrame Introduction
Read columns with numeric values/numbers only in dataframe Introduction
Get the frequency of occurrence of a string in a column DataFrame Introduction
Compare (pattern/ratio of) two different columns, check whether column values match in DataFrame Introduction
Check whether one column contains number only and another column contains letters only or mixture of numbers and letters in DataFrame Introduction
Check the difference between two columns in DataFrame Introduction
Calculations in DataFrame: Add a column, calculate for a new column, delete a column, all the rows with values greater than 30 in "Score A" column Introduction
.DataFrame(): .drop(), index, columns, axes, dtypes, size, shape, ndim, empty, T, values Introduction
Apply a formatting function to all cells in a DataFrame Introduction
DataFrame.equals(): Confirm if the two csv files are the same or not code
Handle NaN value in DataFrame, replace empty cells with ...
Print/remove/delete specific rows of a DataFrame Introduction
Pickling and unpickling of pandas DataFrame (.pkl) Introduction
Store images in pandas dataframe column Introduction
Print/remove/delete specific rows of a DataFrame Introduction
Write contents of DataFrame/memory into text file Introduction
Convert DataFrame to a HTML Table and save as a HTML webpage Introduction
Sort DataFrame by dates/year/month order Introduction
Detection of tables from an image Introduction
Data cleaning examples in csv files Introduction
Remove decimal part in a string with comma Introduction
Form a list of strings from an old string with all the 6 digits by removing all special characters or spaces Introduction
Filters the rows based on the condition of being within n days of today's date Introduction
__add__, __call__, __contains__, __delitem__, __delattr__, __eq__, __enter__, __ge__, __getattribute__, __getnewargs__, __getattr__, __getitem__, __gt__, __hash__, __reduce__, __iadd__, __imul__, __init_subclass__, __index__, __int__, __invert__, __new__, __neg__, __reduce_ex__, __reversed__, __rmul__, __radd__, __rand__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __ror__, __round__,__rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __rxor__, __dir__, __doc__, __divmod__, __iter__, __le__, __lt__, __len__, __ne__, __repr__, __setattr__, __setitem__, __sizeof__, __lshift__, __sub__, __subclasshook__, __str__ Introduction
Delete a single file Introduction
Duplicate/repeat the same words/elements in a string/list Introduction
Comparison between decision tree, random forest and XGBoost (extreme gradient boosting) Introduction
Dimensionality reduction Introduction
Convert a floating-point number (decimal) to exponential format Introduction
Plot images from different DataFrame in a single row Introduction
Partial (e.g. first) portion of headers of DataFrame Introduction
Only use the first 4 characters in the headers of the table for pptx/dataframe Introduction
Search/extract all the 4-digit numbers (with and without extension) from a given text Introduction
Cheatsheet about headers (column names) in DataFrame Introduction
Generate defect data of a wafer map Introduction
Aggregate duplicates in columns of data Introduction
Reindex a DataFrame Introduction
Adjusting space around the plot with dummy categories Introduction
Save dynamic graph as a movie/video or split a movie to image frames Introduction
Comparative overview of multivariate statistical methods (Correlation Analysis, Regression Analysis, Factor Analysis, Cluster Analysis, Principal Component Analysis (PCA), Canonical Correlation Analysis, Discriminant Analysis, Path Analysis, Structural Equation Modeling (SEM), Multivariate Analysis of Variance (MANOVA), Analysis of Covariance (ANCOVA) ): purposes, variables, and outputs Introduction
Software/interface used in data science and machine learning Introduction
Aggregation functions in data manipulation and database queries Introduction
Cheatsheet of dictionary Introduction



del     Introduction
delim_whitespace CSV: Parse whitespace-delimited (spaces or tabs) file (much faster than using a regular expression)
dialect CSV: string or csv.Dialect instance to expose more ways to specify the file format. To expose more ways to specify the file format. code.
dtype CSV: A data type name or a dict of column name to data type. If not specified, data types will be inferred.
(Unsupported with engine=’python’)

CSV: Write the contents of the Frame into a CSV file. If no path is given, then the Frame will be serialized into a string, and that string will be returned.

drop() Delete a column. CSV: (code)
drop_duplicates() CSV: (code)
doublequote CSV: Control quoting of quotechar in fields (default True)
.duplicated() CSV: Determines which elements of a vector or data frame are duplicates of elements with smaller subscripts, and returns a logical vector indicating which elements (rows) are duplicates. (code)
display.max_rows CSV: Sets the maximum number of rows pandas should output when printing out various output. (code)
display.max_columns CSV: Sets the maximum number of columns pandas should output when printing out various output. (code)
dropna() CSV: Allows the user to analyze and drop Rows/Columns with Null values in different ways. code.
sep or delimiter CSV: A delimiter / separator to split fields on. The delimiter can be tab, comma, space, semicolon, etc. With sep=None, read_csv will try to infer the delimiter automatically in some cases by “sniffing”. The separator may be specified as a regular expression, e.g. ‘|\s*’ can be used to indicate a pipe plus arbitrary whitespace.
DictReader() CSV: csv.DictReader(f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds). Create an object that operates like a regular reader but maps the information in each row to a dict whose keys are given by the optional fieldnames parameter. DictReader uses the first row of csv file to setup keys for each dictionary value return for each line. code.
Dictionary Introduction, and data structures.
.doubleClick() .doubleClick(x=moveToX, y=moveToY) (Code)
drag() E.g. .drag(10, 0, 2, button='right') # drag the mouse left 10 pixels over 2 seconds while holding down the right mouse button. (code).
dragTo() dragTo(x, y, duration=num_seconds) drags mouse to XY. E.g. .dragTo(50, 20, button='left'): drag mouse to X of 50, Y of 20 while holding down left mouse button. .dragTo(100, 100, 2, button='left'): drag mouse to X of 100, Y of 100 over 2 seconds while holding down left mouse button. (code).
dragRel() dragRel(xOffset, yOffset, duration=num_seconds) drags mouse relative to its current position. (code).
.displayMousePosition() (code)
delete drop(), del
Remove an item from a dictionary by key or value: Code. Another example: del x; then print(x); shows error.
del() Delete an item from a list.
entry.delete(0, tk.END) Use the special constant tk.END for the second argument of .delete() to remove all text in an Entry
.destroy() (code). (code).
import DigitalMicrograph as DM GMS 3.4 has Python scripting capabilities with some libraries.
def         Is short for define. Introduction. code. code4. code5. code6. code7. code.
dict() Introduction. Declaration: d = dict(); d = {}; d = {'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}. Print dictionary: code.
from pptx.dml.color import RGBColor (code)
divmod() Outputs the quotient and the remainder in a tuple.
dis.dis() Introduction. dis.dis(x=None, *, file=None, depth=None): Disassemble the x object. x can denote either a module, a class, a method, a function, a generator, an asynchronous generator, a coroutine, a code object, a string of source code or a byte sequence of raw bytecode. code.
deepcopy() Creates a new object and recursively adds the copies of nested objects present in the original elements. code.
class watchdog.events.DirMovedEvent(src_path, dest_path) Bases: watchdog.events.FileSystemMovedEvent :: File system event representing directory movement on the file system.
class watchdog.events.DirModifiedEvent(src_path) Bases: watchdog.events.FileSystemEvent :: File system event representing directory modification on the file system.
class watchdog.events.DirCreatedEvent(src_path) Bases: watchdog.events.FileSystemEvent :: File system event representing directory creation on the file system.
class watchdog.events.DirDeletedEvent(src_path) Bases: watchdog.events.FileSystemEvent :: File system event representing directory deletion on the file system.

Dispatches events to the appropriate methods.
event (FileSystemEvent) – The event object representing the file system event.

dispatch(event) Dispatches events to the appropriate methods.
Parameters: event (FileSystemEvent) – The event object representing the file system event.
dispatch_events(event_queue, timeout) Override this method to consume events from an event queue, blocking on the queue for the specified timeout before raising queue.Empty.
Parameters: event_queue (EventQueue) – Event queue to populate with one set of events.
timeout (float) – Interval period (in seconds) to wait before timing out on the event queue.
Raises: queue.Empty
daemon A boolean value indicating whether this thread is a daemon thread (True) or not (False).
This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.
The entire Python program exits when no alive non-daemon threads are left.
class watchdog.utils.dirsnapshot.DirectorySnapshot(path, recursive=True, walker_callback=<function <lambda> at 0x7f78fbeed500>, stat=<built-in function stat>, listdir=<built-in function listdir>) A snapshot of stat information of files in a directory.
path (str) – The directory path for which a snapshot should be taken.
recursive (bool) – True if the entire directory tree should be included in the snapshot; False otherwise.
walker_callback –
Deprecated since version 0.7.2.
stat –
Use custom stat function that returns a stat structure for path. Currently only st_dev, st_ino, st_mode and st_mtime are needed.
A function with the signature walker_callback(path, stat_info) which will be called for every entry in the directory tree.
listdir – Use custom listdir function. See os.listdir for details.

class watchdog.utils.dirsnapshot.DirectorySnapshotDiff(ref, snapshot)

Compares two directory snapshots and creates an object that represents the difference between the two snapshots.
ref (DirectorySnapshot) – The reference directory snapshot.
snapshot (DirectorySnapshot) – The directory snapshot which will be compared with the reference snapshot.


List of directories that were created.


List of directories that were deleted.


List of directories that were modified.


List of directories that were moved.
Each event is a two-tuple the first item of which is the path that has been renamed to the second item in the tuple.

math.degrees() Converts angle x from radians to degrees.
dispatch(event) Dispatches events to the appropriate methods.
event (FileSystemEvent) – The event object representing the file system event.
Default values in functions Introduction
Default value for list Introduction
def __init__(self[, suppliedprop1, suppliedprop2,...]) The syntax for creating an init method. The self part is just a variable name, and it is used to refer to the object being created at the moment. You can use the name of your own choosing instead of self. code. code.
Directory path: code.
Integer/fractions/round /decimal/digits/floating
It does not have any fractional part. Introduction. int: Example code.
Decimal Those having fixed precision
scipy.linalg.dft Create a discrete Fourier transform matrix.
dot(a, b[, out]) Dot product of two arrays.
linalg.cholesky(a) Cholesky decomposition.
linalg.qr(a[, mode]) Compute the qr factorization of a matrix.
linalg.svd(a[, full_matrices, compute_uv, …]) Singular Value Decomposition.
linalg.det(a) Compute the determinant of an array.
__annotations__/dict name/type: parameter and return annotations
__defaults__/tuple name/type: default values for the formal parameters
__globals__/dict name/type: global variables of the module where the function is defined
__kwdefaults__/dict name/type: default values for the keyword-only formal parameters
v2.rectangle(image, start_point, end_point, color of border line, border thickness) border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code.
Get dimensions (sizes) of image:
dimensions = img.shape
Get height, width, number of channels in image
height = img.shape[0]
width = img.shape[1]
channels = img.shape[2]
General, code. code. code. code. code. code.
cv2.destroyAllWindows() Simply destroys all the windows we created. code. code.
cv2.destroyWindow() To destroy any specific window with the exact window name.
.apply(pd.Series) (code)
fillna() (code)
Split a list into columns Introduction
random.default_rng Random generator: General. code.
Count duplicates/occurrence and show unique values in csv files CSV: Introduction
Dash From exploring data to monitoring your experiments, Dash is like the frontend to the analytical Python backend. This productive Python framework is ideal for data visualization apps particularly suited for every Python user.
hotkey('d') Introduction
.hotkey('down') Introduction
.delete_rows() In excel files: (code)
.delete_cols In excel files: (code)
.getAllTitles() Get all the Python program windows, *IDLE Shell window, e.g. *IDLE Shell 3.9.5, the most front window on Dreamweaver, the most front webpage on each Chrome window, the name of each opened applications, e.g. DigitalMicrograph. Introduction
'<Double-1>' Double clicks. (code)
.DISABLED Disable a button. (code)
from collections import defaultdict (code).
split() Introduction. Split a string by dots, split a file name by dots, split a file name from its extension. code. (code).
driver.switch_to.default_content() Introduction.


Move the mouse and then release it. .mouseDown(x=moveToX, y=moveToY, button='left'). .click() function is just a convenient wrapper around these two .mouseDown() and .mouseUp() function calls. (code)
os.path.dirname() (code).
dir() code. (code).
DPI, or dots per inch Is a measure of the resolution of a printed document or digital scan. (code).
Methods of the mapping types dict, collections.defaultdict and collections.OrderedDict (common object methods omitted for brevity). Optional arguments are enclosed in «...»..
Methods Dict Default dict Ordered dict  
d.clear() remove all items
d.__contains__(k) k in d
d.copy() shallow copy
.copy()       Image overlap. Code. copy method.
d.__copy__()     support for copy.copy

callableinvokedby __missing__ toset missing values

d.fromkeys(it, «initial») new mapping from keys in iterable, with optional initial value (defaults to None)
d.get(k, «default»)

get item with key k, return default or none if missing

d.__getitem__(k) d[k] — get item with key k

get view over items — (key, val ue) pairs

d.__iter__() get iterator over keys
d.keys() get view over keys
d.__len__() len(d) — number of items

called when __getitem__ cannot find the key

d.move_to_end(k, «last»)    

movekfirstorlastposition(lastisTrue by default)

d.pop(k, «default») remove and return value at k, or de fault or None if missing

remove and return an arbitrary (key, value) itemb


get iterator for keys from last to first inserted

d.setdefault(k, «de fault») if k in d, return d[k]; else set d[k] = default and return it
d.__setitem__(k, v) d[k] = v — put v at k
d.update(m, «**kargs») update d with items from mapping or iterable of (key, value) pairs
d.values() get view over values
Line detection on image Introduction
Save contents (download pdf files) in the webpages obtained by Google search into a text file Introduction
Calibrate and put a scale bar, and draw a line segment on an image / detect a scale bar/scalebar calibration by clicking the start and end of the scale bar on desktop Introduction
Diversity prediction theorem Introduction
Examples of matplotlib (image/data) visualizations Introduction
Extract the least/most frequency/duplicate/occurrence element in a list Introduction
Three dimensional (3D) shapes/structures Introduction
cosine similarity/distance Introduction
Stochastic gradient descent (SGD) Introduction
Dot notation Introduction
Find duplicate items in a list Introduction
Dummy variables/binary variables Introduction
(Deep) neural network (NN or DNN) Introduction
Deploying process in ML Introduction
pd.get_dummies (Code)
.as_default() Introduction
tf.distribute.Strategy Introduction
tf.data.Dataset Introduction
Axis/dimension of tensor Introduction
Draw smiling face emoji Introduction
Decorators in Python Introduction
Databricks Introduction
Data science  
Data structures (Data science, and comparison between list, tuple, set, dictionary) Introduction
Ranking/most popular programming languages for data analysts Introduction
Critical thinking in data science Introduction
Call and then run your own functions and modules in different/other Python files Introduction
Double click of mouse Introduction, (Introduction)
Double click a specific position Introduction
Drag mouse Introduction
Lock desktop Introduction
Open the page of the downloaded list of webpage browser Introduction
DigitalMicrograph (DM)
Table of digital micrograph (DM) shortcut hotkeys Introduction
Automation of EELS data extraction from DigitalMicrograph Introduction
Positions of features on Gatan DigitalMicrograph on PC screen (introduction)
Load/launch images and ColorMixing in DigitalMicrograph Introduction
Check if all the (and how many, length of a string) characters in the text are digits/numbers (Code)
Work (read, write, insert and delete rows and columns, and merge and unmerge cells, shift/move cell values) in Excel sheets Introduction
Term Frequency-Inverse Document Frequency (TF-IDF) Introduction
Select/input a folder/directory/path for later to be called to use Introduction
Calibrate and put a scale bar, and draw a line segment on an image Introduction
Access elements in a dictionary and subdictionary Introduction
Build databases with different/uncertain number of members Introduction
Set default programs by file extensions and by file types and programs on Windows Introduction
Pint summary of the statistic data, change data format, sort/group columns Introduction
Draw an arrow segment pointing from the start point to the end point in an image. code.
Draw a line in a image code
Draw lines manually and then label them with arrows code
Choose a file with simple dialog. Code. code. code. With a default folder: code1 and code2. .
Copy images to a different folder/Save an image in a new folder code, code. code. code.
Data plot with labels code, code.
Calculate the coordinates of a point in a given rectangle and the distance of a given point to a line code
Compute the difference between two images by using Structural Similarity Index with "pip install --upgrade imutils" code. code
Reverse the digits of a given number code
Get the list of the methods for a function (e.g. using dir) Introduction
Check if a popup dialog is a window or not for Selenium app Introduction
dropdown box Introduction
Draw lines and arrows in a ppt Introduction
Remove an/all/duplicate item(s)/elements from a list Introduction
Count and delete slides from ppt Introduction
Work with all files and subfolders/subdirectory in a folder Introduction
Get/list immediate subdirectories/subfolders; get only the last part of a path Introduction
Move file(s) from one directory to another Introduction
Print/obtain a specific digit in a number, e.g. in a cell value in cvs Introduction
Remove an item/element/duplicates from a list Introduction
Get/list immediate subdirectories/subfolders; get only the last part of a path/folder/drive Introduction
Check/find/get a file name or the last folder name (e.g. from a path/directory) Introduction
Check if a file/folder/path/directory exists or not (Cannot find a specific file/folder?, or check files with extension) Introduction
Check if a directory is empty; find the position/index of a particular file/folder in a file directory/path; remove folder or file level by level (or layer by laer) from its directory/path Introduction
Circular dependencies in Python execution Introduction
Watchdog ignore/skip and take pattern in directory/path Introduction
Move(remove) all files from original folder in a directory to a new directory Introduction
Delete the entire directory and/or all the files in the directory/folder Introduction
List all files and directories which has specific files or files with specific extensions Introduction
Count how many (number) files and folders in a directory Introduction
Take a screenshot using a mouse click and drag method Introduction
Measure (real or pixel) length/distance on an image w/o calibrated bar Introduction
Crop/snip part of a image with definition by a pixel line Introduction
Modify file path/directory by changing folder names by merging a list Introduction
Plot distance between points calculated by coordinates Introduction
Deep learning Introduction
Manual analysis of data Introduction
Libraries used to convert incident documents into numerical vectors Introduction
Decision tree learning Introduction
Keyword extraction methods from documents in Natural Language Processing (NLP) Introduction
Ranking/most popular programming languages/tools for data analysts Introduction
Ranking and votes of essential/most important skills for data analysts Introduction
Remove duplicate/same lines in a text file Introduction
Ranking/most popular machine learning frameworks used by data scientists Introduction
Comparison of qualifications and skills between data science manager, engineering and scientist Introduction
Spatial defect patterns Introduction
Defect Detection and Classification by using Machine Learning Introduction
Human inspection of defects in wafer map Introduction
Automated defect scanning in wafer map Introduction
Global defects and local defects identified by defect denoising Introduction
Denoising/remove noise in images Introduction
(Deep) Convolutional neural networks (CNN) Introduction
Wafer map failure pattern recognition (WMFPR)/wafer failure pattern detection (WFPD)/defect classification Introduction
Public datasets for wafer map analysis Introduction
Methods of data and information visualization Introduction
Draw circles/lines on images Introduction
Root Cause Deconvolution (RCD) Introduction
Detection and classification of defective dies in wafer map
Detection procedures/processes of spatial defect patterns (bins) in wafers Introduction