| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Data integration |
Introduction |
| Dropout in ML |
Introduction |
| ML model complexity versus dataset size |
Introduction |
| Four/five V of big data |
Introduction |
| Data scaling and startup |
Introduction |
| Characteristics of semi-structured data |
Introduction |
| Feature Selection: Chi Square to select dependent and independent variables |
Introduction |
| Data Ingestion in Apache Spark |
Introduction |
| Deploy modes for driver process in Apache Spark: client mode and cluster mode |
Introduction |
| Spark driver program |
Introduction |
| Cheatsheet of PySpark (for SparkSQL) and DataFrames |
Introduction |
| Apache Data Ingestion Frameworks (ADIF) for CSV to DataFrame Conversion |
Introduction |
| User-Defined Schema (UDS) for Domain-Specific Languages (DSL) and Structured Query Language (SQL) |
Introduction |
| Coarsening data |
Introduction |
| Comparison btween data lake and data warehouse |
Introduction |
| Understanding and extracting insights from unstructured data with ML |
Introduction |
| Decision threshold in ML |
Introduction |
| Data selection in ML for semiconductor manufacturing processes |
Introduction |
| Data warehousing |
Introduction |
| Drawbacks of Coursera classes |
Introduction |
| Dataset in Apache Spark |
Introduction |
| Directed Acyclic Graph (DAG) |
Introduction |
| RDD (Resilient Distributed Dataset) |
Introduction |
| Parallel computing and distributed computing |
Introduction |
| Comparison between RDBMS (Relational Database Management Systems) and Apache Hive |
Introduction |
| HDFS (Hadoop Distributed File System) |
Introduction |
| Data Storage in Hadoop (HDFS, HBase and YARN) |
Introduction |
| Machine learning versus data science |
Introduction |
| Martin Zinkevich's "Rule of Machine Learning": dataset quality |
Introduction |
| Exploratory data analysis (EDA) |
Introduction |
| Data quality tools |
Introduction |
| Troubleshooting/debugging and problem solving in Python programming |
Introduction |
| Default mutable argument |
Introduction |
| Decompositions in Python programming |
Introduction |
| Labor cost of data analysis with and without automation and ML techniques |
Introduction |
| Pipelines in Data Science |
Introduction |
| Joint probability |
Introduction |
| Independence (independent events) versus dependence (dependent events) in ML |
Introduction |
| Distributive Property |
Introduction |
| Manhattan distance |
Introduction |
| Depth-first search (DFS) |
Introduction |
| POMDP (Partially Observable Markov Decision Process) |
Introduction |
| Optimal value function in Markov Decision Process (MDP) |
Introduction |
| Formats of datasets for classification |
Introduction |
| Stationary and Non-Stationary State Transitions in Markov Decision Process (MDP) |
Introduction |
| Finite-horizon MDP (Markov Decision Process) |
Introduction |
| Q-Learning with Function Approximation (Deep Q-Network - DQN) |
Introduction |
| Discretization in reinforcement learning |
Introduction |
| Markov Decision Process (MDP) |
Introduction |
| Cumulative Distribution Function (CDF) |
Introduction |
| Maximum Likelihood Estimation (MLE) of single Gaussian (normal) distribution |
Introduction |
| Anomaly detection |
Introduction |
Density estimation algorithms in ML
|
Introduction |
| Experiences of developing machine learning algorithms |
Introduction |
| Edge detection of images in neural network |
Introduction |
| Batch Gradient Descent (BGD), Stochastic Gradient Descent (SGD), Mini-Batch Gradient Descent, Batch Stochastic Gradient Descent, Momentum, (Adagrad, Adadelta, RMSprop), and Adam (Adaptive Moment Estimation) |
Introduction |
| Decorrelating models |
Introduction |
| Regular decision trees and decision trees with bagging |
Introduction |
| Bagging in decision trees |
Introduction |
| Dependently Identically Distributed/Correlated Identically Distributed |
Introduction |
| Ensemble of decision trees |
Introduction |
| Regularization techniques for decision trees |
Introduction |
Linear correlation between two variables with Pearson Correlation Coefficient, Spearman Rank Correlation Coefficient, Kendall's Tau, Linear Regression, Coefficient of Determination and Correlation Ratio
|
Introduction |
| Plot diagram/graph |
Introduction |
| kfp.dsl package versus pipelines and components |
Introduction |
| Data parallelism in distributed training |
Introduction |
| Misclassification loss in decision trees |
Introduction |
| Difference between estimation and approximation errors |
Introduction |
| Deterministic function |
Introduction |
| Assumptions related to distribution of data in ML |
Introduction |
| Example of ML debugging/diagnostic: Anti-Spam |
Introduction |
| Check (difference) whether or not a cell value in a column of a CSV file Matchs a value in a column of another CSV file, then do something: e.g. add a value to another column of a csv file |
Introduction |
| Open datasets, and open-source tools and libraries for ML practice |
Introduction |
| |
|
| Distribution |
|
| |
✔ |
True Distribution |
Introduction |
| |
✔ |
Population Distribution |
Introduction |
| |
✔ |
Sample Distribution |
Introduction |
| |
✔ |
Distribution of θ (parameter distribution) |
Introduction |
| |
✔ |
Posterior distribution |
Introduction |
| |
|
| Distribution of θ (parameter distribution) in ML |
Introduction |
| Tricks for learning three dimensional geometry |
Introduction |
| Train-dev-test split (training-validation-testing split: Ratio for splitting dataset into training, validation and test sets |
Introduction |
| Splitting a training dataset into different subsets |
Introduction |
Representer theorem and its derivation
|
Introduction |
| Comparison among classifier, hyperplane and decision boundary |
Introduction |
| Hyperplane/Decision Boundary in ML |
Introduction |
| Indicator function/Kronecker delta function |
Introduction |
| Hidden layer in deep learning neural network |
Introduction |
| DRAM applications and challenges in machine learning |
Introduction |
| Convolutional Layers (CONV) in Deep Learning |
Introduction |
| Fully Connected Layers (FC) in Deep Learning |
Introduction |
| Deep learning algorithms to enhance defect detection in semiconductor manufacturing |
Introduction |
| Categorical distribution |
Introduction |
| Comparison between Poisson distribution, Gaussian (normal) distribution and logistic regression |
Introduction |
| Logistic regression versus Gaussian discriminant analysis |
Introduction |
| Bernoulli distribution |
Introduction |
| Discriminative algorithms versus generative models |
Introduction |
| Gaussian Discriminant Analysis (GDA) |
Introduction |
| Discriminative algorithms/discriminative models |
Introduction |
| Bayesian Probability, Bayesian Statistics (Distribution Over a Distribution), versus Bayesian Inference |
Introduction |
| Gaussian distribution and standard gaussian distribution (multivariate normal distribution) |
Introduction |
| Probability density function (PDF): comparisons between (normal (gaussian) distribution, uniform distribution, exponential distribution and poisson distribution) |
Introduction |
| Newton's method versus gradient descent |
Introduction |
| Independent and identically distributed (i.i.d./IID) |
Introduction |
| Kernel density estimation (KDE) |
Introduction |
| Algorithms for directly finding the global optimum |
Introduction |
| Direct optimization |
Introduction |
| Discretization error |
Introduction |
| Gradient descent algorithm (for updating θ) |
Introduction |
| Brute force discretization |
Introduction |
| Deviation Threshold in ML |
Introduction |
| Core Steps/Procedure/Designing of Machine Learning |
Introduction |
| Deviation Probability (Hoeffding Bound) |
Introduction |
| Actual Probability of Deviation |
Introduction |
| Joint probability distribution |
Introduction |
| ODBC (Open Database Connectivity) |
Introduction |
| Python drivers for SQL server (pyodbc, pymssql, PyMySQL, cx_Oracle) |
Introduction |
| Access and use SQL Database on SSMS (Microsoft SQL Server Management Studio Express) with pyodbc: localhost, insert rows, update, count updated, delete rows, comparision between extract data by Python and SQL itself |
Introduction |
| Insert data/row into SQL Database on SSMS with pyodbc |
Introduction |
| Check if a key exists in a dictionary |
Introduction |
| Get the latest/newest/most recent file in a folder within certain time/days |
Introduction |
| Plot data into the same graph/figure/image from different csv files |
Introduction |
| Plot data into the same graph/figure/image from different csv files |
Introduction |
| Linear Discriminant Analysis |
Introduction |
| Correlations/similarity/dissimilarity of two columns in csv data |
Introduction |
| Plot from dictionary |
Introduction |
| Plot multiple datasets on the same scatter graph with different x- and y-axis values |
Introduction |
| Plot multiple datasets on the same scatter graph with different x- and y-axis values |
Introduction |
| Sort dates/year/month by order |
Introduction |
| Lock a file to prevent deleting, and then release the file once job is done |
Introduction |
| Two-dimensional neural network |
Introduction |
| Regression tree/decision tree for regression |
Introduction |
| Classification tree/decision tree for classification |
Introduction |
| Batch gradient descent |
Introduction |
| Large and small datasets in ML |
Introduction |
| |
|
| |
|
| |
|
| DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order |
| Add an item to a dictionary |
Introduction |
| Creates a dictionary from a csv file |
Introduction |
| Input a sentence and then output a sentence based on a dictionary obtained from csv |
Introduction |
| IDLE (integrated development and learning environment) and integrated development environment (IDE) |
Introduction |
| Check if a variable does exist/is assigned/defined |
Introduction |
| Find common/different elements/items between two lists/sets |
Introduction |
| Trick: Get coordinate difference between mouse positions |
Introduction |
| |
|
| Check if all the (and how many, length of a string) characters in the text are digits/numbers |
Introduction |
| Create dictionary from nested (sublist) list and get the values with keys |
Introduction |
| Compute the similarity between two text documents/files |
Introduction |
| Clustering versus Classification of texts and documents |
Introduction |
| Sentence, text and document embeddings |
Introduction |
| Get directory/path/file name partially |
Introduction |
| Call and run another script in a different/any (parent or children) directory/path/subfolder from a script |
Introduction |
| Get the current directory/folder path |
Introduction |
| Change the current directory to any directory/path (e.g. with os.chdir) |
Introduction |
| Loop through a Python dictionary |
Introduction |
| Avoid duplicates when creating text file |
Introduction |
| Copy a file or all files (with os.mkdir) to save to somewhere (create a directory first if it does not exist) |
Introduction |
| Delete the column/row in a CSV file if they are empty or less than a number (or header/index only) |
Introduction |
| sort_values(by=... ascending/descending order) |
Introduction |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Data Augmentation |
Introduction |
| One-dimensional neural network |
Introduction |
| |
|
| Compare dates (x days after or before a date), and difference between two dates in days |
Introduction |
| Merge dictionaries (update(), **, chain(), ChainMap(), |, |=) |
Introduction |
| Check if a file exists again (double check) |
Introduction |
| Create a temporary file or directory/folder |
Introduction |
| Download Youtube vedio |
Introduction |
| RegEx (Regular Expression) (characters to check if a string contains a specified search pattern, remove double spaces, and clean texts) |
Introduction |
| Extract any substrings with any pattern (e.g. dot (.)) |
Introduction |
Difference/comparison between real mouse click and click from script/program, e.g. Pyautogui
|
Introduction |
Different behavior of automation execution (e.g. pyautogui) locally or remotely through internet
|
Introduction |
| Fréchet Inception Distance (FID) coefficient |
Introduction |
| |
|
| Date |
|
| ◆ |
year, month, date, hour, minute, and second: "datefmt='%Y-%m-%d %H:%M:%S')": year, month, date, hour, minute, and second |
Instruction |
| ◆ |
Plot figures with date/month/year |
Introduction |
| ◆ |
Change date/month/year format |
Introduction |
| ◆ |
Find and convert the file time/date |
Introduction |
| ◆ |
Last n days/weeks/months (.to_datetime(x), .set_index(y), .last(z), .reset_index(), and .max() in pandas) |
Introduction |
| ◆ |
Time and date used as a file/folder name stamp (e.g. duplicate a file in the same folder) |
Introduction |
| ◆ |
datetime.date: An idealized naive date, assuming the current Gregorian calendar always was, and always will be, in effect |
(code) |
| ◆ |
datetime.datetime: A combination of a date and a time. Attributes: year, month, day, hour, minute, second, microsecond, and tzinfo |
(code) |
| ◆ |
datetime.time: An idealized time, independent of any particular day, assuming that every day has exactly 24*60*60 seconds |
(code) |
| ◆ |
datetime.timedelta: A duration expressing the difference between two date, time, or datetime instances to microsecond resolution |
(code) |
| ◆ |
datetime.tzinfo: An abstract base class for time zone information objects |
(code) |
| ◆ |
datetime.timezone: A class that implements the tzinfo abstract base class as a fixed offset from the UTC |
(code) |
| ◆ |
datetime.strftime(format): Return a string representing the date and time, controlled by an explicit format string |
(code) |
| ◆ |
.date.today(): Get current date: code (in different formats) and code. |
(code) |
| ◆ |
date_parser: function to use to parse strings into datetime objects. If parse_dates is True, it defaults
to the very robust dateutil.parser. Specifying this implicitly sets parse_dates as True. You can also
use functions from community supported date converters from date_converters.py |
|
| ◆ |
date_format: Format string for datetime objects |
|
| ◆ |
dayfirst: if True then uses the DD/MM international/European date format (This is False by default) |
|
| ◆ |
datetime.__format__(format): Same as datetime.strftime() |
(code) |
| ◆ |
Get the date and time (a past date) of N days ago |
Introduction |
| |
|
| DataFrame cheatsheet/basics: number of rows and columns, value of specific single cell, scatter plots, |
Introduction |
| ◆ |
DataFrame and DataFrame templates |
Introduction |
| ◆ |
DataFrame.from_csv(): from_csv(path, header=0, sep=', ', index_col=0, parse_dates=True, encoding=None, tupleize_cols=False, infer_datetime_format=False): Read CSV file. It is preferable to use the more powerful pandas.read_csv() for most general purposes, but from_csv makes for an easy roundtrip to and from a file, especially with a DataFrame of time series data. |
|
| ◆ |
Remove duplicate cell values from a csv file/dataframe (keeping the first/top one) |
Introduction |
| ◆ |
Get header/column names from DataFrame |
Introduction |
| ◆ |
Convert dataframe row/column into a comma separated string |
Introduction |
| ◆ |
Iterate over rows in a DataFrame/read and print row-by-row (number of columns and rows, df.shape[0]/df.shape[1]) |
Introduction |
| ◆ |
Cheatsheet of PySpark (for SparkSQL) |
Introduction |
| ◆ |
Convert all elements of specific column or in entire dataframe into strings |
Introduction |
| ◆ |
Compare string entries/cells/elements of columns in different dataframes
|
Introduction |
| ◆ |
Create table on pptx with certain rows and columns in DataFrame |
Introduction |
| ◆ |
Check and drop negative from dataframe pandas |
Introduction |
| ◆ |
DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order |
| ◆ |
csv workflow: Read into dataframe, Select a specific column from DataFrame, Select several specific columns to plot |
| ◆ |
Plot workflow: Create new empty column/row in DataFrame, Move the cells in a column to another column under certain condition, Select specific columns for scatter plot |
| ◆ |
Skip/remove empty rows (row-by-row) in DataFrame/csv |
Introduction |
| ◆ |
Remove/delete the duplicated/same rows in dataframe::>> df_unique = df.drop_duplicates() |
|
| ◆ |
Extract a subset of DataFrame from a DataFrame |
Introduction |
| ◆ |
Select specific columns from a DataFrame to form a new DataFrame |
Introduction |
| ◆ |
Output the row into dataframe if the value of the cell in a column contains a specific substring in a csv file (with headers) |
Introduction |
| ◆ |
Extract a SubDataFrame from A DataFrame |
Introduction |
| ◆ |
Merge columns with character/symbol Separation |
Introduction |
| ◆ |
Extract the last column as subdataframe |
Introduction |
| ◆ |
Plot graph/figure/image from CSV file/DataFrame |
Introduction |
| ◆ |
Plot graph/figure/image from CSV file/DataFrame by removing/hiding blank/empty cells with axis range (plt.xlim()) |
Introduction |
| ◆ |
Skip/replace empty cells from DataFrame/CSV file |
Introduction |
| ◆ |
Separately plot data into the same graph/figure/image from different csv files for each category (import multiple CSV files and concatenate into one DataFrame): append row-by-row or column-by-column |
Introduction |
| ◆ |
Convert csv/dataframe column to a list or vice versa |
Introduction |
| ◆ |
Remove unwanted/unnecessary parts from strings in a column of dataframe |
Introduction |
| ◆ |
Select/skip columns by index in DataFrame without changing the DataFrame itself, and change the order of of the selected columns |
Introduction |
| ◆ |
Find the same elements in columns in two separate dataframes and then merge them |
Introduction |
| ◆ |
Remove rows if (multiple) NaN is more than a number in DataFrame |
Introduction |
| ◆ |
Read columns with numeric values/numbers only in dataframe |
Introduction |
| ◆ |
Get the frequency of occurrence of a string in a column DataFrame |
Introduction |
| ◆ |
Compare (pattern/ratio of) two different columns, check whether column values match in DataFrame |
Introduction |
| ◆ |
Check whether one column contains number only and another column contains letters only or mixture of numbers and letters in DataFrame |
Introduction |
| ◆ |
Check the difference between two columns in DataFrame |
Introduction |
| ◆ |
|
|
| ◆ |
Calculations in DataFrame:
Add a column, calculate for a new column, delete a column, all the rows with values greater than 30 in "Score A" column |
Introduction |
| ◆ |
.DataFrame(): .drop(), index, columns, axes, dtypes, size, shape, ndim, empty, T, values |
Introduction |
| ◆ |
Apply a formatting function to all cells in a DataFrame |
Introduction |
| ◆ |
DataFrame.equals(): Confirm if the two csv files are the same or not |
code |
| ◆ |
Handle NaN value in DataFrame, replace empty cells with ...
|
Introduction |
| ◆ |
Print/remove/delete specific rows of a DataFrame |
Introduction |
| ◆ |
Pickling and unpickling of pandas DataFrame (.pkl) |
Introduction |
| ◆ |
Store images in pandas dataframe column |
Introduction |
| ◆ |
Print/remove/delete specific rows of a DataFrame |
Introduction |
| ◆ |
Write contents of DataFrame/memory into text file |
Introduction |
| ◆ |
Convert DataFrame to a HTML Table and save as a HTML webpage |
Introduction |
| ◆ |
Sort DataFrame by dates/year/month order |
Introduction |
|
|
| |
|
| Detection of tables from an image |
Introduction |
| Data cleaning examples in csv files |
Introduction |
| Remove decimal part in a string with comma |
Introduction |
| Form a list of strings from an old string with all the 6 digits by removing all special characters or spaces |
Introduction |
| Filters the rows based on the condition of being within n days of today's date |
Introduction |
| __add__, __call__, __contains__, __delitem__, __delattr__, __eq__, __enter__, __ge__, __getattribute__, __getnewargs__, __getattr__, __getitem__, __gt__, __hash__, __reduce__, __iadd__, __imul__, __init_subclass__, __index__, __int__, __invert__, __new__, __neg__, __reduce_ex__, __reversed__, __rmul__, __radd__, __rand__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __ror__, __round__,__rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __rxor__, __dir__, __doc__, __divmod__, __iter__, __le__, __lt__, __len__, __ne__, __repr__, __setattr__, __setitem__, __sizeof__, __lshift__, __sub__, __subclasshook__, __str__ |
Introduction |
| Delete a single file |
Introduction |
| Duplicate/repeat the same words/elements in a string/list |
Introduction |
| Comparison between decision tree, random forest and XGBoost (extreme gradient boosting) |
Introduction |
| Dimensionality reduction |
Introduction |
| Convert a floating-point number (decimal) to exponential format |
Introduction |
| Plot images from different DataFrame in a single row |
Introduction |
| Partial (e.g. first) portion of headers of DataFrame |
Introduction |
| Only use the first 4 characters in the headers of the table for pptx/dataframe |
Introduction |
| Search/extract all the 4-digit numbers (with and without extension) from a given text |
Introduction |
| Cheatsheet about headers (column names) in DataFrame |
Introduction |
| Generate defect data of a wafer map |
Introduction |
| Aggregate duplicates in columns of data |
Introduction |
| Reindex a DataFrame |
Introduction |
| Adjusting space around the plot with dummy categories |
Introduction |
| Save dynamic graph as a movie/video or split a movie to image frames |
Introduction |
| Comparative overview of multivariate statistical methods (Correlation Analysis, Regression Analysis, Factor Analysis, Cluster Analysis, Principal Component Analysis (PCA), Canonical Correlation Analysis, Discriminant Analysis, Path Analysis, Structural Equation Modeling (SEM), Multivariate Analysis of Variance (MANOVA), Analysis of Covariance (ANCOVA) ): purposes, variables, and outputs |
Introduction |
| Software/interface used in data science and machine learning |
Introduction |
| Aggregation functions in data manipulation and database queries |
Introduction |
| Cheatsheet of dictionary |
Introduction |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
| del |
Introduction |
| |
|
| delim_whitespace |
CSV: Parse whitespace-delimited (spaces or tabs) file (much faster than using a regular
expression) |
| dialect |
CSV: string or csv.Dialect instance to expose more ways to specify the file format. To expose more ways to specify the file format. code. |
| dtype |
CSV: A data type name or a dict of column name to data type. If not specified, data types will be inferred.
(Unsupported with engine=’python’) |
| |
|
| datatable.Frame.to_csv() |
CSV: Write the contents of the Frame into a CSV file. If no path is given, then the Frame will be serialized into a string, and that string will be returned. |
| drop() |
Delete a column. CSV: (code) |
| drop_duplicates() |
CSV: (code) |
| doublequote |
CSV: Control quoting of quotechar in fields (default True) |
| .duplicated() |
CSV: Determines which elements of a vector or data frame are duplicates of elements with smaller subscripts, and returns a logical vector indicating which elements (rows) are duplicates. (code) |
| display.max_rows |
CSV: Sets the maximum number of rows pandas should output when printing out various output. (code) |
| display.max_columns |
CSV: Sets the maximum number of columns pandas should output when printing out various output. (code) |
| dropna() |
CSV: Allows the user to analyze and drop Rows/Columns with Null values in different ways. code. |
| sep or delimiter |
CSV: A delimiter / separator to split fields on. The delimiter can be tab, comma, space, semicolon, etc. With sep=None, read_csv will try to infer
the delimiter automatically in some cases by “sniffing”. The separator may be specified as a regular expression, e.g. ‘|\s*’ can be used to indicate a pipe plus arbitrary whitespace. |
| |
|
| |
|
| DictReader() |
CSV: csv.DictReader(f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds).
Create an object that operates like a regular reader but maps the information in each row to a dict whose keys are given by the optional fieldnames parameter. DictReader uses the first row of csv file to setup keys for each dictionary value return for each line. code. |
| Dictionary |
Introduction, and data structures. |
| .doubleClick() |
.doubleClick(x=moveToX, y=moveToY) (Code) |
| drag() |
E.g. .drag(10, 0, 2, button='right') # drag the mouse left 10 pixels over 2 seconds while holding down the right mouse button. (code). |
| dragTo() |
dragTo(x, y, duration=num_seconds) drags mouse to XY. E.g. .dragTo(50, 20, button='left'): drag mouse to X of 50, Y of 20 while holding down left mouse button. .dragTo(100, 100, 2, button='left'): drag mouse to X of 100, Y of 100 over 2 seconds while holding down left mouse button.
(code). |
| dragRel() |
dragRel(xOffset, yOffset, duration=num_seconds) drags mouse relative to its current position. (code). |
| .displayMousePosition() |
(code) |
| delete |
drop(), del |
del() |
Remove an item from a dictionary by key or value: Code. Another example: del x; then print(x); shows error. |
| del() |
Delete an item from a list. |
| entry.delete(0, tk.END) |
Use the special constant tk.END for the second argument of .delete() to remove all text in an Entry |
| .destroy() |
(code). (code). |
| import DigitalMicrograph as DM |
GMS 3.4 has Python scripting capabilities with some libraries. |
| def |
Is short for define. Introduction. code. code4. code5. code6. code7. code. |
| dict() |
Introduction. Declaration: d = dict(); d = {}; d = {'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}. Print dictionary: code. |
| from pptx.dml.color import RGBColor |
(code) |
| divmod() |
Outputs the quotient and the remainder in a tuple. |
| dis.dis() |
Introduction. dis.dis(x=None, *, file=None, depth=None): Disassemble the x object. x can denote either a module, a class, a method, a function, a generator, an asynchronous generator, a coroutine, a code object, a string of source code or a byte sequence of raw bytecode. code. |
| deepcopy() |
Creates a new object and recursively adds the copies of nested objects present in the original elements. code. |
| denominator |
|
| class watchdog.events.DirMovedEvent(src_path, dest_path) |
Bases: watchdog.events.FileSystemMovedEvent ::
File system event representing directory movement on the file system. |
| class watchdog.events.DirModifiedEvent(src_path) |
Bases: watchdog.events.FileSystemEvent :: File system event representing directory modification on the file system. |
| class watchdog.events.DirCreatedEvent(src_path) |
Bases: watchdog.events.FileSystemEvent :: File system event representing directory creation on the file system. |
| class watchdog.events.DirDeletedEvent(src_path) |
Bases: watchdog.events.FileSystemEvent :: File system event representing directory deletion on the file system. |
| dispatch(event) |
Dispatches events to the appropriate methods.
event (FileSystemEvent) – The event object representing the file system event. |
| dispatch(event) |
Dispatches events to the appropriate methods.
Parameters: event (FileSystemEvent) – The event object representing the file system event. |
| dispatch_events(event_queue, timeout) |
Override this method to consume events from an event queue, blocking on the queue for the specified timeout before raising queue.Empty.
Parameters: event_queue (EventQueue) – Event queue to populate with one set of events.
timeout (float) – Interval period (in seconds) to wait before timing out on the event queue.
Raises: queue.Empty |
| daemon |
A boolean value indicating whether this thread is a daemon thread (True) or not (False).
This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.
The entire Python program exits when no alive non-daemon threads are left. |
| class watchdog.utils.dirsnapshot.DirectorySnapshot(path, recursive=True, walker_callback=<function <lambda> at 0x7f78fbeed500>, stat=<built-in function stat>, listdir=<built-in function listdir>) |
A snapshot of stat information of files in a directory.
Parameters:
path (str) – The directory path for which a snapshot should be taken.
recursive (bool) – True if the entire directory tree should be included in the snapshot; False otherwise.
walker_callback –
Deprecated since version 0.7.2.
stat –
Use custom stat function that returns a stat structure for path. Currently only st_dev, st_ino, st_mode and st_mtime are needed.
A function with the signature walker_callback(path, stat_info) which will be called for every entry in the directory tree.
listdir – Use custom listdir function. See os.listdir for details. |
class watchdog.utils.dirsnapshot.DirectorySnapshotDiff(ref, snapshot) |
Compares two directory snapshots and creates an object that represents the difference between the two snapshots.
Parameters:
ref (DirectorySnapshot) – The reference directory snapshot.
snapshot (DirectorySnapshot) – The directory snapshot which will be compared with the reference snapshot. |
dirs_created |
List of directories that were created. |
dirs_deleted |
List of directories that were deleted. |
dirs_modified |
List of directories that were modified. |
dirs_moved |
List of directories that were moved.
Each event is a two-tuple the first item of which is the path that has been renamed to the second item in the tuple. |
| math.degrees() |
Converts angle x from radians to degrees. |
| dispatch(event) |
Dispatches events to the appropriate methods.
event (FileSystemEvent) – The event object representing the file system event. |
| Default values in functions |
Introduction |
| Default value for list |
Introduction |
| def __init__(self[, suppliedprop1, suppliedprop2,...]) |
The syntax for creating an init method. The self part is just a variable name, and it is used to refer to the object being created at the moment.
You
can use the name of your own choosing instead of self. code. code. |
| |
|
| Directory |
path: code. |
Integer/fractions/round /decimal/digits/floating
/ceil/floor |
It does not have any fractional part. Introduction. int: Example code. |
| Decimal |
Those having fixed precision |
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| scipy.linalg.dft |
Create a discrete Fourier transform matrix. |
| dot(a, b[, out]) |
Dot product of two arrays. |
| linalg.cholesky(a) |
Cholesky decomposition. |
| linalg.qr(a[, mode]) |
Compute the qr factorization of a matrix. |
| linalg.svd(a[, full_matrices, compute_uv, …]) |
Singular Value Decomposition. |
| |
| linalg.det(a) |
Compute the determinant of an array. |
| __annotations__/dict |
name/type: parameter and return annotations |
| __defaults__/tuple |
name/type: default values for the formal parameters |
| __globals__/dict |
name/type: global variables of the module where the function is defined |
| __kwdefaults__/dict |
name/type: default values for the keyword-only formal parameters |
| v2.rectangle(image, start_point, end_point, color of border line, border thickness) |
border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code. |
Get dimensions (sizes) of image:
dimensions = img.shape
Get height, width, number of channels in image
height = img.shape[0]
width = img.shape[1]
channels = img.shape[2] |
General, code. code. code. code. code. code. |
| cv2.destroyAllWindows() |
Simply destroys all the windows we created. code. code. |
| cv2.destroyWindow() |
To destroy any specific window with the exact window name. |
| .apply(pd.Series) |
(code) |
| fillna() |
(code) |
| Split a list into columns |
Introduction |
| |
| random.default_rng |
Random generator: General. code. |
| Count duplicates/occurrence and show unique values in csv files |
CSV: Introduction |
| Dash |
From exploring data to monitoring your experiments, Dash is like the frontend to the analytical Python backend. This productive Python framework is ideal for data visualization apps particularly suited for every Python user. |
| hotkey('d') |
Introduction |
| .hotkey('down') |
Introduction |
| .delete_rows() |
In excel files: (code) |
| .delete_cols |
In excel files: (code) |
| .getAllTitles() |
Get all the Python program windows, *IDLE Shell window, e.g. *IDLE Shell 3.9.5, the most front window on Dreamweaver, the most front webpage on each Chrome window, the name of each opened applications, e.g. DigitalMicrograph. Introduction |
| '<Double-1>' |
Double clicks. (code) |
| .DISABLED |
Disable a button. (code) |
| from collections import defaultdict |
(code). |
| split() |
Introduction. Split a string by dots, split a file name by dots, split a file name from its extension. code. (code). |
| driver.switch_to.default_content() |
Introduction. |
.mouseDown() |
Move the mouse and then release it. .mouseDown(x=moveToX, y=moveToY, button='left'). .click() function is just a convenient wrapper around these two .mouseDown() and .mouseUp() function calls. (code) |
| os.path.dirname() |
(code). |
| dir() |
code. (code). |
| DPI, or dots per inch |
Is a measure of the resolution of a printed document or digital scan. (code). |
| |
|
| Methods of the mapping types dict, collections.defaultdict and collections.OrderedDict (common object methods omitted for brevity). Optional arguments are enclosed in «...».. |
| Methods |
Dict |
Default
dict |
Ordered dict |
|
| d.clear() |
✔ |
✔ |
✔ |
remove all items |
| d.__contains__(k) |
✔ |
✔ |
✔ |
k in d |
| d.copy() |
✔ |
✔ |
✔ |
shallow copy |
| .copy() |
|
|
|
Image overlap. Code. copy method. |
| d.__copy__() |
|
✔ |
|
support for copy.copy |
| d.default_factory |
|
✔ |
|
callableinvokedby __missing__ toset missing values |
| d.fromkeys(it, «initial») |
✔ |
✔ |
✔ |
new mapping from keys in iterable, with
optional initial value (defaults to None) |
| d.get(k, «default») |
✔ |
✔ |
✔ |
get item with key k, return default or none if missing |
| d.__getitem__(k) |
✔ |
✔ |
✔ |
d[k] — get item with key k |
| d.items() |
✔ |
✔ |
✔ |
get view over items — (key, val ue) pairs |
| d.__iter__() |
✔ |
✔ |
✔ |
get iterator over keys |
| d.keys() |
✔ |
✔ |
✔ |
get view over keys |
| d.__len__() |
✔ |
✔ |
✔ |
len(d) — number of items |
| d.__missing__(k) |
|
✔ |
|
called when __getitem__ cannot find the key |
| d.move_to_end(k, «last») |
|
|
✔ |
movekfirstorlastposition(lastisTrue by default) |
| d.pop(k, «default») |
✔ |
✔ |
✔ |
remove and return value at k, or de
fault or None if missing |
| d.popitem() |
✔ |
✔ |
✔ |
remove and return an arbitrary (key, value) itemb |
| d.__reversed__() |
|
|
✔ |
get iterator for keys from last to first inserted |
| d.setdefault(k, «de
fault») |
✔ |
✔ |
✔ |
if k in d, return d[k]; else set d[k]
= default and return it |
| d.__setitem__(k, v) |
✔ |
✔ |
✔ |
d[k] = v — put v at k |
| d.update(m, «**kargs») |
✔ |
✔ |
✔ |
update d with items from mapping or
iterable of (key, value) pairs |
| d.values() |
✔ |
✔ |
✔ |
get view over values |
|
| |
|
| |
|
| Line detection on image |
Introduction |
| Save contents (download pdf files) in the webpages obtained by Google search into a text file |
Introduction |
| Calibrate and put a scale bar, and draw a line segment on an image / detect a scale bar/scalebar calibration by clicking the start and end of the scale bar on desktop |
Introduction |
| Diversity prediction theorem |
Introduction |
| Examples of matplotlib (image/data) visualizations |
Introduction |
| Extract the least/most frequency/duplicate/occurrence element in a list |
Introduction |
| Three dimensional (3D) shapes/structures |
Introduction |
| cosine similarity/distance |
Introduction |
| Stochastic gradient descent (SGD) |
Introduction |
| Dot notation |
Introduction |
| Find duplicate items in a list |
Introduction |
| Dummy variables/binary variables |
Introduction |
| (Deep) neural network (NN or DNN) |
Introduction |
| Deploying process in ML |
Introduction |
| pd.get_dummies |
(Code) |
| .as_default() |
Introduction |
| tf.distribute.Strategy |
Introduction |
| tf.data.Dataset |
Introduction |
| Axis/dimension of tensor |
Introduction |
| Draw smiling face emoji |
Introduction |
| Decorators in Python |
Introduction |
| |
|
| |
|
Databricks |
Introduction |
| Data science |
|
| Data structures (Data science, and comparison between list, tuple, set, dictionary) |
Introduction |
| Ranking/most popular programming languages for data analysts |
Introduction |
| Critical thinking in data science |
Introduction |
| Call and then run your own functions and modules in different/other Python files |
Introduction |
| Double click of mouse |
Introduction, (Introduction) |
| Double click a specific position |
Introduction |
| Drag mouse |
Introduction |
| Lock desktop |
Introduction |
| Open the page of the downloaded list of webpage browser |
Introduction |
DigitalMicrograph (DM) |
| Table of digital micrograph (DM) shortcut hotkeys |
Introduction |
| Automation of EELS data extraction from DigitalMicrograph |
Introduction |
| Positions of features on Gatan DigitalMicrograph on PC screen |
(introduction) |
| Extract .dm3/.dm4/.dm5/ images (from DigitalMicrograph) to a csv file |
Introduction |
| Load/launch images and ColorMixing in DigitalMicrograph |
Introduction |
| |
| Check if all the (and how many, length of a string) characters in the text are digits/numbers |
(Code) |
| Work (read, write, insert and delete rows and columns, and merge and unmerge cells, shift/move cell values) in Excel sheets |
Introduction |
| Term Frequency-Inverse Document Frequency (TF-IDF) |
Introduction |
| Select/input a folder/directory/path for later to be called to use |
Introduction |
| Calibrate and put a scale bar, and draw a line segment on an image |
Introduction |
| Access elements in a dictionary and subdictionary |
Introduction |
| Build databases with different/uncertain number of members |
Introduction |
| Set default programs by file extensions and by file types and programs on Windows |
Introduction |
| Pint summary of the statistic data, change data format, sort/group columns |
Introduction |
| Draw an arrow segment pointing from the start point to the end point in an image. |
code. |
| Draw a line in a image |
code |
| Draw lines manually and then label them with arrows |
code |
| Choose a file with simple dialog. |
Code. code. code. With a default folder: code1 and code2. . |
| Copy images to a different folder/Save an image in a new folder |
code, code. code. code. |
| Data plot with labels |
code, code. |
| Calculate the coordinates of a point in a given rectangle and the distance of a given point to a line |
code |
| Compute the difference between two images by using Structural Similarity Index
with "pip install --upgrade imutils" |
code. code |
| Reverse the digits of a given number |
code |
| Get the list of the methods for a function (e.g. using dir) |
Introduction |
| Check if a popup dialog is a window or not for Selenium app |
Introduction |
| dropdown box |
Introduction |
| Draw lines and arrows in a ppt |
Introduction |
| Remove an/all/duplicate
item(s)/elements from a list |
Introduction |
| Count and delete slides from ppt |
Introduction |
| Work with all files and subfolders/subdirectory in a folder |
Introduction |
| Get/list immediate subdirectories/subfolders; get only the last part of a path |
Introduction |
| Move file(s) from one directory to another |
Introduction |
| Print/obtain a specific digit in a number, e.g. in a cell value in cvs |
Introduction |
| Remove an item/element/duplicates from a list |
Introduction |
| Get/list immediate subdirectories/subfolders; get only the last part of a path/folder/drive |
Introduction |
| Check/find/get a file name or the last folder name (e.g. from a path/directory) |
Introduction |
| Check if a file/folder/path/directory exists or not (Cannot find a specific file/folder?, or check files with extension) |
Introduction |
| |
|
| Check if a directory is empty; find the position/index of a particular file/folder in a file directory/path; remove folder or file level by level (or layer by laer) from its directory/path |
Introduction |
| Circular dependencies in Python execution |
Introduction |
| Watchdog ignore/skip and take pattern in directory/path |
Introduction |
| Move(remove) all files from original folder in a directory to a new directory |
Introduction |
| Delete the entire directory and/or all the files in the directory/folder |
Introduction |
| List all files and directories which has specific files or files with specific extensions |
Introduction |
| |
|
| Count how many (number) files and folders in a directory |
Introduction |
| Take a screenshot using a mouse click and drag method |
Introduction |
| Measure (real or pixel) length/distance on an image w/o calibrated bar |
Introduction |
| Crop/snip part of a image with definition by a pixel line |
Introduction |
| Modify file path/directory by changing folder names by merging a list |
Introduction |
| Plot distance between points calculated by coordinates |
Introduction |
| Deep learning |
Introduction |
| Manual analysis of data |
Introduction |
| Libraries used to convert incident documents into numerical vectors |
Introduction |
| Decision tree learning |
Introduction |
| Keyword extraction methods from documents in Natural Language Processing (NLP) |
Introduction |
| Ranking/most popular programming languages/tools for data analysts |
Introduction |
| Ranking and votes of essential/most important skills for data analysts |
Introduction |
| Remove duplicate/same lines in a text file |
Introduction |
| Ranking/most popular machine learning frameworks used by data scientists |
Introduction |
| Comparison of qualifications and skills between data science manager, engineering and scientist |
Introduction |
| Spatial defect patterns |
Introduction |
| Defect Detection and Classification by using Machine Learning |
Introduction |
| Human inspection of defects in wafer map |
Introduction |
| Automated defect scanning in wafer map |
Introduction |
| Global defects and local defects identified by defect denoising |
Introduction |
| Denoising/remove noise in images |
Introduction |
| (Deep) Convolutional neural networks (CNN) |
Introduction |
| Wafer map failure pattern recognition (WMFPR)/wafer failure pattern detection (WFPD)/defect classification |
Introduction |
| Public datasets for wafer map analysis |
Introduction |
| Methods of data and information visualization |
Introduction |
| Draw circles/lines on images |
Introduction |
| Root Cause Deconvolution (RCD) |
Introduction |
Detection and classification of defective dies in wafer map
|
Introduction |
| Detection procedures/processes of spatial defect patterns (bins) in wafers |
Introduction |
