| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| socket library |
Introduction |
| LASSO (Least Absolute Shrinkage and Selection Operator) |
Introduction |
| ML model complexity versus dataset size |
Introduction |
| Data scaling and startup |
Introduction |
| Characteristics of semi-structured data |
Introduction |
| Spark installation |
Introduction |
| Practice project of data processing using Spark |
Introduction |
| Data Ingestion in Apache Spark |
Introduction |
| Memory resources in Apache Spark applications |
Introduction |
| OLS (Ordinary Least Squares) regression model |
Introduction |
| Apache Spark User Interface (UI) |
Introduction |
| Clusters (Kubernetes, Apache Mesos, Spark Standalone, Apache Hadoop YARN) in Apache Spark |
Introduction |
| Configuring Spark |
Introduction |
| Semantic Segmentation Using U-Net with EfficientNet and Pixelshuffle |
Introduction |
| Deploy modes for driver process in Apache Spark: client mode and cluster mode |
Introduction |
| Apache Spark applications to a Kubernetes cluster |
Introduction |
| Skills Network Labs (SN Labs, IBM) |
Introduction |
| Apache Spark on IBM Cloud |
Introduction |
| Spark environments and options |
Introduction |
| Setup Apache Spark and run an Apache Spark application |
Introduction |
| Apache Spark architecture |
Introduction |
| Spark driver program |
Introduction |
| Schema |
Introduction |
| Aggregating data in SparkSQL |
Introduction |
| User-Defined Schema (UDS) for Domain-Specific Languages (DSL) and Structured Query Language (SQL) |
Introduction |
| Tungsten in Spark |
Introduction |
| Feature Selection: Chi Square to select dependent and independent variables |
Introduction |
| Catalyst in Spark |
Introduction |
| Principles of ethical and responsible ML (selection bias, confirmation bias, automation bias, model fairness) |
Introduction |
| Analyzing the impact of fabrication conditions on semiconductor wafer fail rates |
Introduction |
| Best practices for implementing ML in semiconductor manufacturing |
Introduction |
| Data selection in ML for semiconductor manufacturing processes |
Introduction |
| Feature engineering in semiconductor manufacturing ML |
Introduction |
| Text summarization for ML |
Introduction |
| Google Cloud Shell |
Introduction |
| Specialized tools and APIs lacking in GCP (Google Cloud) for semiconductor applications |
Introduction |
| Dataset in Apache Spark |
Introduction |
| SparkSQL |
Introduction |
| Spark Core of Apache Spark |
Introduction |
| Platform Security Engineering (PSE) and ML |
Introduction |
| Comparisons among SparkML, MLlib, and AutoML |
Introduction |
| Analyzing Data in Hadoop (HDFS, YARN, Apache Hive, Pig, HBase, Spark) |
Introduction |
| Data Storage in Hadoop (HDFS, HBase and YARN) |
Introduction |
| Ingesting data in Hadoop (Sqoop, Flume, Kafka, NiFi) |
Introduction |
| Comparison between Apache Spark's MLlib and Python |
Introduction |
| Apache Spark |
Introduction |
| Mathematical algorithms of artificial intelligence for semiconductor industry |
Introduction |
| Timeseries prediction in ML |
Introduction |
| squared Pearson correlation coefficient |
Introduction |
| Bisection search (binary search) |
Introduction |
| Mistakes that beginner machine learning (ML) students often make |
Introduction |
| Spotify and Evernote |
Introduction |
| Self-attention in ML |
Introduction |
| Replaces a symbol/character/letter in a string |
Introduction |
| Fit and Smooth Plotted Curves |
Introduction |
| Backtracking search |
Introduction |
| Constraint Satisfaction Problems (CSPs) as Search Problems |
Introduction |
| Optimal margin classifier/maximum margin separator |
Introduction |
| Soft constraints and hard constraints |
Introduction |
| Constraint satisfaction |
Introduction |
| Simulated Annealing |
Introduction |
| Local Search in ML |
Introduction |
| Search patterns in a text file |
Introduction |
| Sensor Model |
Introduction |
| Hidden state |
Introduction |
| Sampling Methods for Approximate Inference |
Introduction |
| Summary and cheatsheet of command for csv file |
Introduction |
| Logical statements |
Introduction |
| Rule-Based Systems |
Introduction |
| Proposition symbols |
Introduction |
| A* (A-star) Search |
Introduction |
| Informed search algorithms/heuristic search algorithms |
Introduction |
| Uninformed search/blind search algorithms |
Introduction |
| Breadth-First Search (BFS) |
Introduction |
| Depth-first search (DFS) |
Introduction |
| Search problem (search algorithm) in ML |
Introduction |
| Goal State and Goal Test in ML |
Introduction |
| State and state space in ML |
Introduction |
| Policy search algorithms and "normal" reinforcement learning (RL) algorithms |
Introduction |
| Policy Search Algorithms |
Introduction |
| Stationary and Non-Stationary State Transitions in Markov Decision Process (MDP) |
Introduction |
| State-action rewards in Markov Decision Process (MDP) |
Introduction |
| Algorithm sensitivity to zero values |
Introduction |
| State transition function (probability) in reinforcement learning |
Introduction |
| Smooth images (make image blurry) |
Introduction |
| Output the web links obtained by Google Search (from googlesearch import search) |
Introduction |
| except SomeSpecificException |
Introduction |
| self and __init__ method in Class |
Introduction |
| Three dimensional (3D) shapes/structures (e.g. cylinder) |
Introduction |
| Mean squared error (MSE) (L2 loss function, Euclidean loss) and root mean squared error (RMSE) |
Introduction |
| Save the webpages obtained by Google search into a text file |
Introduction |
| Support-vector machines(SVM)/support-vector networks(SVN) |
Introduction |
| .pack(side=LEFT)/.pack(side=RIGHT)/.place(x=, y=) --- position of the buttons |
(Code) |
| Machine learning example step-by-step (wafer fail analysis) |
Introduction |
| Vertex AI Feature Store |
Introduction |
| Common values in two pandas series |
Introduction |
| (Find) file size/find the largest/smallest file in a directory/folder |
Introduction |
| Save the image in clipboard to an image file |
Introduction |
| Folder for saving images from screenshot obtained by Snipping Tool on windows |
Introduction |
| Trained model and automatic model selection |
Introduction |
| Locate/find the center/coordinates of a bright (maximum/highest intensity) spot in an image & Find nearest white pixel to a given/specifical pixel location on an binary image |
Introduction |
| Plot horizontal stacked bar/histogram |
Introduction |
| Machine learning example step-by-step (prediction of house price) |
Introduction |
| Save the text in clipboard to a txt file |
Introduction |
| Comparison between steps and epochs in TensorFlow |
Introduction |
| t-SNE (t-distributed stochastic neighbor embedding, from sklearn.manifold import TSNE) |
Introduction |
| Sequential API to create a Keras model with TensorFlow (e.g. on Vertex AI platform) |
Introduction |
| Speed in machine learning process |
Introduction |
| cosine similarity/distance |
Introduction |
| Draw smiling face emoji |
Introduction |
| Size in tensor |
Introduction |
| Stochastic gradient descent (SGD) |
Introduction |
| save_checkpoints_steps |
Introduction |
| .save() |
Introduction |
| tf.saved_model.save() |
Introduction |
| tf.distribute.Strategy |
Introduction |
| Shape of tensor |
Introduction |
| tf.keras.model.save() |
Introduction |
| serving_input_fn |
Introduction |
| Least squares fit |
Introduction |
| get_shape() |
Introduction |
| tf.Session |
Introduction |
| Long short-term memory (LSTM) |
Introduction |
| stop_if_no_decrease_hook |
Introduction |
| find_element(CSS_SELECTOR, " ") |
Introduction |
| tf.keras.layers.StringLookup |
Introduction |
| Reasons of automation and how to start |
Introduction |
| |
|
| Supervised learning |
Introduction |
| ◆ |
Supervised learning with tensorFlow |
Introduction |
| ◆ |
Labeling in supervised machine learning |
Introduction |
| ◆ |
Self-supervised machine learning |
Introduction |
| |
|
| Wafer map failure pattern recognition (WMFPR) |
Introduction |
| ◆ |
Feature extraction using radon transform |
Introduction |
| ◆ |
Wafer map similarity ranking (WMSR) |
Introduction |
| ◆ |
Euclidean distance and Euclidian similarity for images |
Introduction |
| |
|
| Match on images to find and to highlight unsimilar (threshold=0) to identical (threshold=1) regions of an image that match a template with a cross-correlation method |
code |
| Automatically review, scroll, click webpage and its link |
Introduction |
| Left click a specific position |
Introduction |
| Right click a specific position |
Introduction |
| Double click a specific position |
Introduction |
| Scroll mouse |
Introduction |
| Snipping/crop tool: take a screenshot, e.g. from the full screen |
Introduction |
| Open an application window through search at start |
Introduction |
| Save files (e.g. powerpoint/ppt) |
Introduction |
| Change the font size for selected text |
Introduction |
| Start the slide show |
Introduction |
| Search in a powerpoint file |
Introduction |
| Summary of working on ppt |
Introduction |
| Add a new slide into an existing ppt, or work on existing slides |
Introduction |
| shift on keyboard |
Introduction |
| Launch the existing opened application if there is or start a new one if there is not |
Introduction |
| Move the cursor/mouse to the found, similar spots one-by-one |
Introduction |
| Table of powerPoint shortcut hotkeys |
Introduction |
| Table of PC/computer/Windows shortcut hotkeys |
Introduction |
| Table of Word shortcut hotkeys |
Introduction |
| Table of Excel shortcut hotkeys |
Introduction |
| Table of Chrome shortcut hotkeys |
Introduction |
| (Single and multiple) selection between choices or options |
Introduction |
| Table of digital micrograph (DM) shortcut hotkeys |
Introduction |
| Move the active window to make space for other apps |
Introduction |
| Work (read, write, insert and delete rows and columns, and merge and unmerge cells, shift/move cell values) in Excel sheets |
Introduction |
| Calculation in an Excel Sheet, Style, Bold, and Color |
Introduction |
| Minimize/maximize/restore/activate/resize/move/close Window objects |
Introduction |
| Check to see if or get a window with a name containing specific titles or texts |
Introduction |
| stopwatch and timing a process |
Introduction |
| Turn on and off with mouse press or a process/button switch |
Introduction |
| Mouse left single click |
Introduction |
| Mouse right single click |
Introduction |
| Turn on and off with mouse press or a process/button switch, button state |
Introduction |
| Copy and then store it into memory and it can be pasted for use later |
Introduction |
| Text classification/sort/prediction, train/test e.g. Youtube spam |
Introduction |
| Find the best word/text similarity |
Introduction |
| Send emails in HTML and text formats |
Introduction |
| (Single and multiple enter/input) box for pop-up window |
Introduction |
| Select/input a folder/directory/path for later to be called to use |
Introduction |
| Image similarity search |
code, introduction |
| Skip, remove, extract, use specific columns |
Introduction |
| Get a specific output for every input |
code. |
| Check the letters and symbols starting and ending with |
code. |
| Calibrate and put a scale bar, and draw a line segment on an image / detect a scale bar/scalebar calibration by clicking the start and end of the scale bar on desktop |
Introduction |
| Calculator of length accuracy in 3D structure |
Introduction |
| Option/selection/choice methods ("pop-up windows of Yes and No ") |
Introduction |
| Swap the order of arguments in a function |
Introduction |
| Highlight texts or make selection |
Introduction |
| Sum a list of numbers in any length |
Introduction |
| Mixing of using numbers and strings by conversions |
Introduction |
| Slicing and indexing in string |
Introduction |
| Data structures (Data science, and comparison between list, tuple, set, dictionary) |
Introduction |
| Access elements in a dictionary and subdictionary |
Introduction |
| Access elements in a list and sublist |
Introduction |
| Set default programs by file extensions and by file types and programs on Windows |
Introduction |
| Wheatstone bridge and its simulation |
Introduction. code |
| Numpy: Access the element at the second row, the third entry, access a specific row or a column, access some elements (submatrix), or replace/modify an element in the array, print a transfer of an array, access array under conditions or filtering |
Introduction |
| Pint summary of the statistic data, change data format, sort/group columns |
Introduction |
| Count the numbers of uppercase letters, lowercase letters and spaces in a string and then swap the cases of the letters. |
code |
| Markers (e.g. color cross, scatter, and circles) at specific coordinates with x- and y-axis |
Matplotlib |
| Save key and escape (ESC) key |
code. code. |
| Keyword search function/check whether or not a string is within another string (a space is included as a string character) |
Introduction |
| Find a similar feature and then click it |
Introduction |
| Draw an arrow segment pointing from the start point to the end point in an image. |
code. |
| Choose a file with simple dialog. |
Code. code. code. With a default folder: code1 and code2. |
| Open an image to file from URL (webpage), then it can be saved in PC |
code, code. open with color changed. |
| Swap two numbers |
code |
| Copy images to a different folder/Save an image in a new folder |
code, code. code. code. |
| Set the output image to zero everywhere except my mask (color filter), and display red, green, and blue (RGB) channels of an image. |
code, code. |
| Show/open images in any image viewer |
code, code. |
| Remove letters or characters on either side (both left and right sides) and stops when neither such letters no characters on either side |
code |
| Split columns and merge in csv: Split columns and then merge the splits in a csv file. |
Introduction |
| Skip rows and/or columns in csv |
CSV: Introduction. |
| sort_values(by=... ascending/descending order) |
Introduction |
| Compute the difference between two images by using Structural Similarity Index
with "pip install --upgrade imutils" |
code. code |
| Subtract (minus) two images after resizing them |
code, code. |
| Search position of numbers |
code |
| Loop through a string from left to right |
code1, code2 |
| Comparison between strings |
code |
| Find/search birthyear by name |
code |
| Repeated printing the same string |
code |
| Sending emails |
code |
| Open and close specific files |
Introduction |
| Selenium and pop-up windows |
Introduction |
| dropdown box/option/selection/choice |
Introduction |
| Find the color of a pixel on the screen |
Introduction |
| Create slides with text only |
Introduction |
| Check if a file/folder exists or not (Cannot find a specific file/folder? a specific folder in the path? select specific folders to form a string, split a dos path into its components, and then print the list, or check files with extension) |
Introduction |
| Count and delete slides from ppt |
Introduction |
| Randomly open an image/file in a specific folder |
Introduction |
| Work with (e.g. open) all/every files and subfolders/subdirectory in a folder (does not include any files or include all files in subfolders) |
Introduction |
| Insert all the images into a ppt file (one image per slide) |
(Introduction) |
| Stability/reliability of locateCenterOnScreen() |
Introduction |
| Stability/reliability of find_element(By.xxx, "") |
Introduction |
| Get/list immediate subdirectories/subfolders; split a dos path into its components, and then print the list |
Introduction |
| Replace/substitute a item in a list |
Introduction |
| Infinite loops (e.g. stop infinite cycling of opening the same images) |
Introduction |
| Write to a specific cell in a csv file |
Introduction |
| Mean (average, .mean())/.sum()/maximum(.max())/minimum(.min())/number of non-null values(.count())/.median()/variance(.var())/standard deviation(.std()/pstdev()) |
Introduction |
| Find a specific word in a webpage and count occurrences |
Introduction |
| Web Scraping (save contents from a webpage) |
Introduction |
| Copy a file to save to somewhere |
Introduction |
| (Find) file size |
Introduction |
| Check if both files are the same file, e.g. symbolic link, shortcut |
Introduction |
| Call and then run your own functions and modules in different/other Python files; Python run another Python script |
Introduction |
| Run multiple Python files/scripts one after another |
Introduction |
| Placement/position of Python import statements |
Introduction |
| matplotlib.pyplot axis/text color (xticks, rotation, xlabel, ylabel, title, fontsize, grid(), legend(), show()) |
Introduction |
| Watchdog for monitoring specific file or files with specific extension |
Introduction |
| Watchdog for monitoring specific file or files with specific extension, and then run another file from watchdog |
Introduction |
| Monitor specific new files, and execute the file or another file (and then restart the monitoring program itself to continue its monitoring by standby, with watchdog) |
Introduction |
| Launch script from another script using subprocess.run/subprocess.call |
Introduction |
| Simple watchdog to monitor file/folder changes |
Introduction |
| Watchdog ignore/skip and take pattern in directory/path |
Introduction |
| Find files with a specific file extension/type or with file names ending with specific characters |
Introduction |
| Execute scheduled jobs (time-schedule) |
Introduction |
| List all files and directories which has specific files or files with specific extensions |
Introduction |
| Subtract/minus one image from another image |
Introduction |
| Time and date used as a file/folder name stamp (e.g. duplicate a file in the same folder) |
Introduction |
| Check if a letter/character is in a string |
Introduction |
| Take a screenshot using a mouse click and drag method. For instance, take a screenshot, and then insert the image and/or a text into a ppt file. |
Introduction |
| Sort a text file |
Introduction |
| Search/extract text on an image |
Introduction |
| Extract text/check specific text from multiple powerpoint/pptx files |
Introduction |
| Resize and then sum/mix/overlap two images |
Introduction |
| Limit event/action numbers in the event List, then stop |
Introduction |
| Crop/snip part (without opening the image) of a image with definition by a pixel line |
Introduction |
| Quit/exit/stop a process (including by pressing a letter) |
Introduction |
| Shift/translate image along x-axis/y-axis |
Introduction |
| Change/swap values in a list |
Introduction |
| Sort a list |
Introduction |
| Email providers and their SMTP servers |
Introduction |
| Send emails through outlook |
Introduction |
| Split a sentence/string into list of words, remove all special characters/space from a sentence |
Introduction |
| Find the file names of the images in a pptx, (and then save/extract the image as a file) |
Introduction |
| Merge/combine two pptx files into one, including merging the pptx files with the words in a sentence as file names (not all words has pptx files) |
Introduction |
| Break/exit/skip a function/code line after a certain time |
Introduction |
| Comparison between spaCy and Natural Language Toolkit (NLTK) |
Introduction |
| Convert a text file to a string |
Introduction |
| Remove/repace (part) character(s)/spaces from string |
Introduction |
| Change/capitalize the case of the first letter of a string |
Introduction |
| Ranking and votes of essential/most important skills for data analysts |
Introduction |
| Write/save content to a text file/append a string into a text file. |
Introduction |
| Generate text file with the bank of collecting all words, characters and strings from news |
Introduction |
| Remove \n in string or new line in txt/text file |
Introduction |
| Remove duplicate/same lines in a text file |
Introduction |
| Detection procedures/processes of spatial defect patterns (bins) in wafers |
Introduction |
| Store images in pandas dataframe column |
Introduction |
| Cost (expense) and speed (fastest and slowest) of computation in ML |
Introduction |
| Open datasets, and open-source tools and libraries for Python and ML practice |
Introduction |
| Example of building robot (self-driving) systems with automated ML: helicopter |
Introduction |
| Maximum Likelihood Estimation (MLE) of single Gaussian (normal) distribution |
Introduction |
| Finding a correct loss (risk, objective) function for a specific problem |
Introduction |
| Number (size) of features in ML |
Introduction |
| Batch Gradient Descent (BGD), Stochastic Gradient Descent (SGD), Mini-Batch Gradient Descent, Batch Stochastic Gradient Descent, Momentum, (Adagrad, Adadelta, RMSprop), and Adam (Adaptive Moment Estimation) |
Introduction |
| Comparison among sigmoid, hyperbolic tangent (tanh) and rectified linear unit (ReLU) functions |
Introduction |
Linear correlation between two variables with Pearson Correlation Coefficient, Spearman Rank Correlation Coefficient, Kendall's Tau, Linear Regression, Coefficient of Determination and Correlation Ratio
|
Introduction |
| Example of ML debugging: Anti-Spam |
Introduction |
| Additive structure/additive model in ML |
Introduction |
| Structure of PowerPoint reports |
Introduction |
| DataFrame workflow: Drop/delete rows with empty cells in a column, Sort DataFrame by time/date order |
| Comparison among Grid Search, Bayesian Optimization, Random Search and Manual Search |
Introduction |
| Batch sizes |
Introduction |
| Comparisons among Manual Search, Vertex Vizier, AutoML and Early stopping on google cloud |
Introduction |
| Weight space |
Introduction |
| Statistical efficiency |
Introduction |
| Modify/replace the line in a text file if a line contains specific string |
Introduction |
| Sample Distribution in ML |
Introduction |
| Sample Complexity |
Introduction |
| Forward Search method of feature selection |
Introduction |
| Train-dev-test split (training-validation-testing split: Ratio for splitting dataset into training, validation and test set |
Introduction |
| Splitting a training dataset into different subsets |
Introduction |
| Training score/training error |
Introduction |
| Ranking/most popular IT automation software tools |
Introduction |
| Comparison of qualifications and skills between data science manager, engineering and scientist |
Introduction |
| Save/show generated images |
Introduction |
| Support-vector clustering (SVC) |
Introduction |
| Spatial defect patterns |
Introduction |
| Automated defect scanning in wafer map |
Introduction |
| Spatial filter for wafer map analysis |
Introduction |
| Image segmentation ("clustering") in color |
Introduction |
| WM-811K semiconductor data sets |
Introduction |
| Stack/overlap wafer bin map |
Introduction |
| Print specific rows of a DataFrame |
Introduction |
| Convert DataFrame to a HTML Table and save as a HTML webpage |
Introduction |
| Knuth-Morris-Pratt (KMP) algorithm (a string-searching algorithm) |
Introduction |
| Soft margin versus hard margin in ML |
Introduction |
| Support Vector Machines (SVM) and Logistic Regression |
Introduction |
| Single Naive Bayes (Gaussian Naive Bayes) versus Multinomial Naive Bayes |
Introduction |
| Laplace smoothing/Laplace correction/add-one smoothing |
Introduction |
| Single parameter estimation versus multiple parameter estimation |
Introduction |
| Large and small sample size/example/datasets in ML |
Introduction |
| Training set |
Introduction |
| Softmax regression (multinomial logistic regression)/softmax multi-class network/softmax classifier |
Introduction |
| Exponential Family: Parameter, Sufficient Statistic, Natural Parameter, Base Measure and Log-Partition Function (Bernoulli distribution and Gaussian distribution) |
Introduction |
| Bayesian Probability, Bayesian Statistics (Distribution Over a Distribution), versus Bayesian Inference |
Introduction |
| Logistic function/sigmoid function |
Introduction |
Comparison between mean squared error (MSE), absolute error (L1 Loss) and fourth-power loss
|
Introduction |
| Feature selection |
Introduction |
| Trace of a square matrix |
Introduction |
| Input data (sample and feature) (multiple and single sample/example) |
Introduction |
| Convex optimization, convex functions and convex sets |
Introduction |
| Types of predictions with Supervised Learning |
Introduction |
| Core Steps/Procedure/Designing of Machine Learning |
Introduction |
| Spearman Rank Correlation/Spearman's rho/Spearman correlation |
Introduction |
| Machine learning in yield analysis in semiconductor manufacturing |
Introduction |
| Extract the index of a string element in a list |
Introduction |
| Two-sample t-test |
Introduction |
| Hypothesis space/model space/search space |
Introduction |
| Epochs and sample size |
Introduction |
| Symbols/notations used in ML |
Introduction |
| Similarity-based clustering method (SCM) |
Introduction |
| Print the files and keyword occurrence which have been searched from a ppt file |
Introduction |
| Natural Language Processing (NLP) approaches in addressing the Failure Analysis (FA) search problem |
Introduction |
| Put the keywords in a grouped string into the first available cells in the corresponding columns in a csv file |
Introduction |
| Supervised, unsupervised and reinforcement learning |
Introduction |
| Send a variable from one script (back) to another script with a function |
Introduction |
| Compute the syntactic similarity between two text documents/files (with heatmap) |
Introduction |
| Correlations/similarity/dissimilarity of two columns in csv data |
Introduction |
| Check if two lists are same/identical |
Introduction |
| Percentages of information received through different senses (eye, nose, ear and hand feeling) |
Introduction |
| "Input space" in machine learning |
Introduction |
| "Label space" (X, y) in machine learning |
Introduction |
| |
|
| Remove the substring after the first or last character "::" in a given string, or extract the substring between the first and last "::" |
Introduction |
| Trick: generic code/script templates for complex automation |
Introduction |
| Convert a csv column to a string seperated by comma |
Introduction |
| Scalability in automation and machine learning projects |
Introduction |
| Check if a string can be converted to float |
Introduction |
| OverflowError (too large to store) |
Introduction |
| Remove empty strings from list of strings |
Introduction |
| Extract substrings between brackets (including brackets) |
Introduction |
| Extract any substrings with any pattern |
Introduction |
| Remove unwanted/unnecessary parts from strings in a column of dataframe |
Introduction |
| Convert strings to number (integers/float) |
Introduction |
| Iterate over rows in a DataFrame/read and print row by row (number of columns and rows, df.shape[0]/df.shape[1]) |
Introduction |
| Select/skip columns by index in DataFrame without changing the DataFrame itself, and change the order of of the selected columns |
Introduction |
| Stopwords/stoplist |
Introduction |
| Keyword scores |
Introduction |
| Extract the first or last N letters from a string |
Introduction |
| Nearest/most similar lyrics of a sentence to a CSV file |
Introduction |
| Check if a string is empty or space only |
Introduction |
| hyperspy application in STEM, EDS, and EELS analysis |
Introduction |
| send_keys() and its uploading images to webpage |
Introduction |
| Find common/different elements/items between two lists/sets |
Introduction |
| Combine multiple images into a single multi-page/frame image or vice versa (split a single multi-page image to multiple images) |
Introduction |
| Count the number of the pages in a single multi-page/frame image |
Introduction |
| Create dictionary from nested (sublist) list and get the values with keys |
Introduction |
| Skip/replace empty cells from DataFrame/CSV file |
Introduction |
| .stem() |
Introduction |
| Separately plot data into the same graph/figure/image from different csv files for each category (import multiple CSV files and concatenate into one DataFrame) |
Introduction |
| Plot images with certain image size and in color |
Introduction |
| Plot multiple datasets on the same scatter graph with different x- and y-axis values |
Introduction |
| Convert set into a list and vice versa |
Introduction |
| Summary/templates/examples of pptx and PowerPoint format |
Introduction |
| Exception LookupError (string index) |
Introduction |
| Remove empty strings from list of strings |
Introduction |
| Calculating the area fraction of each circle overlapped by a square grid and build wafer map |
Introduction |
| Convert all elements of specific column or in entire dataframe into strings |
Introduction |
| Get the frequency of occurrence of a string in a column DataFrame |
Introduction |
| Codes: Automation of Mouse Movements and Clicks, and keyboard control (comparison among pyautogui, pygetwindow, pydirectinput, autoit, Quartz, platform, ctypes, uiautomation and Sikuli) |
Introduction |
Difference/comparison between real mouse click and click from script/program, e.g. Pyautogui
|
Introduction |
| Check if a CSV file contains all of the specified strings |
Introduction |
| Principle and troubleshooting: Automation of Mouse Movements and Clicks (comparison among pyautogui, pygetwindow, pydirectinput, autoit, Quartz, platform, ctypes, uiautomation and Sikuli) |
Introduction |
| |
|
| Python applications in semiconductors |
|
| ◆ |
RegEx (Regular Expression) (characters to check if a string contains a specified search pattern, remove double spaces, and clean texts) |
Introduction |
| ◆ |
|
|
| ◆ |
|
|
| ◆ |
|
|
| |
|
| SentenceTransformers |
Introduction |
| ◆ |
Semantic clustering |
Introduction |
| |
✔ |
Clustering/classification of texts and documents |
Introduction |
| |
✔ |
Image segmentation ("clustering") in color |
Introduction |
| ◆ |
Semantic similarity |
Introduction |
| |
✔ |
Compute the similarity between two text documents/files (with heatmap) |
Introduction |
| |
✔ |
Image similarity search |
Introduction |
| |
✔ |
Multimodal text and image similarity |
Introduction |
| ◆ |
Natural language inference |
Introduction |
| ◆ |
Paraphrase mining |
Introduction |
| ◆ |
Semantic search |
Introduction |
| |
✔ |
Multimodal text and image search |
Introduction |
| |
✔ |
Image similarity search |
Introduction |
| |
✔ |
Text search |
Introduction |
| ◆ |
Embeddings |
Introduction |
| |
✔ |
Image embeddings |
Introduction |
| |
✔ |
Sentence, text and document embeddings |
Introduction |
| ◆ |
Question answering retrieval |
Introduction |
| |
|
|
| ◆ |
|
|
| |
|
|
| Summary/templates of plotting graphs/figures |
Introduction |
| Write special/certain rows (row-by-row) of one csv file to another csv file |
Introduction |
| y axis values are not ordered (disordered) |
Introduction |
| Extract any substrings with any pattern |
Introduction |
| Convert dataframe row/column into a comma separated string |
Introduction |
| Find the same elements in columns in two separate dataframes and then merge them |
Introduction |
| What are the students in university Mainly focusing on? |
Introduction |
| scipy.optimize.linprog function |
Introduction |
| Module import and execution are skipped during script execution |
Introduction |
| |
|
| Compare/check if two text files have the same contents |
Introduction |
| pyodbc for bridging SQL to Python |
Introduction |
| Python drivers for SQL server (pyodbc, pymssql, PyMySQL, cx_Oracle) |
Introduction |
| YAML sorting in Python |
Introduction |
| Insert paragraphs of texts into Python script |
Introduction |
| Input a sentence and then output a sentence based on a dictionary obtained from csv |
Introduction |
| Check if an element in a sublist of a list |
Introduction |
| Convert/change the case of all letters/word into uppercase (capital) or lowercase in a list of strings |
Introduction |
| Check if a variable is a number or string |
Introduction |
| Format strings |
Introduction |
| String template class for formating strings (F-strings (for calculation) (f"{}"), format() method ({}), %s, %d, Template ($)) |
Introduction |
| Check if two lists have the same elements |
Introduction |
| Last n days/weeks/months (.to_datetime(x), .set_index(y), .last(z), .reset_index(), and .max() in pandas) |
Introduction |
| Check if Windows/PC screen is locked |
Introduction |
| Check if all the (and how many, length of a string) characters in the text are digits/numbers |
Introduction |
| Plot multiple images on the same figure by hiding x- and y-labels |
Introduction |
| Sort dates/year/month by order |
Introduction |
| Create a Batch File to Run a Python Script |
Introduction |
| Remove/reload/unload an imported module/function/script |
Introduction |
| Access and use SQL Database on SSMS (Microsoft SQL Server Management Studio Express) with pyodbc: localhost, insert rows, update, count updated, delete rows, comparision between extract data by Python and SQL itself |
Introduction |
| Insert data/row into SQL Database on SSMS with pyodbc |
Introduction |
| Call and run another script in a different/any (parent or children) directory/path/subfolder from a script |
Introduction |
| Python modules to interact with the operating system (os, platform, subprocess, shutils, glob and sys) |
Introduction |
| Check if a string is empty or space only |
Introduction |
| Convert between numpy array, string or list of string |
Introduction |
| Count how many empty strings in a list |
Introduction |
| Form a list of strings from an old string with all the 6 digits by removing all special characters or spaces |
Introduction |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| StatsModels |
Introduction |
| Stacking/stacked ensembling |
Introduction |
| Sample Mean in ML |
Introduction |
| Sample Size versus Bounds |
Introduction |
| sklearn//Scikit-learn |
Introduction,
(code), (code). (code) |
| sklearn.cluster.KMeans() |
Introduction |
| Comparison between scikit-learn and tensorflow |
Introduction |
| Check if one list is subset of another (partially (part of)) |
Introduction |
| Get an element from a set |
Introduction |
| Automatically restart script execution after it breaks/fails/error |
Introduction |
| Good research topics in the field of semiconductor manufacturing and computer vision |
Introduction |
| Recall (Sensitivity or True Positive Rate) in machine learning |
Introduction |
| Adjusted R-squared values of two or more regression models |
Introduction |
| RSquare (R^2) versus RASE (Root Average Squared Error) |
Introduction |
| Remove decimal part in a string with comma |
Introduction |
| Consistency in Statistics |
Introduction |
| Well-specified case |
Introduction |
| Plot a figure with a colored arrow between text lines/steps |
Introduction |
| Multiple headers in a csv file: Count the number of header rows first and then split a single csv file to multiple csv files |
Introduction |
| __str__ method for a class |
Introduction |
|
|
|
|
| __add__, __call__, __contains__, __delitem__, __delattr__, __eq__, __enter__, __ge__, __getattribute__, __getnewargs__, __getattr__, __getitem__, __gt__, __hash__, __reduce__, __iadd__, __imul__, __init_subclass__, __index__, __int__, __invert__, __new__, __neg__, __reduce_ex__, __reversed__, __rmul__, __radd__, __rand__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __ror__, __round__,__rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __rxor__, __dir__, __doc__, __divmod__, __iter__, __le__, __lt__, __len__, __ne__, __repr__, __setattr__, __setitem__, __sizeof__, __lshift__, __sub__, __subclasshook__, __str__ |
Introduction |
| Remove/delete the duplicated/same rows in dataframe::>> df_unique = df.drop_duplicates() |
|
| Estimate the file size in memory before saving to PC |
Introduction |
| slice() |
Introduction |
| sorted() |
Introduction |
| Safely use credentials (username and password) in Python project |
Introduction |
| Convert a sentence/text to a list |
Introduction |
| Merge columns which contain specific strings |
Introduction |
| |
|
| Select specific columns from a DataFrame to form a new DataFrame |
Introduction |
| Output the row into dataframe if the value of the cell in a column contains a specific substring in a csv file (with headers) |
Introduction |
| Extract a subset of DataFrame from a DataFrame |
Introduction |
| Add letter/commas/numbers/characters to the end/beginning of strings in a list |
Introduction |
| Remove string 0s from the end of back/end of a list until non-zero values |
Introduction |
| Duplicate/repeat the same words/elements in a string/list |
Introduction |
| Extract a SubDataFrame from A DataFrame |
Introduction |
| Extract the last column as subdataframe |
Introduction |
| ML for failure analysis in the semiconductor industry |
Introduction |
| Define/measure the size of PowerPoint/pptx slides |
Introduction |
| Selecting only numeric/number columns, and then select two specific columns for plot |
Introduction |
| Sort DataFrame by dates/year/month order |
Introduction |
| Merge columns with character/symbol separation |
Introduction |
| Plot workflow: Create new empty column in DataFrame, Move the cells in a column to another column under certain condition, Select specific columns for scatter plot |
| Plot images from different DataFrame in a single row |
Introduction |
| plt.scatter() |
Introduction |
| plt.subplots_adjust(left=None, bottom=None, right=None, top=None,
wspace=None, hspace=None), or plt.subplots_adjust(wspace=0, hspace=0) |
Introduction |
| Font size of a (single/multiple) cell in table in PowerPoint |
Introduction |
| Search/extract all the 4-digit numbers (with and without extension) from a given text |
Introduction |
| Adjusting space around the plot with dummy categories |
Introduction |
| Cheatsheet of PySpark (for SparkSQL) and DataFrames |
Introduction |
| Save dynamic graph as a movie/video or split a movie to image frames |
Introduction |
| Cheatsheet of SQL (Structured Query Language) |
Introduction |
| Software/interface/API (Application Programming Interface) used in data science and machine learning |
Introduction |
| Comparative overview of multivariate statistical methods (Correlation Analysis, Regression Analysis, Factor Analysis, Cluster Analysis, Principal Component Analysis (PCA), Canonical Correlation Analysis, Discriminant Analysis, Path Analysis, Structural Equation Modeling (SEM), Multivariate Analysis of Variance (MANOVA), Analysis of Covariance (ANCOVA) ): purposes, variables, and outputs |
Introduction |
| Output data if any or same element in a string are in two lists |
Introduction |
| Cheatsheet of string |
Introduction |
| Cheatsheet of sets |
Introduction |
| Overcoming automation challenges and forward-looking suggestions |
Introduction |
| Scala |
Introduction |
| .Series() |
Introduction |
| .str.split() |
CSV: (code). |
| sep or delimiter |
CSV: A delimiter / separator to split fields on. The delimiter can be tab, comma, space, semicolon, etc. With sep=None, read_csv will try to infer
the delimiter automatically in some cases by “sniffing”. The separator may be specified as a regular expression, e.g. ‘|\s*’ can be used to indicate a pipe plus arbitrary whitespace. |
| skip_blank_lines |
CSV: whether to skip over blank lines rather than interpreting them as NaN values |
| skiprows |
CSV: A collection of numbers for rows in the file to skip. Can also be an integer to skip the first n rows |
| squeeze |
CSV: if True then output with only one column is turned into Series |
| sep |
CSV: Field delimiter for the output file (default ”,”) |
| Series.to_csv() |
CSV: Is a 1-D ndarray with axis labels and writes the given series object to a comma-separated values (csv) file/format. code. |
| .set_option() |
CSV: Set the value of a single option. (code) |
| skip_footer |
CSV: number of lines to skip at bottom of file (default 0) (Unsupported with engine=’c’) |
| csv.Sniffer() |
CSV: The Sniffer class is used to deduce the format of a CSV file. It expects a sample string, not a file. code. |
| skiprows |
Code. code. code. |
| skipfooter |
CSV: skip rows from bottom. code. |
| skipinitialspace |
CSV: boolean, default False, Skip spaces after delimiter |
| skiprows = lambda x: |
Code. code. code. |
| |
|
| Search in csv file |
Instruction. code. code. |
| .sum() |
Sum and percentage for csv. Introduction. |
| StandardScaler |
(code) |
| .size() |
The size (height and width) of the PC screen, or called resolution. Introduction. |
|
Locators |
|
| driver.get("") |
|
| Codes |
Chrome Inspect |
Getting path |
Application |
| driver.find_element(By.xpath("")) |
//label/span |
More complicated: need to scroll the mouse and trace down the code to find the correct path (code). Introduction. |
|
| driver.find_element(By.xpath("")).click() |
<button class = ""> </button> By.XPATH |
Copy XPath (code). Introduction. |
click() |
| driver.find_element(By.xpath("").sendKeys("") |
<input type=> |
Copy XPath (code). Introduction. |
sendKeys("") |
| Copy full XPath (code). Introduction. |
| More complicated: need to scroll the mouse and trace down the code to find the correct path (code) (code). Introduction. |
| find_element_by_xpath() |
XPath is the language used for locating nodes in an XML document. One of the main reasons of using XPath is when you don’t have a suitable id or name attribute for the element.
<html>
<body>
<form id="loginForm">
<input name="username" type="text" />
<input name="password" type="password" />
<input name="continue" type="submit" value="Login" />
<input name="continue" type="button" value="Clear" />
</form>
</body>
</html>
The first form elements can like this:
login_form = driver.find_element_by_xpath("/html/body/form[1]")
login_form = driver.find_element_by_xpath("//form[1]")
login_form = driver.find_element_by_xpath("//form[@id='loginForm']")
The second element (username) can be like this:
username = driver.find_element_by_xpath("//form[input/@name='username']")
username = driver.find_element_by_xpath("//form[@id='loginForm']/input[1]")
username = driver.find_element_by_xpath("//input[@name='username']")
The third element (Clear Button) can be like this:
clear_button = driver.find_element_by_xpath("//input[@name='continue'][@type='button']")
clear_button = driver.find_element_by_xpath("//form[@id='loginForm']/input[4]")
(code). Introduction. |
| |
|
|
|
| driver.findElement(By.id("")) |
<input id=> (code). Introduction. |
Find ID |
By.ID click() |
| driver.findElement(By.id("")).sendKeys("") |
<input type=> (code). Introduction. |
Find ID |
sendKeys("") |
| find_element_by_id() |
Use this when the id attribute of the element known, e.g.:
<html>
<body>
<form id="loginForm">
<input name="username" type="text" />
<input name="password" type="password" />
<input name="Submit" type="Submit" value="Login" />
</form>
</body>
</html>
Then, the form element can be:
MyLogin = driver.find_element_by_id('loginForm')
(code). Introduction. |
| dom =document.getElementById("") |
|
|
|
| driver.findElement(By.linkText("NextPage")).click() |
|
|
|
find_element_by_link_text()
find_element_by_partial_link_text() |
Use those when the link text used within an anchor tag is known: the first element with the link text matching the provided value will be returned, e.g.:
<html>
<body>
<p>Are you sure you want to do this?</p>
<a href="continue.html">Continue</a>
<a href="cancel.html">Cancel</a>
</body>
</html>
The continue.html link can be like this:
continue_link = driver.find_element_by_link_text('Continue')
continue_link = driver.find_element_by_partial_link_text('Conti')
(code). Introduction. |
| driver.findElement(By.partialLinkText(" NextP")).click() |
|
|
|
| find_element_by_css_selector() |
Use this, when you want to locate an element using CSS selector syntax, e.g.:
<html>
<body>
<p class="content">Site content goes here.</p>
</body>
</html>
The “p” element can be like this:
content = driver.find_element_by_css_selector('p.content')
(code). Introduction. |
| driver.FindElement(By.CssSelector("")) |
|
|
|
| chooseFile.sendKeys("") |
|
|
|
| driver.findElement(By.NAME("q")).sendKeys ("") |
|
|
|
| find_element_by_name() |
Use this when the name attribute of the element is known, e.g.:
<html>
<body>
<form id="loginForm">
<input name="username" type="text" />
<input name="password" type="password" />
<input name="continue" type="submit" value="Login" />
<input name="continue" type="button" value="Clear" />
</form>
</body>
</html>
Then, the username & password elements can be:
username = driver.find_element_by_name('username')
password = driver.find_element_by_name('password')
continue = driver.find_element_by_name('continue')
(code). Introduction. |
| driver.findElement(By.tagName("select")).Click() |
|
|
|
| find_element_by_tag_name() |
Use this, when you want to locate an element by tag name, e.g.:
<html>
<body>
<h1>Welcome</h1>
<p>Site content goes here.</p>
</body>
</html>
The heading (h1) element can be like this:
heading1 = driver.find_element_by_tag_name('h1')
(code). Introduction. |
| driver.findElement(By.className("")) |
|
|
|
| find_element_by_class_name() |
Use this, when you want to locate an element by class name, e.g.:
<html>
<body>
<p class="content">Site content goes here.</p>
</body>
</html>
The “p” element can be like this:
content = driver.find_element_by_class_name('content')
(code). Introduction. |
Navigators |
| driver.get(“http://globalsino.com”) |
Navigate to URL |
driver.navigate().to(“http://globalsino.com”)
|
Navigate to URL |
| driver.navigate().refresh() |
Refresh page |
| driver.navigate().forward() |
Navigate forwards in browser history |
| driver.navigate().back() |
Navigate backward in browser history |
| Upload files to webpages |
Introduction |
| Webdriver |
|
| Methods to locate elements in a page |
| find_element() |
(code) |
| find_elements() |
(code) |
| click() |
(code), (code). |
| scroll() |
Scroll the mouse wheel function and pass an integer number of “clicks” to scroll. .scroll(amount_to_scroll, x=moveToX, y=moveToY). (code) |
| .close() |
(code) |
| .quit() |
(code) |
|
|
| .onScreen() |
Code. |
| .screenshot() |
(code) |
| sort |
Similar to sorted(). |
| .locateOnScreen() |
.locateOnScreen('looksLikeThis.png') returns (left, top, width, height) on the image which the screenshot is taken from. Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match. Introduction. (code) |
| .locateAllOnScreen() |
Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match. Introduction. (code) |
| .locateCenterOnScreen() |
Uses pyscreeze. x, y = MySearch_img to get the x- and y-coordinates of centers of the feature. Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match. Introduction. (code) |
| from selenium.webdriver.common.keys import Keys |
(code) |
| split() |
Introduction. Split a string by dots, split a file name by dots, split a file name from its extension. code. (code). |
| splitlines() |
Split a string into a list where each line is a list item. code. |
| sys.modules |
Is a dictionary mapping the names of imported modules to the module object holding the code. code. |
sys.platform
|
(code) |
| sys.version |
(code) |
| sys.version_info |
(code) |
| sys.argv |
Introduction |
sys.stderr.write()
|
Prints the message as the given parameter to the stderr.(code) |
| sys.stderr.flush() |
Forces it to "flush" the buffer, meaning that it will write everything in the buffer to the terminal, even if normally it would wait before doing so.(code) |
sys.path
|
Is a list of strings that specifies the search path for modules. Basically this tells Python what locations to look in when it tries to import a module. code. |
| sys.platform |
Is a platform identifier. You can use this to append platform specific modules to sys.path, import different modules depending on platform or run different pieces of code. code. |
| sys.exit() |
Allows the developer to exit from Python. The exit function takes an optional argument, typically an integer, that gives an exit status. Zero is considered a “successful termination”. |
| .shapes.add_textbox |
(code) |
| from pptx.enum.shapes import MSO_AUTO_SHAPE_TYPE |
(code) |
| slide_layouts[] |
(code) |
| slides.add_slide() |
(code) |
| shapes.title |
(code) |
| .shapes.title |
(code) |
| .shapes.add_picture() |
(code) |
| MyPresentation.save() |
(code) |
| watchdog.utils.bricks.SkipRepeatsQueue |
Thread-safe event queue based on a special queue that skips adding the same event (FileSystemEvent) multiple times consecutively. Thus avoiding dispatching multiple event handling calls when multiple identical events are produced quicker than an observer can consume them. |
| schedule(event_handler, path, recursive=False) |
Schedules watching a path and calls appropriate methods specified in the given event handler in response to file system events.
Parameters: event_handler (watchdog.events.FileSystemEventHandler or a subclass) – An event handler instance that has appropriate event handling methods which will be called by the observer in response to file system events.
path (str) – Directory path that will be monitored.
recursive (bool) – True if events will be emitted for sub-directories traversed recursively; False otherwise.
Returns: An ObservedWatch object instance representing a watch. |
| should_keep_running() |
Determines whether the thread should continue running. |
| stop() |
Signals the thread to stop. |
stat_info(path) |
Returns a stat information object for the specified path from the snapshot.
Attached information is subject to change. Do not use unless you specify stat in constructor. Use inode(), mtime(), isdir() instead.
Parameters: path – The path for which stat information should be obtained from a snapshot. |
| .schedule() |
(code) |
| Sort txt files |
Introduction |
| os.path.splitext() |
Introduction. |
| math.sqrt() |
Calculate the square root of a
number by importing math and using math.sqrt() |
| math.sin(x) |
Returns the arc sine of x, in radians. |
| shutil |
(code) |
| shutil.move() |
(code) |
| Syntax of a function |
def <name of the function> (list of parameters):
<body>
code |
| os.startfile() |
(code) |
| with mss.mss() as sct |
(code) |
| seek() |
Sets the file's current position at the offset, namely to go back the file (otherwise the file cannot be read again). The argument is optional and defaults to 0, which means absolute file positioning, other values are 1 which means seek relative to the current position and 2 means seek relative to the file's end. Introduction. txt. code. (code) |
| startswith() and endswith() |
Introduction. String.startswith(value-required, start-optional, end-optional).
returns True if the string starts with the specified value, otherwise False. Start: an Integer specifying at which position to start the search; end: an Integer specifying at which position to end the search. code. (code) |
| "skip" |
Skip characters/letters in a string: Introduction. |
| sct.grab() |
(code) |
| strip() |
Returns a copy of the string by removing both the leading and the trailing characters. string.strip([chars]): chars (optional) - a string specifying the set of characters to be removed from the left and right part of the string. Introduction. code. txt. |
| strip([chars]) |
Returns copy with leading/trailing characters removed. |
| year, month, date, hour, minute, and second |
"datefmt='%Y-%m-%d %H:%M:%S')": year, month, date, hour, minute, and second. Instruction. |
| space used in scripts |
Example code |
| showinfo |
code. code. |
| set_axis_off() |
image. |
| style |
code. |
| side= |
code. (code). |
| Sets |
Introduction, and data structures. Abstraction of a mathematical set |
| str() |
Introduction. Returns a string which is fairly human readable. code. |
| "str()" and "," difference |
(code) |
| datetime.strftime(format) |
Return a string representing the date and time, controlled by an explicit format string. (code) |
string / multilinestring |
A string is a predefined object which contains characters. Loop through a string from left to right. Introduction. code1, code2. |
String
|
A string is a predefined object which contains characters. Example code |
| os.path.splitext() method |
Is used to split the path name into a pair root and ext. Here, ext stands for extension and has the extension portion of the specified path while root is everything except ext part. ext is empty if specified path does not have any extension. If the specified path has leading period (‘.’), it then will be ignored. Introduction. Examples are:
| path name |
root |
ext |
| /Desktop/file.txt /home/User/Desktop/file .txt |
/Desktop /home/User/Desktop |
{empty} |
| file.py |
file |
.py |
| .txt |
.txt |
{empty} |
|
| shutil.copyfileobj |
code. |
| shutil.copy() |
(code) (code) |
| SQLAlchemy |
It was designed for efficient and high-performing database-access. It is a library with well-known enterprise-level patterns. |
| SciPy |
Has a number of user-friendly and efficient numerical routines. These include routines for optimization and numerical integration. |
| scipy.signal |
Contains a few functions that can perform the brute force correlations. For
a 1D signal, the function is "correlate", and for a 2D signal, the correct
function is correlate2d. This function will receive two arrays and the second array should be small
in size. |
| scipy.integrate |
Numerical integration |
| scipy.linalg |
Linear algebra routines and matrix decompositions extending beyond those provided in numpy.linalg. scipy.linalg contains all the functions that are in numpy.linalg. Additionally, scipy.linalg also has some other advanced functions that are not in numpy.linalg. scipy.linalg operations can be applied equally to numpy.matrix or to 2D numpy.ndarray objects. code. |
| scipy.linalg.norm |
This function is able to return one of eight different matrix norms, or one of an infinite number of vector norms, depending on the value of the ord parameter. For tensors with rank different from 1 or 2, only ord=None is supported. It is for an old release of SciPy (version 0.14.0). |
| from scipy.linalg import norm |
code |
| scipy.linalg.block_diag |
Create a block diagonal matrix from the provided arrays. |
| scipy.linalg.circulant |
Create a circulant matrix. |
| scipy.linalg.companion |
Create a companion matrix. |
| scipy.linalg.convolution_matrix |
Create a convolution matrix. |
| scipy.linalg.dft |
Create a discrete Fourier transform matrix. |
| scipy.linalg.fiedler |
Create a symmetric Fiedler matrix. |
| scipy.linalg.fiedler_companion |
Create a Fiedler companion matrix. |
| scipy.linalg.hadamard |
Create an Hadamard matrix. |
| scipy.linalg.hankel |
Create a Hankel matrix. |
| scipy.linalg.helmert |
Create a Helmert matrix. |
| scipy.linalg.hilbert |
Create a Hilbert matrix. |
| scipy.linalg.invhilbert |
Create the inverse of a Hilbert matrix. |
| scipy.linalg.leslie |
Create a Leslie matrix. |
| scipy.linalg.pascal |
Create a Pascal matrix. |
| scipy.linalg.invpascal |
Create the inverse of a Pascal matrix. |
| scipy.linalg.toeplitz |
Create a Toeplitz matrix. |
| from scipy.linalg import eigh |
Print "selected eigenvalues" and "complex ndarray": code. |
| skimage.metrics |
from skimage.metrics import structural_similarity as compare_ssim
import argparse
import imutils
import cv2
code |
| skimage.measure.approximate_polygon(coords, ...) |
Approximate a polygonal chain with the specified tolerance. |
| skimage.measure.block_reduce(image, block_size) |
Down-sample image by applying function to local blocks. |
| skimage.measure.compare_mse(im1, im2) |
Compute the mean-squared error between two images. |
| skimage.measure.compare_nrmse(im_true, im_test) |
Compute the normalized root mean-squared error (NRMSE) between two images. |
| skimage.measure.compare_psnr(im_true, im_test) |
Compute the peak signal to noise ratio (PSNR) for an image. |
| skimage.measure.correct_mesh_orientation(...) |
Correct orientations of mesh faces. |
| skimage.measure.find_contours(array, level) |
Find iso-valued contours in a 2D array for a given level value. |
| skimage.measure.grid_points_in_poly |
Test whether points on a specified grid are inside a polygon. |
| skimage.measure.label(input[, neighbors, ...]) |
Label connected regions of an integer array. |
| skimage.measure.marching_cubes(volume, level) |
Marching cubes algorithm to find iso-valued surfaces in 3d volumetric data |
| skimage.measure.mesh_surface_area(verts, faces) |
Compute surface area, given vertices & triangular faces |
| skimage.measure.moments(image[, order]) |
Calculate all raw image moments up to a certain order. |
| skimage.measure.moments_central(image, cr, cc) |
Calculate all central image moments up to a certain order. |
| skimage.measure.moments_hu(nu) |
Calculate Hu’s set of image moments. |
| skimage.measure.moments_normalized(mu[, order]) |
Calculate all normalized central image moments up to a certain order. |
| skimage.measure.perimeter(image[, neighbourhood]) |
Calculate total perimeter of all objects in binary image. |
| skimage.measure.points_in_poly |
Test whether points lie inside a polygon. |
| skimage.measure.profile_line(img, src, dst) |
Return the intensity profile of an image measured along a scan line. |
| skimage.measure.ransac(data, model_class, ...) |
Fit a model to data with the RANSAC (random sample consensus) algorithm. |
| skimage.measure.regionprops(label_image[, ...]) |
Measure properties of labeled image regions. |
| skimage.measure.structural_similarity(*args, ...) |
Deprecated function. Use compare_ssim instead. |
| skimage.measure.subdivide_polygon(coords[, ...]) |
Subdivision of polygonal curves using B-Splines. |
| skimage.measure.CircleModel() |
Total least squares estimator for 2D circles. |
| skimage.measure.EllipseModel() |
Total least squares estimator for 2D ellipses. |
| skimage.measure.LineModel() |
Total least squares estimator for 2D lines. |
| skimage.measure.LineModelND() |
Total least squares estimator for N-dimensional lines. |
| linalg.svd(a[, full_matrices, compute_uv, …]) |
Singular Value Decomposition. |
| linalg.slogdet(a) |
Compute the sign and (natural) logarithm of the determinant of an array. |
| linalg.solve(a, b) |
Solve a linear matrix equation, or system of linear scalar equations. |
| __name__/str |
name/type: Introduction. the function name. Example code |
| __qualname__/str |
name/type: the qualified function name. Example code |
| Image.save() |
Saves this image under the given filename. If no format is specified, the format to use is determined from the filename extension, if possible. code1, code2. |
| Workbook.save |
(code) |
| .sheetnames |
Access sheet names from an excel sheet. (code) |
| Scikit-image |
Is a collection of algorithms for image processing, which is a Python image processing toolbox for SciPy.
Installation:
# Update pip
python -m pip install -U pip
# Install scikit-image
or, pip install --upgrade scikit-image
python -m pip install -U scikit-image |
| SymPy |
Is an open-source library for symbolic math. With very simple and comprehensible code that is easily extensible, SymPy is a full-fledged Computer Algebra System (CAS). It is written in Python, and hence does not need external libraries. |
| Scrapy |
Is a good library if your motive is fast, high-level screen scraping and web crawling. It can be used for purposes from data mining to monitoring and automated testing. |
| SSIM-PIL |
Compare two images using the structural similarity algorithm (SSIM), or compute image difference. The images need to have the same resolution. Installation: pip install SSIM-PIL. code. code. |
| .show() |
code. code. |
| cv2.imshow() and "xyz"..imshow() |
A method is used to display an image in a window. The window automatically fits to the image size. First argument is a window name which is a string. Second argument is
our image. Introduction. code. code. Code. |
| skimage.measure.structural_similarity(*args, ...) |
Deprecated function. Use compare_ssim instead. |
| skimage.measure.subdivide_polygon(coords[, ...]) |
Subdivision of polygonal curves using B-Splines. |
| Statsmodel |
Is for finance and statistical analysis |
| |
|
| Seaborn |
(code). When it comes to visualisation of statistical models like heat maps, Seaborn is among the reliable sources. This library is derived from Matplotlib and closely integrated with Pandas data structures. |
| SimplelTK |
Povides a simplified interface to ITK in a variety of languages. A user can either download pre-built binaries, if they are available for the desired platform and language, or SimpleITK can be built from the source code. |
| SKiDL |
|
| smtp.sendmail() |
code. |
| smtp.login() |
code. |
| smtp.starttls() |
code. |
| smtplib.SMTP() |
code. |
| smtp.ehlo() |
code. |
| smtp.starttls() |
code. |
| smtplib.SMTP() |
code. |
| v2.rectangle(image, start_point, end_point, color of border line, border thickness) |
border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code. |
| v2.rectangle(image, start_point, end_point, color of border line, border thickness) |
border. Compute the bounding box of the contour and then draw the bounding box on an image to represent where the ROI is. code. code. code. |
Get dimensions (sizes) of image:
dimensions = img.shape
Get height, width, number of channels in image
height = img.shape[0]
width = img.shape[1]
channels = img.shape[2] |
Introduction. General, code. code. code. code. code. code. |
| cv2.setMouseCallback |
Mouse clicks: introduction. code. code. |
| Split a list into columns |
Introduction |
| ScaleBar() |
Introduction. scalebar = ScaleBar(dx,
units="m",
dimension="si-length",
label=None,
length_fraction=None, height_fraction=None,
width_fraction=None,
location=None,
pad=None,
border_pad=None,
sep=None,
frameon=None,
color=None,
box_color=None,
box_alpha=None,
scale_loc=None,
label_loc=None,
font_properties=None,
label_formatter=None,
scale_formatter=None,
fixed_value=None, fixed_units=None,
animated=False,
rotation=None)
dx (required):
Size of one pixel in units specified by the next argument.
units:
Units of dx. The units needs to be valid for the specified dimension. Default: m.
label:
Optional label associated with the scale bar. Default: None, no label is shown. The position of the label with respect to the scale bar can be adjusted using label_loc argument.
length_fraction:
Desired length of the scale bar as a fraction of the subplot's width. Default: None, value from matplotlibrc or 0.2.
height_fraction:
Deprecated, use width_fraction.
width_fraction:
Width of the scale bar as a fraction of the subplot's height. Default: None, value from matplotlibrc or 0.01.
loc:
Alias for location.
pad:
Padding inside the box, as a fraction of the font size. Default: None, value from matplotlibrc or 0.2.
border_pad:
Padding outside the box, fraction of the font size. Default: None, value from matplotlibrc or 0.1.
sep:
Separation in points between the scale bar and scale, and between the scale bar and label. Default: None, value from matplotlibrc or 5.
frameon:
Whether to draw a box behind the scale bar, scale and label. Default: None, value from matplotlibrc or True.
color:
Color for the scale bar, scale and label. Default: None, value from matplotlibrc or k (black).
box_color:
Background color of the box. Default: None, value from matplotlibrc or w (white).
box_alpha:
Transparency of box. Default: None, value from matplotlibrc or 1.0 (opaque).
scale_loc:
Location of the scale with respect to the scale bar. Either bottom, top, left, right. Default: None, value from matplotlibrc or bottom.
label_loc:
Location of the label with respect to the scale bar. Either bottom, top, left, right. Default: None, value from matplotlibrc or top.
font_properties:
Font properties of the scale and label text, specified either as dict or str. See FontProperties for the arguments. Default: None, default font properties of matplotlib.
label_formatter:
Deprecated, use scale_formatter.
scale_formatter:
Custom function called to format the scale. Needs to take 2 arguments - the scale value and the unit. Default: None which results in.
fixed_value:
Value for the scale. The length of the scale bar is calculated based on the specified pixel size dx. Default: None, the value is automatically determined based on length_fraction.
fixed_units:
Units of the fixed_value. Default: None, if fixed value is not None, the units of dx are used.
animated:
Animation state. Default: False
rotation:
Whether to create a scale bar based on the x-axis (default) or y-axis. rotation can either be horizontal or vertical. Note you might have to adjust scale_loc and label_loc to achieve desired layout. Default: None, value from matplotlibrc or horizontal.
Dimension of dx and units. It can either be equal:
si-length (default): scale bar showing km, m, cm, etc.
imperial-length: scale bar showing in, ft, yd, mi, etc.
si-length-reciprocal: scale bar showing 1/m, 1/cm, etc.
pixel-length: scale bar showing px, kpx, Mpx, etc.
angle: scale bar showing °, ʹ (minute of arc) or ʹʹ (second of arc)
a matplotlib_scalebar.dimension._Dimension object.
Dimension of dx and units. It can either be equal:
si-length (default): scale bar showing km, m, cm, etc. |
| style |
Style string, like 'ko--', to be passed to matplotlib |
| sharex |
If subplots=True, share the same x-axis, linking ticks and limits |
| sharey |
If subplots=True, share the same y-axis |
| sort_columns |
Plot columns in alphabetical order; by default uses existing column order |
| .subplots/.subplot |
Plot each DataFrame column in a separate subplot. E.g. .subplots(nrows=5, ncols=10): 5 rows and 13 columns. The third
argument specifies which rectangle will contain the plot specified by
the following function calls. As a convenience, the commas separating the three arguments in the subplot
routine can be omitted, provided they are all single-digit arguments. E.g. plt.subplot(2, 1, 1) = plt.subplot(211). Can be used to compare different views of data side by side in an array. code. code. image. code. (code). |
| sharex |
All subplots should use the same x-axis ticks (adjusting the xlim will affect all subplots) |
| sharey |
All subplots should use the same y-axis ticks (adjusting the ylim will affect all subplots) |
| subplot_kw |
Dict of keywords passed to add_subplot call used to create each subplot |
| Speed comparison with and without numpy |
Introduction |
| shape/shape[] |
(code) |
| size |
(code) |
| Numpy splicing |
draw a grey or color point or shape in an image. code. |
| .sort(axis=0) |
code. |
| Split columns and merge in csv |
CSV: Split columns and then merge the splits in a csv file. Introduction |
| Search/print/output the rows |
CSV: Print the rows if their cell values are greater than a specific value, in the csv file with numbers only; output the rows if the cell value is in a specific range. Introduction. code. |
| Sort a csv file with column |
CSV: Instruction: used "key = operator.itemgetter()". |
| Skip rows and/or columns in csv |
CSV: Introduction. |
| t.shape() |
“classic”, “arrow”, “turtle”, “circle”, “square” and “triangle”. (code) |
| t.speed() |
Is used to change the speed of the turtle by the value of the argument that it takes. E.g. .speed(speed=None). The turtle’s speed lies in the range 0-10. If input is a number greater than 10 or smaller than 0.5, speed is set to 0. ‘fastest’ : 0;
‘fast’ : 10;
‘normal’ : 6;
‘slow’ : 3;
‘slowest’ : 1. (code) |
| t.Screen() |
Is used to set the size and position of the main window. (code) |
| t.screensize() |
turtle.screensize(canvwidth=None, canvheight=None, bg=None). If there is no parameter, then it is getting the actual screen size. (code) |
| Collections of geometric shape |
(code) |
| .set_window_position() |
(code) |
| execute_script("window.scrollBy(0, 250)") |
(code) |
| win32api.SetCursorPos() |
(code) |
| space bar on keyboard |
Press space bar. Introduction |
| hotkey('s') |
Introduction |
| hotkey('Spacebar') |
Introduction |
| .resize() |
(code) |
| .resizeTo() |
(code) |
| .size |
(code) |
| FLAT, RAISED, SUNKEN, GROOVE and RIDGE in Tkinter button relief styles |
Introduction |
| .set() |
Introduction |
| StringVar() |
(code) |
SikuliX |
|
| |
|
|
|
| .feature_extraction. |
(code). |
| ensemble |
(code). |
| RandomForestClassifier |
(code). |
| .metrics |
(code). |
| confusion_matrix |
(code). |
| model_selection |
(code). |
| cross_val_score |
(code). |
| TfidfTransformer |
(code). |
| .sample() |
(code). |
| |
| sg= |
The training algorithm, either CBOW(0) or skip gram (1). The default training alogrithm is CBOW. (code) |
| .status_code == |
(code). |
| ssl |
(code). |
| Spacy |
Introduction. |
| model.wv.similarity() |
(code). |
| model.wv.most_similar() |
(code). |
| skipgram model (for training) |
(code) |
| vector_size= |
The number of dimensions of the embeddings and the default is 100. (code). (code). |
| Slider() |
(code). |
| import subprocess |
(code). |
| Screen’s resolutions |
Introduction. |
| .switch_to. |
An object containing all options to switch focus into. Introduction |
| .switch_to.parent_frame() |
Introduction |
| driver.switch_to.default_content() |
Introduction. |
| from selenium.webdriver.chrome.service import Service |
(code) |
| from selenium.webdriver.support.select import Select |
(code) |
| .select_by_visible_text() |
(code). |
| MSO_CONNECTOR.STRAIGHT |
(code). |
| .series |
(code) |
| .slide_width |
(code). |
| slide_height |
(code). |
| random.seed() |
(code). |
| soup.find() and soup.find_all() |
Introduction |
| write() |
Save as a text file. Instruction |
| os.path.samefile() |
(code) |
| os.stat().st_size |
(code) |
| .set_xticks() |
(code) |
| .set_xticklabels() |
(code) |
| |
|
Similar functions |
| decorator |
@staticmethod, @classmethod |
| list |
list, tuple |
| Fill in NaN, for missing values |
SimpleImputer(strategy='constant'), .fillna() |
| sort |
sort, sorted() |
| .DataFrame() |
Introduction. .drop(),
index,
columns,
axes,
dtypes,
size,
shape,
ndim,
empty,
T,
values, .sample() (randomly print some rows). |
| |
|
import time
time.sleep(2) |
from time import sleep. time.sleep(secs) suspends the execution of the current thread for the given number of seconds.
sleep(2) |
