| |
|
|
K-means clustering and PCA for failure analysis |
|
|
Software/interface/API (Application Programming Interface) used in data science and machine learning |
|
|
Spark driver program |
|
|
Managing machine learning (ML) projects |
|
|
Practice project of data processing using Spark |
|
|
Personalizing applications with ML |
|
|
Trade-offs between fairness and performance |
|
|
Best practices for implementing ML in semiconductor manufacturing |
|
|
Semantic Segmentation Using U-Net with EfficientNet and Pixelshuffle |
|
|
Using proxy labels, building a labeling system, and utilizing a labeling service when historical labeled data is unavailable for ML projects |
|
|
Parametric and non-parametric learning algorithms |
|
|
Platform Security Engineering (PSE) and Machine Learning |
|
|
Parallel computing and distributed computing |
|
|
Analyzing Data in Hadoop (HDFS, YARN, Apache Hive, Pig, HBase, Spark) |
|
|
Comparison between Apache Spark's MLlib and Python |
|
|
Timeseries prediction in ML |
|
|
Calculation of Principal Component Analysis (PCA) |
|
|
Squared Pearson correlation coefficient |
|
|
Principal Component Analysis (PCA) versus Uniform Manifold Approximation and Projection (UMAP) |
|
|
Uniform Manifold Approximation and Projection (UMAP) |
|
|
Proxy |
|
|
Dataset and data preparation |
|
|
Pre-training in ML |
|
|
GPT (Generative Pre-trained Transformer) |
|
|
Performance metrics |
|
|
Max-pooling |
|
|
TensorFlow Playground |
|
|
Open datasets, and open-source tools and libraries for Python and ML practice |
|
|
Python Tutor |
|
|
Python Conference |
|
|
Python introduction and application examples |
|
|
Plot with letters/words as x-/y-axis |
|
|
Fit and Smooth Plotted Curves |
|
|
Constraint Satisfaction Problems (CSPs) as Search Problems |
|
|
Correlations/similarity/dissimilarity/pair/match of two columns in csv data |
|
|
Linear Programming (LP) algorithm |
|
|
Evaluation (Precision and Recall) in Text classification with Naive Bayes |
|
|
Precision and Recall Tradeoff |
|
|
Precision, Recall, False Positive Rate, and False Negative Rate (Miss Rate or False Negative Proportion) |
|
|
Search patterns in a text file |
|
|
pgmpy |
|
|
Unconditional probability |
|
|
Distributive Property |
|
|
Model Checking Algorithms and Modus Ponens Algorithms |
|
|
Modus ponens (a logical inference rule) |
|
|
Logic Puzzle in ML |
|
|
Propositional Logic Algorithms in ML |
|
|
Proposition symbols |
|
|
Search problem in ML |
|
|
Path and Path Cost in ML |
|
|
PySpice: Provides the interface to circuit simulators like Ngspice or Xyce. |
|
|
import PySpice.Logging.Logging as Logging |
|
|
from PySpice.Spice.Netlist import Circuit |
|
|
from PySpice.Unit import * |
|
|
.simulator() |
|
|
NgSpice/PySpice for electrical circuit analysis |
|
|
Save contents (download pdf files) in the webpages obtained by Google search into a text file |
|
|
Analysis of ROI (region of interest)/portion of an image |
|
|
Find nearest white pixel to a given/specifical pixel location on an binary image |
|
|
Diversity prediction theorem |
|
|
Categorical features preprocessing layers |
|
|
Reasons of using Python for automation |
|
|
Machine learning example step-by-step (prediction of house price) |
|
|
.pack(side=LEFT)/.pack(side=RIGHT)/.place(x=, y=) --- position of the buttons |
|
| |
|
|
Add padding/black/colored edge to images |
|
|
.pad() |
|
| |
|
|
.paste() |
|
|
Common values in two pandas series |
|
|
Keras preprocessing layers |
|
|
pywinauto |
|
|
Machine learning workflow/procedure |
|
|
Predictive/predict model (with "best"-option) |
|
| |
|
|
Pipelines in ML and Data Science |
|
|
Pipelines in ML |
|
|
Pipelines in Data Science |
|
|
kfp.dsl package versus pipelines and components |
|
|
Learning algorithm (ensemble learning) and pipeline |
|
| |
|
|
preprocessing. |
|
|
PyTorch |
|
|
Comparison between TensorFlow, PyTorch, Theano and OpenCV |
|
|
Troubleshooting and problem solving in Python programming |
|
|
Heatmap with input from a csv/pkl file |
|
|
Positions and colors of mouse/cursor and features |
|
|
Left click a specific position |
|
|
Right click a specific position |
|
|
Double click a specific position |
|
|
Print document |
|
|
Search in a powerpoint file |
|
| Integrating Slides from Existing PowerPoint Files into New Presentations |
|
|
Save files (e.g. powerpoint/ppt) |
|
|
Summary of working on ppt |
|
|
Analysis of papers/publications/literature in machine learning and Python applications |
|
|
Add a new slide into an existing ppt, or work on existing slides, check the existence of a pptx file, if does not exist then create it |
|
|
Table of powerPoint shortcut hotkeys |
|
|
Table of PC/computer/Windows shortcut hotkeys |
|
|
Comparison between Python and C/C++ |
|
|
stopwatch and timing a process |
|
|
Turn on and off with mouse press or a process |
|
|
Quit/exit/stop a process (including by pressing a letter) |
|
|
Periodic Table |
|
|
(Single and multiple enter/input) box for pop-up window |
|
|
Copy and then store it into memory and it can be pasted for use later |
|
|
Tricks in Python Programming and principles and practices in good programming |
|
|
Text classification/sort/prediction, train/test e.g. Youtube spam |
|
|
Print colored text in Python IDLE's terminal |
|
|
API (Application Programming Interface), e.g. weather, temperature |
|
|
Find/predict the best word similarity, e.g. car |
|
|
Principal Component Analysis (PCA) |
|
|
Select/input a folder/directory/path for later to be called to use |
|
|
Convert CSV to images, row by row, with pixel values: each row is an image |
|
|
Calibrate and put a scale bar, and draw a line segment on an image |
|
|
Option/selection/choice methods ("pop-up windows of Yes and No ") |
|
|
Copy text into clipboard and then you can paste it anywhere |
|
|
Calculate/pass the arbitrary (any) number of variables or input arguments |
|
|
Set default programs by file extensions and by file types and programs on Windows |
|
|
Numpy: Access the element at the second row, the third entry, access a specific row or a column, access some elements (submatrix), or replace/modify an element in the array, print a transfer of an array, access array under conditions or filtering |
|
|
Pint summary of the statistic data, change data format, sort/group columns |
|
|
Popup windows with inputs |
|
|
Popup windows with frames |
|
|
Copy text into clipboard and then you can paste it a webpage, text/txt, word or powerpoint file automatically |
|
|
Draw an arrow segment pointing from the start point to the end point in an image. |
|
|
Prevent other applications to modify the content until other Python script runs |
|
|
Repeated printing the same string |
|
|
Palindrome repeat |
|
|
Selenium and pop-up windows |
|
|
Check if a popup dialog is a window or not for Selenium app |
|
|
Draw lines and arrows in a ppt |
|
|
Find the color of a pixel on the screen |
|
|
Create slides with text only |
|
|
Draw lines, elbow connectors and arrows (shapes) in a pptx |
|
|
Count and delete slides from ppt |
|
|
Insert all the images into a ppt file (one image per slide) |
|
|
Check file existence with partial filename |
|
|
Get/list immediate subdirectories/subfolders; get only the last part of a path/folder/drive; split a dos path into its components, and then print the list |
|
|
Move file(s) from one directory to another |
|
|
Get maximum and minimum value of column and its index |
|
|
Get the csv/pandas cell value with certain condition |
|
|
Find latitude and longitude of a place in a map |
|
|
Check/find/get a file name or the last folder name (e.g. from a path/directory) |
|
|
Print and set file path as a variable (e.g. convert all characters in the pathname to lowercase) |
|
|
Check if a file/folder/path/directory exists or not (Cannot find a specific file/folder?, or check files with extension, select specific folders to form a string, split a dos path into its components, and then print the list) |
|
|
Check if a file/folder exists or not (Cannot find a specific file/folder? a specific folder in the path?) |
|
|
Check if a directory is empty; find the position/index of a particular file/folder in a file directory/path; remove folder or file level by level (or layer by laer) from its directory/path |
|
|
Placement/position of Python import statements |
|
|
Watchdog ignore/skip and take pattern in directory/path |
|
|
top and left, width and height for pptx (e.g. align the top-left corner of the image to the center of the slide no matter how the size of the images changes) |
|
|
Extract text/check specific text from multiple powerpoint files (Some methods can extract text from most of the document extensions such as pptx and pptm) |
|
|
Get mouse position/coordinates on click |
|
|
Measure (real or pixel) length/distance on an image w/o calibrated bar |
|
|
Resize (by real dimension or pixel) and then sum/mix/overlap two images |
|
|
Get pixel location/coordinates on an image using mouse click/events |
|
|
Crop/snip part of a image with definition by a pixel line |
|
|
Modify file path/directory by changing folder names by merging a list |
|
|
Email providers and their SMTP servers |
|
|
Merge/combine two pptx files into one |
|
|
Find the file names of the images in a pptx, (and then save/extract the image as a file) |
|
|
Count occurrence/nubmer of words/phrase in a text file |
|
|
Remove/repace (part) character(s) from string |
|
|
Ranking/most popular programming languages for data analysts |
|
|
Convert PDF file to text file |
|
|
Extract pdf pages to form new pdf files |
|
|
Ranking/most popular automation testing tools |
|
|
Take a screenshot/snipping using a mouse click and drag method. For instance, take a screenshot, and then insert the image and/or a text into a ppt file. |
|
|
Ranking/most popular IT automation software tools |
|
|
Comparison between Python, Blue Prism, UiPath, Automation Anywhere |
|
|
Robots and Robotic Process Automation (RPA) |
|
|
Ranking/most popular machine learning frameworks used by data scientists |
|
|
Wafer map failure pattern recognition (WMFPR) |
|
|
Histogram for wafer analysis (e.g. percentage and frequency of grey level in the image) |
|
|
Search position of numbers |
|
|
Spatial defect patterns |
|
|
Probe data |
|
|
Non-zero pixel values from an image |
|
|
Pixel values on specific pixel in an image |
|
|
Public datasets for wafer map analysis |
|
|
Pass variables between functions/from one to another |
|
|
Methods of physical failure analysis (PFA)/root cause analysis of ICs |
|
|
Convert images between Cartesian and Polar forms |
|
|
Fault analysis/PFA (Physical Failure Analysis) time and efficiency |
|
|
Patterns on wafers commonly reflecting specific process failure information |
|
|
Print specific rows of a DataFrame |
|
|
Table of applications of Python and its libraries |
|
|
Detection procedures/processes of spatial defect patterns (bins) in wafers |
|
|
Read/print a text/txt file |
|
|
Store images in pandas dataframe column |
|
|
Pickling and unpickling of pandas DataFrame (.pkl) |
|
|
Probability bounds analysis (PBA) |
|
|
Print the files and keyword occurrence which have been searched from a ppt file |
|
|
Percentages of information received through different senses (eye, nose, ear and hand feeling) |
|
|
Expected risk (population risk, expected value of loss or error) |
|
|
Empericial loss versus population loss |
|
|
Parameterized family and model parameters |
|
|
Predicted label |
|
|
Keyword Module in Python |
|
|
Python drivers for SQL server (pyodbc, pymssql, PyMySQL, cx_Oracle and psycopg2) |
|
|
Remove unwanted/unnecessary parts from strings in a column of dataframe |
|
|
Iterate over rows in a DataFrame/read and print row by row (number of columns and rows, df.shape[0]/df.shape[1]) |
|
|
Trick: Output a portion of rows and columns from a csv file cell by cell into another csv file |
|
|
Paraphrase mining |
|
|
pyodbc for bridging SQL to Python |
|
|
Create a Batch File to Run a Python Script |
|
|
Read a frequently updated file periodically (similar to watchdog) |
|
|
Codes: Automation of Mouse Movements and Clicks, and keyboard control (comparison among pyautogui, pygetwindow, pydirectinput, autoit, Quartz, platform, pynput, ctypes, uiautomation and Sikuli) |
|
|
Principle and troubleshooting: Automation of Mouse Movements and Clicks (comparison among pyautogui, pygetwindow, pydirectinput, autoit, Quartz, platform, ctypes, uiautomation and Sikuli) |
|
|
Trick: Get coordinate difference between mouse positions |
|
|
Evaluation of Precision in Machine Learning Process |
|
|
Probabilistic model |
|
|
pyodbc for bridging SQL to Python |
|
|
Access and use SQL Database on SSMS (Microsoft SQL Server Management Studio Express) with pyodbc: localhost, insert rows, update, count updated, delete rows, comparision between extract data by Python and SQL itself |
|
|
Insert data/row into SQL Database on SSMS with pyodbc |
|
|
Get directory/path/file name partially |
|
|
Call and run another script in a different/any (parent or children) directory/path/subfolder from a script |
|
|
Get the current directory/folder path |
|
|
Python modules to interact with the operating system (os, platform, subprocess, shutils, glob and sys) |
|
|
Change the current directory to any directory/path (e.g. with os.chdir) |
|
|
Check existence of phrase on text file line-by-line |
|
|
"@echo off" and "pause" in Command Prompt Window |
|
|
Last n days/weeks/months (.to_datetime(x), .set_index(y), .last(z), .reset_index(), and .max() in pandas) |
|
|
Check if Windows/PC screen is locked |
|
|
Combine multiple images into a single multi-page image or vice versa |
|
|
Count the number of the pages in a single multi-page/frame image |
|
|
Extract elements from a list (different way from removing elements to get part of the list) |
|
|
Lock a file to prevent deleting, and then release the file once job is done |
|
|
PermissionError (E.g. file cannot be written when it is open) |
|
|
Extract/confirm any substrings with any pattern (e.g. dot (.)) |
|
|
Get username and encoded password with getpass or or base64 |
|
|
Find repeating patterns in columns, group them as cycles, and column correlations |
|
|
Create table on pptx with certain rows and columns of strings |
|
|
Create table on pptx with certain rows and columns in DataFrame |
|
|
hotkey('h') in pyautogui |
|
|
Difference/comparison between real mouse click and click from script/program, e.g. Pyautogui |
|
|
Comparisons between JMP (John's Macintosh Project) and Python |
|
|
RegEx (Regular Expression) (characters to check if a string contains a specified search pattern, remove double spaces, and clean texts) |
|
| |
|
|
Compare (pattern/ratio of) two different columns, check whether column values match in DataFrame |
|
|
Check whether one column contains number only and another column contains letters only or mixture of numbers and letters in DataFrame |
|
|
Check the difference between two columns in DataFrame |
|
| |
|
|
from keyboard import is_pressed (Esc, check pressed key) |
|
|
Execute a command on Command Prompt of Windows |
|
|
Check if one list is subset of another (partially (part of)) |
|
|
Insert paragraphs of texts into Python script (f"""/f''') |
|
|
Plot figures with date/month/year |
|
|
Summary/templates/examples of pptx and PowerPoint format |
|
|
.bat (batch) files for Command Prompt Windows |
|
|
pd.concat() |
|
|
Penalized regression (Lasso and Ridge) |
|
|
Joint probability distribution (p) |
|
|
Predicted label versus predictor (feature) |
|
|
Hypothesis class/hypothesis family/predictor class/model class/hypothesis family/predictor family/model family |
|
|
Predicted values (ŷ, y^) |
|
|
Phrase |
|
|
Summary/templates of plotting graphs/figures |
|
|
Plot from dictionary |
|
|
Data plot with labels |
|
|
Plot multiple images on the same figure by hiding x- and y-labels |
|
|
Plot table |
|
|
Plot graph/figure/image from CSV file/DataFrame |
|
|
Plot graph/figure/image from CSV file/DataFrame by removing/hiding blank/empty cells with axis range (plt.xlim()) |
|
| plt plot functions |
|
|
Plot images with certain image size and in color |
|
|
Plot multiple datasets on the same scatter graph with different x- and y-axis values |
|
|
Separately plot data into the same graph/figure/image from different (multiple) csv files for each category |
|
|
Plot a list of x, y coordinates to an image |
|
|
|Plot distance between points calculated by coordinates |
|
|
Plot pixel intensity (histogram) along a line (row/column/x-axis/y-axis) of an image |
|
|
Plot histogram |
|
|
Plot curve/chart in pptx |
|
|
Transparency of marker (e.g. for plots) |
|
|
Loops (e.g. for loop) for 2D (two-dimensional) plot (or map) |
|
|
plotly.graph_objects |
|
|
Commands/concepts of plot with matplotlib.pyplot to plot/generate images: e.g. axis/colored text or annotation, ScaleBar() |
|
|
plots |
|
|
.xticks() and .yticks() |
|
|
plt.scatter(): Plot scattered curves |
|
|
plt.subplots_adjust(left=None, bottom=None, right=None, top=None,
wspace=None, hspace=None), or plt.subplots_adjust(wspace=0, hspace=0): Adjusting the spacing around subplots |
|
|
Plotly: Is used for plotting graphs easily. This works very well in interactive web applications. With this, we can make different types of basic charts like line, pie, scatter, heat maps, polar plots and so on. Whatever visualisation we can think of, we can easily plot a graph of it using Plotly. |
|
|
from matplotlib import pyplot as plt: == import matplotlib.pyplot as plt. |
|
|
bokeh.plotting.figure.circle() |
|
|
Inside/outside edges/margins of plotted images |
|
|
Avoid two or multiple plots being wrongly/incorrectly/unnecessarily mixed/overlap |
|
|
Recall (Sensitivity or True Positive Rate) in machine learning |
|
|
Plot a figure with a colored arrow between text lines/steps |
|
|
Python IDLE restart during execution |
|
|
|
|
|
|
|
|
Box and Whisker plots |
|
|
Get the date and time (a past date) of N days ago |
|
|
std() and pstdev() standard deviation in csv/dataframe |
|
|
Waterfall plot |
|
|
Safely use credentials (username and password) in Python project |
|
|
Tableau integration with Python |
|
|
Percentage |
|
|
Color the Tables in pptx (PowerPoint) |
|
|
Define/measure the size of PowerPoint/pptx slides |
|
|
Color and rotate/vertical text in pptx |
|
|
Selecting only numeric/number columns, and then select two specific columns for plot |
|
|
Hide x-axis tick labels (only show some labels) where x values are under certain conditions |
|
Create table with merged cells on pptx |
|
|
Plot images from different DataFrame in a single row |
|
|
Cheatsheet of image plot |
|
|
plt.subplots_adjust(left=None, bottom=None, right=None, top=None,
wspace=None, hspace=None), or plt.subplots_adjust(wspace=0, hspace=0) |
|
|
Font size of tick labels in plot |
|
|
PowerPoint (pptx) cheatsheet |
|
|
Font size of a (single/multiple) cell in table in PowerPoint |
|
|
Change the width of the cell in ppt |
|
|
Only use the first 4 characters in the headers of the table for pptx/dataframe |
|
|
Populate the table with logarithmic format in pptx |
|
|
Cheatsheet of PySpark (for SparkSQL) and DataFrames |
|
|
Comparative overview of multivariate statistical methods (Correlation Analysis, Regression Analysis, Factor Analysis, Cluster Analysis, Principal Component Analysis (PCA), Canonical Correlation Analysis, Discriminant Analysis, Path Analysis, Structural Equation Modeling (SEM), Multivariate Analysis of Variance (MANOVA), Analysis of Covariance (ANCOVA) ): purposes, variables, and outputs |
|
|
HTTP(Hypertext Transfer Protocol)/URL (Uniform Resource Locator) |
|
|
Cheatsheet of Python programming |
|
|
Plotly.com |
|
|
Cheatsheet of Pandas and DataFrame |
|
|
import pandas as pd |
|
|
pandas for CSV |
|
|
loc[] and iloc[] |
|
|
describe(): Print statistic summary of the data. |
|
|
.fillna(): Replace empty cells with anything. |
|
|
del |
|
|
head() |
|
|
path_or_buf: A string path to the file to write or a StringIO |
|
|
inplace=True |
|
|
.T (Transfer of array in Python) |
|
|
predict() |
|
|
password(): password(text='', title='', default='', mask=''). |
|
|
write() |
|
|
.screenshot() |
|
|
from pptx import Presentation |
|
|
from pptx.util import Inches |
|
|
from pptx.util import Pt |
|
|
Add and work on a new slide |
|
|
from pptx.dml.color import RGBColor |
|
|
.slide_layouts[0] |
|
|
.slide_layouts[1] |
|
|
.slide_layouts[2] |
|
|
.slide_layouts[3] |
|
|
.slide_layouts[4] |
|
|
.slide_layouts[5] |
|
|
.slide_layouts[6]: Simple pptx slide generation. |
|
|
.slide_layouts[7] |
|
|
.slide_layouts[8] |
|
|
.slide_layouts[9] |
|
|
.slide_layouts[10] |
|
|
.slide_layouts[11] |
|
|
Presentation() |
|
|
paragraphs[] |
|
|
from pptx.enum.shapes import MSO_AUTO_SHAPE_TYPE |
|
|
Quit from Powerpoint show |
|
| |
|
|
from pptx.chart.data import ChartData |
|
|
from pptx.enum.chart import XL_CHART_TYPE |
|
|
.categories |
|
|
.add_series() |
|
|
.add_chart() |
|
|
.LINE |
|
|
.chart |
|
|
. |
|
|
.series |
|
|
.format.line |
|
|
Image rotation in pptx (.rotation=) |
|
|
MyPresentation.save() |
|
|
from PIL import Image |
|
|
.placeholders[] |
|
|
.slide_width |
|
|
slide_height |
|
|
os.path.basename() |
|
|
os.path.abspath() |
|
|
os.path.dirname() |
|
|
os.path.expanduser() |
|
|
os.path.expandvars() |
|
|
os.path.normcase() |
|
|
os.path.lexists() |
|
|
os.path.isabs() |
|
|
os.path.isfile() |
|
|
os.path.isdir() |
|
|
os.path.islink() |
|
|
os.path.ismount() |
|
|
os.path.getatime() |
|
|
os.path.getmtime() |
|
|
os.path.samefile() |
|
|
os.path.getsize() |
|
| POMDP (Partially Observable Markov Decision Process) |
|
| Policy search algorithms and "normal" reinforcement learning (RL) algorithms |
|
| Policy Search Algorithms |
|
| Stationary and Non-Stationary State Transitions in Markov Decision Process (MDP) |
|
| State transition function (probability) in reinforcement learning |
|
| Policy iteration versus value iteration |
|
| Policy iteration |
|
| Markov Decision Process (MDP) |
|
| Finding a correct loss (risk, objective) function for a specific problem |
|
| (Forward and backward) propagation equations |
|
Linear correlation between two variables with Pearson Correlation Coefficient, Spearman Rank Correlation Coefficient, Kendall's Tau, Linear Regression, Coefficient of Determination and Correlation Ratio
|
|
| Model = architecture + parameters |
|
| Structure of PowerPoint reports |
|
| Automated Content (image and text) Insertion and Formatting in PowerPoint Slides Using Python |
|
| Data parallelism in distributed training |
|
| Linear model versus polynomial model |
|
| Posterior distribution |
|
| Distribution of θ (parameter distribution) in ML |
|
| Population Distribution in ML |
|
| Updating Hypothesis (ĥ) and/or Parameter θ^ in ML |
|
| Probably Approximately Correct (PAC) learning |
|
| Polynomial models in ML |
|
| Choice of parameters for training models |
|
| Minimum A Priori (MAP) |
|
| Mean Average Precision (MAP) |
|
| Maximum A Posteriori (MAP) |
|
| "Norm" of parameters, and L1 Norm (Manhattan Norm) and L2 Norm (Euclidean Norm) |
|
| Conditional probability |
|
| Posterior probability and prior probability |
|
| Comparison between Poisson distribution, Gaussian (normal) distribution and logistic regression |
|
| Single parameter estimation versus multiple parameter estimation |
|
| Parameterizations in ML |
|
| Exponential Family: Parameter, Sufficient Statistic, Natural Parameter, Base Measure and Log-Partition Function (Bernoulli distribution and Gaussian distribution) |
|
| Bayesian Probability, Bayesian Statistics (Distribution Over a Distribution), versus Bayesian Inference |
|
| Parameters, features and examples in ML |
|
Comparison between mean squared error (MSE), absolute error (L1 Loss) and fourth-power loss
|
|
| Practice ML projects for beginners |
|
| Perceptron algorithm and logistic regression |
|
| Actual Probability of Deviation |
|
| Deviation Probability (Hoeffding Bound) |
|
| Core Steps/Procedure/Designing of Machine Learning |
|
| Pearson Correlation Coefficient/Pearson's r/Correlation Coefficient |
|
| Point-Biserial Correlation |
|
| Types of predictions with Supervised Learning |
|
| Cocktail party problem |
|
| Hypothesis (predicted output (h(x))) |
|
| Parametric learning algorithm |
|
| Non-parametric learning algorithm |
|
| Update parameters θj using gradient of the loss function |
|
| Perceptron algorithm |
|
| Bandwidth parameter (τ) in LWR and KDE |
|
| Probability density function (PDF): comparisons between (normal (gaussian) distribution, uniform distribution, exponential distribution and poisson distribution) |
|
| |
|
| Plot workflow: Create new empty column in DataFrame, Move the cells in a column to another column under certain condition, Select specific columns for scatter plot |
| Pop-up windows/messages |
tkinter, ctypes, easygui |
| File name, folder name. {}{}....format. Manipulation of file and folder names (rename file name and folder name): i) Create a new folder and then copy all files from a folder to the new folder and rename the file, and then open the file. If the folder exists, then no file will be copied, but the file will still be opened. ii) Print and export the folder names and file names (with or without extensions) from a folder into a text file. iii) csv2image filename. |
Introduction |
| Open an image to file from URL (webpage), then it can be saved in PC |
code, code. open with color changed. |
| Calculate the coordinates of a point in a given rectangle and the distance of a given point to a line |
code |
| Compute the difference between two images by using Structural Similarity Index
with "pip install --upgrade imutils" |
code. code |
| Reference list items by position |
code1, code2 |
| Percentage Estimation of Pixels Within a Defined Intensity Contrast Range in Grayscale Images |
|
| csv workflow: Read into dataframe, Select a specific column from DataFrame, Select several specific columns to plot |
| Change/convert a colored image to a grey image(, and then show pixel values). |
cv2, cv2, cv2/skimage. cv2/skimage. PIL. matplotlib. |
| code. code |
code |
| plt.plot(): Combine multiple plots and plot continuous curve, solid green ('-g'), dashed green ('--g'), dashdot ('.g'), dotted (':g'). Plot by different grouping and summing. code. |
Introduction |
| Pandas |
Introduction. Is a must for data-science since it is one of the most used packages for analyzing data, data exploration, and manipulation. It provides fast, expressive, and flexible data structures to easily (and intuitively) work with structured (tabular, multidimensional, potentially heterogeneous) and time-series data. Pandas can do well:
Easy handling of missing floating point and non-floating point data
Columns can be inserted and deleted from DataFrame and higher dimensional objects
Automatic and explicit data alignment: objects can be aligned or be ignored to a set of labels
Powerful, flexible grouping
Convert ragged, differently-indexed data in other data structures into
DataFrame objects
Intelligent label-based slicing, fancy indexing, and subsetting of large data sets
Intuitive merging and joining data sets
Reshaping and pivoting of data sets
Hierarchical labeling of axes
Robust IO tools for loading data from flat files (CSV and delimited), Excel files, databases
Compatible with ultrafast HDF5 format
Time series-specific functionality: date range generation and frequency conversion, moving window statistics,
moving window linear regressions, date shifting and lagging |
| pandas.read_csv() |
CSV: Both read_csv() and read_table() use the same parsing code to intelligently convert tabular data into a DataFrame.
pandas.read_csv(filepath_or_buffer, sep=NoDefault.no_default, delimiter=None, header='infer', names=NoDefault.no_default, index_col=None, usecols=None, squeeze=False, prefix=NoDefault.no_default, mangle_dupe_cols=True, dtype=None, engine=None, converters=None, true_values=None, false_values=None, skipinitialspace=False, skiprows=None, skipfooter=0, nrows=None, na_values=None, keep_default_na=True, na_filter=True, verbose=False, skip_blank_lines=True, parse_dates=False, infer_datetime_format=False, keep_date_col=False, date_parser=None, dayfirst=False, cache_dates=True, iterator=False, chunksize=None, compression='infer', thousands=None, decimal='.', lineterminator=None, quotechar='"', quoting=0, doublequote=True, escapechar=None, comment=None, encoding=None, encoding_errors='strict', dialect=None, error_bad_lines=None, warn_bad_lines=None, on_bad_lines=None, delim_whitespace=False, low_memory=True, memory_map=False, float_precision=None, storage_options=None). code. |
| pandas.DataFrame.to_csv() |
CSV: DataFrame.to_csv(path_or_buf=None, sep=',', na_rep='', float_format=None, columns=None, header=True, index=True, index_label=None, mode='w', encoding=None, compression='infer', quoting=None, quotechar='"', line_terminator=None, chunksize=None, date_format=None, doublequote=True, escapechar=None, decimal='.', errors='strict', storage_options=None). Write the contents of the Frame into a CSV file: code. |
| pandas.read_table() |
CSV: Both read_csv() and read_table() use the same parsing code to intelligently convert tabular data into a DataFrame. Read general delimited file into DataFrame.
pandas.read_table(filepath_or_buffer, sep=NoDefault.no_default, delimiter=None, header='infer', names=NoDefault.no_default, index_col=None, usecols=None, squeeze=False, prefix=NoDefault.no_default, mangle_dupe_cols=True, dtype=None, engine=None, converters=None, true_values=None, false_values=None, skipinitialspace=False, skiprows=None, skipfooter=0, nrows=None, na_values=None, keep_default_na=True, na_filter=True, verbose=False, skip_blank_lines=True, parse_dates=False, infer_datetime_format=False, keep_date_col=False, date_parser=None, dayfirst=False, cache_dates=True, iterator=False, chunksize=None, compression='infer', thousands=None, decimal='.', lineterminator=None, quotechar='"', quoting=0, doublequote=True, escapechar=None, comment=None, encoding=None, dialect=None, error_bad_lines=None, warn_bad_lines=None, on_bad_lines=None, encoding_errors='strict', delim_whitespace=False, low_memory=True, memory_map=False, float_precision=None)[source]. code. |
| .groupby() |
sort/group columns. (code). CSV: (code) |
| info() |
Print information in CSV. (code) |
| .to_datetime() |
Change data format. (code) |
| parse_dates |
CSV: if True then index will be parsed as dates (False by default). You can specify more complicated
options to parse a subset of columns or a combination of columns into a single date column (list of ints or names,
list of lists, or dict) [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column [[1, 3]] -> combine
columns 1 and 3 and parse as a single date column {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result
‘foo’ |
| pop() |
Removes the item at the given index from the list and returns the removed item. (code) |
sys.platform
|
(code) |
sys.path
|
Is a list of strings that specifies the search path for modules. Basically this tells Python what locations to look in when it tries to import a module. code. |
| sys.platform |
Is a platform identifier. You can use this to append platform specific modules to sys.path, import different modules depending on platform or run different pieces of code. code. |
| from PIL import ImageTk |
|
| tkinter popup windows |
Introduction |
| .place() |
(code) |
| .pack() |
(code). (code). |
| PhotoImage() |
(code) |
| pady= |
(code) |
| padx= |
(code) |
PyAutoGUI |
|
| pyautogui.position() |
(Code) |
| press() |
Several key strings which will be pressed down in order, and then released in reverse order. Strings can be passed to this function: '\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(',
')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`', 'a', 'b', 'c', 'd', 'e','f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~',
'accept', 'add', 'alt', 'altleft', 'altright', 'apps', 'backspace',
'browserback', 'browserfavorites', 'browserforward', 'browserhome',
'browserrefresh', 'browsersearch', 'browserstop', 'capslock', 'clear',
'convert', 'ctrl', 'ctrlleft', 'ctrlright', 'decimal', 'del', 'delete',
'divide', 'down', 'end', 'enter', 'esc', 'escape', 'execute', 'f1', 'f10',
'f11', 'f12', 'f13', 'f14', 'f15', 'f16', 'f17', 'f18', 'f19', 'f2', 'f20',
'f21', 'f22', 'f23', 'f24', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9',
'final', 'fn', 'hanguel', 'hangul', 'hanja', 'help', 'home', 'insert', 'junja',
'kana', 'kanji', 'launchapp1', 'launchapp2', 'launchmail',
'launchmediaselect', 'left', 'modechange', 'multiply', 'nexttrack',
'nonconvert', 'num0', 'num1', 'num2', 'num3', 'num4', 'num5', 'num6',
'num7', 'num8', 'num9', 'numlock', 'pagedown', 'pageup', 'pause', 'pgdn',
'pgup', 'playpause', 'prevtrack', 'print', 'printscreen', 'prntscrn',
'prtsc', 'prtscr', 'return', 'right', 'scrolllock', 'select', 'separator',
'shift', 'shiftleft', 'shiftright', 'sleep', 'space', 'stop', 'subtract', 'tab',
'up', 'volumedown', 'volumemute', 'volumeup', 'win', 'winleft', 'winright', 'yen',
'command', 'option', 'optionleft', 'optionright', 'spacebar'. (code) |
| hotkey() |
| keyDown() |
| keyUp() |
| winleft and winright |
Introduction |
| .pixelMatchesColor() |
Introduction |
| pyautogui.PAUSE = 1 |
Wait 1 second after performing an action |
| pyautogui.FAILSAFE |
"pyautogui.FAILSAFE = True": moving the mouse cursor to the
upper-left corner of the screen will cause PyAutoGUI to raise the pyautogui
.FailSafeException exception. "pyautogui.FAILSAFE = False": disable this feature. |
| enter |
Press the enter key or add "\n" in .typewrite function. Introduction |
| popup windows with pyautogui |
Introduction |
| pyautogui.pixel(x,y)[] |
e.g. pyautogui.pixel(30,60)[0] == 0, here, [0] = R (red) value, [1] = G (green) value, [2] = B (blue) value. |
| Mouse clicks |
(code) |
| mouseUp() |
Move the mouse and then release it. .mouseUp(x=moveToX, y=moveToY, button='left'). (code) |
mouseDown() |
Move the mouse and then release it. .mouseDown(x=moveToX, y=moveToY, button='left'). (code) |
| .middleClick() |
.middleClick(x=moveToX, y=moveToY) (Code) |
| dragTo() |
dragTo(x, y, duration=num_seconds) drags mouse to XY. (code). |
| dragRel() |
dragRel(xOffset, yOffset, duration=num_seconds) drags mouse relative to its current position. (code). |
| displayMousePosition() |
(code) |
| .moveRel() |
.moveRel(xOffset, yOffset, duration=num_seconds). (Code). |
| .moveTo(x, y, t) |
x and y: coordinates, and t: time. By default, pyautogui.MINIMUM_DURATION is 0.1. (Code) |
| .tripleClick() |
.tripleClick(x=moveToX, y=moveToY) (Code) |
| .typewrite() |
E.g. .typewrite('Hello world!\n', interval=secs_between_keys); .typewrite(['a', 'b', 'c', 'left', 'backspace', 'enter', 'f1'], interval=secs_between_keys).
(Code) |
| .typewrite() |
E.g. .typewrite('Hello world!\n', interval=secs_between_keys); .typewrite(['a', 'b', 'c', 'left', 'backspace', 'enter', 'f1'], interval=secs_between_keys).
(Code) |
| .rjust() |
Right-justify/align them so
that they take up the same amount of space, whether the coordinate has one, two, three, or four letters or digits. (code). |
| alert() |
alert(text='', tilte='', button='Ok'). (code) |
| confirm() |
confirm(text='', title='', buttons=['OK', 'Cancel']) (code) |
| prompt() |
prompt(text='', title='', default=''). (code) |
| .locateOnScreen() |
.locateOnScreen('looksLikeThis.png') returns (left, top, width, height) on the image which the screenshot is taken from. Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match.
This function mostly is useless. Introduction. (code) |
| .locateCenterOnScreen() |
Uses pyscreeze. x, y = MySearch_img to get the x- and y-coordinates of centers of the feature. Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match. Introduction. (code) |
| .locateAllOnScreen() |
Difference between pyautogui.locateOnScreen("anImage") and pyautogui.locateOnScreen("anImage", minSearchTime=): minSearchTime = amount of time in seconds to repeat taking screenshots and trying to locate a match.
(code) |
| PyKeyboard Attributes |
|
| 'a', 'b', 'c', 'A', 'B', 'C', '1', '2', '3',
'!', '@', '#', and so on |
The keys for single characters |
| 'enter' (or 'return' or '\n') |
The enter key |
| 'esc' |
The esc key |
| 'shiftleft', 'shiftright' |
The left and right shift keys |
| 'altleft', 'altright' |
The left and right alt keys |
| 'ctrlleft', 'ctrlright' |
The left and right ctrl keys |
| 'tab' (or '\t') |
The tab key |
| 'backspace', 'delete' |
The backspace and delete keys |
| 'pageup', 'pagedown' |
The page up and page down keys |
| 'home', 'end' |
The home and end keys |
| 'up', 'down', 'left', 'right' |
The up, down, left, and right arrow keys |
| 'f1', 'f2', 'f3', and so on |
The F1 to F12 keys |
| 'volumemute', 'volumedown', 'volumeup' |
The mute, volume down, and volume up
keys (some keyboards do not have these
keys, but your operating system will still
be able to understand these simulated
keypresses) |
| 'pause' |
The pause key |
| 'capslock', 'numlock', 'scrolllock' |
The caps lock, num lock, and scroll lock keys |
| 'insert' |
The ins or insert key |
| 'printscreen' |
The prtsc or print screen key |
| 'winleft', 'winright' |
The left and right win keys (on Windows) |
| 'command' |
The Command (z) key (on OS X) |
| 'option' |
The option key (on OS X) |
ppt |
| Text Format (e.g. ALIGN) in pptx |
PP_ALIGN has CENTER, DISTRIBUTE, JUSTIFY, JUSTIFY_LOW, LEFT, RIGHT, THAI_DISTRIBUTE, and MIXED, and MSO_VERTICAL_ANCHOR has TOP, MIDDLE, BOTTOM, MIXED. Introduction |
| pass() |
Means "Do nothing": Tell Python that you’re just not
quite ready to finish writing the class, so that it does't throw an error message. Instruction. |
| __pos__ |
|
| __pow__ |
|
| patterns |
(Read-only) Patterns to allow matching event paths. |
| path |
The path that this watch monitors. |
| class watchdog.observers.polling.PollingObserver(timeout=1) |
| class watchdog.observers.polling.PollingObserverVFS(stat, listdir, polling_interval=1) |
path(id) |
Returns path for id. None if id is unknown to this snapshot. |
paths |
Set of file/directory paths in the snapshot. |
| math.pi |
Returns the mathematical constant pi (3.141592 . . .). |
| math.pow(x, y) |
Returns x raised to the power y. code1, code2. |
| pow(x, y, z) |
x raise to the power y and reminder by z. |
| patterns=['*.*'] |
(code) |
| .PatternMatchingEventHandler |
(code) |
| os.path.join(path |
Introduction. code. code. (code) |
| os.path.exists() |
(code) (code) |
| path |
code. (code) |
| .pack |
code. code. |
| Floating Point |
It can store number with a fractional part |
| property |
The word
are synonymous with attribute. code. code. Change a attribute, code. |
| os.path.splitext() method |
Is used to split the path name into a pair root and ext. Here, ext stands for extension and has the extension portion of the specified path while root is everything except ext part. ext is empty if specified path does not have any extension. If the specified path has leading period (‘.’), it then will be ignored. Introduction. Examples are:
| path name |
root |
ext |
| /Desktop/file.txt /home/User/Desktop/file .txt |
/Desktop /home/User/Desktop |
{empty} |
| file.py |
file |
.py |
| .txt |
.txt |
{empty} |
|
| pyperclip.paste() |
(code). (code). |
| Py2exe |
(code) |
| pyinstaller |
(code) |
| scipy.linalg.pascal |
Create a Pascal matrix. |
| linalg.pinv(a[, rcond, hermitian]) |
Compute the (Moore-Penrose) pseudo-inverse of a matrix. |
| Pyglet |
Is an excellent choice for an object-oriented programming interface in developing games. In fact, it also finds use in developing other visually-rich applications for Mac OS X, Windows, and Linux. In the 90s, when people were bored, they resorted to playing Minecraft on their computers. Pyglet is the engine behind Minecraft. |
| PyGame |
Provides an extremely easy interface to the Simple Directmedia Library (SDL) platform-independent graphic, audio, and input libraries. |
| Python Twisted |
Is an event-driven networking engine, Twisted is written in Python, and licensed under the open-source MIT license. |
| PIL |
Pillow is a friendly fork of PIL (Python Imaging Library), but is more user-friendly. If you work with images, then Pillow is your best friend. code1, code2. |
| from PIL import Image |
code1, code2. |
| skimage.measure.perimeter(image[, neighbourhood]) |
Calculate total perimeter of all objects in binary image. |
| skimage.measure.points_in_poly |
Test whether points lie inside a polygon. |
| skimage.measure.profile_line(img, src, dst) |
Return the intensity profile of an image measured along a scan line. |
| import pyperclip |
(code) |
| pywin32 |
Provides useful methods and class for interaction with Windows, as the name suggests. |
| PyGTK |
Lets you easily create programs with a GUI (Graphical User Interface) with Python. |
| Pipenv |
Is a production-ready tool that aims to bring the best of all packaging worlds to the Python world. The cardinal purpose is to provide users with a working environment which is easy to set up. Pipenv, the “Python Development Workflow for Humans” was created by Kenneth Reitz for managing package discrepancies. |
| |
|
| |
|
| Pydot |
Is used for generating complex oriented and non-oriented graphs. It is specially used while developing algorithms based on neural networks and decision trees. |
| PyOD |
As the name suggests, it is a Python toolkit for detecting outliers in multivariate data. It provides access to a wide range of outlier detection algorithms. Outlier detection, also known as anomaly detection, refers to the identification of rare items, events or observations which differ from the general distribution of a population. |
| Pint |
|
| Pynance |
Work wonders for a stock market trader. It is an open-source python package that retrieves, analyses and visualizes the data from stock market derivatives. With this library in hand you can generate labels and features for machine learning models. |
| Pyfolio |
Evaluates the trading performance with the help of Pyfolio. It provides risk analysis reports and performance results of financial portfolios based on the returns. This was developed by Quantopian and works well with Zipline which is a backtesting library (will discuss later). Pyfolio specializes in creating tear sheet models and bayesian analysis. |
| cv2.putText() |
cv2.putText(image, text, org, font, fontScale, color[, thickness[, lineType[, bottomLeftOrigin]]]) is used to draw a text string on any image. The font types are FONT_HERSHEY_SIMPLEX = 0, FONT_HERSHEY_PLAIN = 1, FONT_HERSHEY_DUPLEX = 2, FONT_HERSHEY_COMPLEX = 3, FONT_HERSHEY_TRIPLEX = 4, FONT_HERSHEY_COMPLEX_SMALL = 5, FONT_HERSHEY_SCRIPT_SIMPLEX = 6, FONT_HERSHEY_SCRIPT_COMPLEX = 7, and FONT_ITALIC = 16. The thickness of the line is in pixel. lineType: This is an optional parameter.It gives the type of the line to be used. bottomLeftOrigin: This is an optional parameter. When it is true, the image data origin is at the bottom-left corner. Otherwise, it is at the top-left corner. code. (code) |
| .random.permutation() |
code. |
| Search/print/output the rows |
CSV: Print the rows if their cell values are greater than a specific value, in the csv file with numbers only; output the rows if the cell value is in a specific range. Introduction. code. |
| PyBrain |
Contains algorithms for neural networks that can be used by entry-level students yet can be used for state-of-the-art research. The goal is to offer simple, flexible yet sophisticated and powerful algorithms for machine learning with many pre-determined environments to test and compare your algorithms. |
| t.penup()/t.pu()/t.up() |
Pull the pen up – no drawing when moving. (code) |
| t.pendown()/t.pd()/t.down() |
Pull the pen down – drawing when moving. (code) |
| .getcanvas().postscript(file="") |
Picture saved in current working directory. (code) |
| .set_window_position() |
(code) |
| .position |
(code) |
| from pynput.mouse import Button |
(code) |
| from pynput.mouse import Controller |
(code) |
| .press(Button.left) |
(code) |
| .press(Button.right) |
(code) |
| space bar on keyboard |
Press space bar. Introduction |
| PyScreeze |
Is a simple, cross-platform screenshot module for Python 2 and 3. |
| hotkey('p') |
Introduction |
| press(pagedown) |
Introduction |
| press(pagedown) |
Introduction |
| PyGetWindow |
A simple, cross-platform module for obtaining GUI information on application's windows. |
| .Popen() |
Opens a pipe from a command. This pipe allows the command to send its output to another command. Introduction |
| from time import perf_counter |
(code) |
| .button_pressed() |
(code) |
| os.path.getctime |
(code)(code) |
| make_pipeline |
(code). |
| PySimpleGUI27 |
(code). |
| .switch_to. and .switch_to.parent_frame() |
Introduction |
| Pt |
E.g. run.font.size = Pt(15). Inches, Emu, Cm, Mm, Pt, and Px are base class for length classes, providing properties for converting length values to convenient units. |
| Px |
Inches, Emu, Cm, Mm, Pt, and Px are base class for length classes, providing properties for converting length values to convenient units. |
| os.path.normpath() |
Normalize/format the path string into a proper string for the OS (code). |
| |
|
| |
|
| |
|
