| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Skills Network Labs (SN Labs, IBM) |
Introduction |
| Mask R-CNN (Mask Region-based Convolutional Neural Network) |
Introduction |
| Google Cloud Natural Language API |
Introduction |
| Semantic Segmentation Using U-Net with EfficientNet and Pixelshuffle |
Introduction |
| Attention-Guided Neural Network (AGNN) |
Introduction |
| ResNet (Residual Network)
|
Introduction |
| Parametric and non-parametric learning algorithms |
Introduction |
| Ingesting data in Hadoop (Sqoop, Flume, Kafka, NiFi) |
Introduction |
| Hadoop MapReduce used by Google, Netflix, Amazon and Machine Learning |
Introduction |
| Impact of corpus narrowness on language model training |
Introduction |
| Comparison between Naive Bayes algorithms and Bayesian machine learning techniques |
Introduction |
| n-grams |
Introduction |
| Feedforward neural network |
Introduction |
| Biological (human brain) and AI neural networks |
Introduction |
| Nearest-neighbor (NN) classification |
Introduction |
| Node consistency |
Introduction |
| Convert a number type to another |
Introduction |
| Try a number of times before exception or fail |
Introduction |
| Check updated new files in a folder |
Introduction |
| Find nearest white pixel to a given pixel location on an binary image |
Introduction |
| Dot notation |
Introduction |
| Non-linearity in machine learning |
Introduction |
| t-SNE (t-distributed stochastic neighbor embedding, from sklearn.manifold import TSNE) |
Introduction |
| Trainable and non-trainable layers |
Introduction |
| .norm() (Taxicab Norm, Manhattan Norm, Euclidian Norm and Vector Max Norm) |
Introduction |
| .new() |
Introduction |
| tf.keras.layers.normalization |
Introduction |
| (Deep) neural network |
Introduction |
| ◆ |
Neural networks with TensorFlow |
Introduction |
| ◆ |
(Deep) Convolutional neural networks (CNN) |
Introduction |
| Use __name__ to control execution of the code |
code |
| Open a new tab in an application window |
Introduction |
| Create a new presentation |
Introduction |
| Add a new slide into an existing ppt or a created ppt file |
Introduction |
| Launch the existing opened application if there is or start a new one if there is not |
Introduction |
| Check if all the (and how many, length of a string) characters in the text are digits/numbers |
(Code) |
| Convert capital alphabet letters/characters to number |
Introduction |
| Get the name of the current/most front window |
Introduction |
| Notepad window |
Introduction |
| Check to see if or get a window with a name containing specific titles or texts |
Introduction |
| Option/selection/choice methods ("pop-up windows of Yes and No ") |
Introduction |
| Calculate/pass the arbitrary (any) number of variables or input arguments |
Introduction |
| Convert strings to number (integers/float) |
Introduction |
| Merge/combine two text files into a new text file, add a new line to the beginning of a text file |
Introduction |
| Sum a list of numbers in any length |
Introduction |
| Mixing of using numbers and strings by conversions |
Introduction |
| Build databases with different/uncertain number of members |
Introduction |
| Cheatsheet of numpy |
Introduction |
| Numpy: Access the element at the second row, the third entry, access a specific row or a column, access some elements (submatrix), or replace/modify an element in the array, print a transfer of an array, access array under conditions or filtering |
Introduction |
| Calculations in DataFrame:
Add a column, calculate for a new column, delete a column, all the rows with values greater than 30 in "Score A" column |
Introduction |
Handle NaN value in DataFrame, replace empty cells with ...
|
Introduction |
Count the numbers of uppercase letters, lowercase letters and spaces in a string and then swap the cases of the letters.
|
code |
| Swap two numbers |
code |
| Copy images to a different folder/Save an image in a new folder |
code, code. code. code. |
| Count the number of lines (rows) and columns in a txt (and a csv) file, count different numbers in each region in a column, count missing or not available values |
CSV: Introduction. code. |
| Search position of numbers |
code |
| Loop through numbers in a range |
code |
| Find/search birthyear by name |
code |
| Find the greatest of three numbers |
code1, code2, code3. |
| Reverse the digits of a given number |
code |
| Handle "No Results", "Not Found" (Error vs. Exceptions) |
Introduction |
| Check if file exists or not (Cannot find a specific file?) |
Introduction |
| Check if an item/element is in a list or not |
Introduction |
| Add a new slide into an existing ppt, or work on existing slides |
Introduction |
| Check file existence with partial filename |
Introduction |
| Print/obtain a specific digit in a number, e.g. in a cell value in cvs |
Introduction |
| Empty and None |
Introduction |
| Check whether a file is empty or not |
Introduction |
| Find files with a specific file extension/type or with file names ending with specific characters |
Introduction |
| Watchdog for monitoring specific file or files with specific extension and/or file names |
Introduction |
| Move(remove) all files from original folder in a directory to a new directory |
Introduction |
| Check whether or not a cell value in a column of a CSV file matchs a value in a column of another CSV file, then do something: e.g. add a value to another column of a csv file |
Introduction |
| Check if CSV cell value is NaN |
Introduction |
| Time and date used as a file/folder name stamp (e.g. duplicate a file in the same folder) |
Introduction |
| Count how many (number) files and folders in a directory |
Introduction |
| Limit event/action numbers in the event List, then stop |
Introduction |
| Modify file path/directory by changing folder names by merging a list |
Introduction |
| Find the file names of the images in a pptx |
Introduction |
| Merge/combine two pptx files into one, including merging the pptx files with the words in a sentence as file names (not all words has pptx files) |
Introduction |
| Libraries used to convert incident documents into numerical vectors |
Introduction |
| Natural Language Processing (NLP) |
Introduction |
| ◆ |
Keyword extraction methods from documents in Natural Language Processing (NLP) |
|
Introduction |
| Count occurrence/nubmer of words/phrase in a text file |
Introduction |
| Extract pdf pages to form new pdf files |
Introduction |
| Generate text file with the bank of collecting all words, characters and strings from news |
Introduction |
| Remove \n in string or new line in txt/text file |
Introduction |
| Support-vector machines(SVM)/support-vector networks(SVN) |
Introduction |
| Nearest-neighbor (NN) cluster removal |
Introduction |
| Denoising/remove noise in images |
Introduction |
| Non-zero (Nonzero) pixel values from an image |
Introduction |
| |
|
| List of notations for machine learning application to wafers |
Introduction |
| Evaluation (Precision and Recall) in Text classification with Naive Bayes |
Introduction |
| Precision, Recall, False Positive Rate, and False Negative Rate (Miss Rate or False Negative Proportion) |
Introduction |
| Text classification with Naive Bayes |
Introduction |
| Stationary and Non-Stationary State Transitions in Markov Decision Process (MDP) |
Introduction |
| Q-Learning with Function Approximation (Deep Q-Network - DQN) |
Introduction |
| Grid world navigation |
Introduction |
| Nonlinear extensions of Independent Component Analysis (ICA) |
Introduction |
| Bayesian networks |
Introduction |
| Maximum Likelihood Estimation (MLE) of single Gaussian (normal) distribution |
Introduction |
| Number (size) of features in ML |
Introduction |
| Normalizing (normalization of) input in neural network |
Introduction |
| Logistic regression as a one-neuron/single-layer neural network (connection between linear & activation parts) |
Introduction |
| Number of neurons and layers in neural network |
Introduction |
| Edge detection of images in neural network |
Introduction |
Neural network vs. end-to-end learning vs. black box model
|
Introduction |
| Softmax regression (multinomial logistic regression)/softmax multi-class network/softmax classifier |
Introduction |
| Neuron (= linear + activation) introduction |
Introduction |
| No Free Lunch Theorems |
Introduction |
| "Norm" of parameters, and L1 Norm (Manhattan Norm) and L2 Norm (Euclidean Norm) |
Introduction |
| Logistic regression and Naive Bayes |
Introduction |
| Apple Neural Engine (ANE) |
Introduction |
| Feature vector and number of features |
Introduction |
| Single Naive Bayes (Gaussian Naive Bayes) versus Multinomial Naive Bayes |
Introduction |
| Artificial Neural Networks (ANNs) |
Introduction |
| Gaussian distribution and standard gaussian distribution (multivariate normal distribution) |
Introduction |
| K-Nearest Neighbours (KNN algorithm):sklearn, model_selection, train_test_split, preprocessing, StandardScaler, transform(), fit_transform(), .fit(), neighbors, KNeighborsClassifier, KNeighborsClassifier(), predict(), metrics, accuracy_score(), and classification_report |
Introduction |
| Normalized ratios |
Introduction |
| Negative log likelihood (NLL) |
Introduction |
| Probability density function (PDF): comparisons between (normal (gaussian) distribution, uniform distribution, exponential distribution and poisson distribution) |
Introduction |
| Exponential Family: Parameter, Sufficient Statistic, Natural Parameter, Base Measure and Log-Partition Function (Bernoulli distribution and Gaussian distribution) |
Introduction |
| Newton's method |
Introduction |
| Newton's method versus gradient descent |
Introduction |
| Non-parametric learning algorithm |
Introduction |
| Big O notation |
Introduction |
| Epsilon cover/ε-cover/epsilon-net |
Introduction |
| CycleGAN (Cycle-Consistent Adversarial Networks) |
Introduction |
| Symbols/notations used in ML |
Introduction |
| Nonasymptotic Analysis |
Introduction |
| Two-dimensional neural network |
Introduction |
| One-dimensional neural network |
Introduction |
| Multinomial Naive Bayes algorithm |
Introduction |
| Strong machine learning and NLP departments in universities |
Introduction |
| Percentages of information received through different senses (eye, nose, ear and hand feeling) |
Introduction |
| Elastic Net |
Introduction |
| Various names or terms that describe similar concepts or techniques in ML |
Introduction |
| Nonasymptotic versus asymptotic analysis |
Introduction |
| Natural Language Processing (NLP) approaches in addressing the Failure Analysis (FA) search problem |
Introduction |
| Generative Adversarial Network (GAN) technologies |
Introduction |
| Natural Language Processing (NLP) versus Text |
Introduction |
| Recurrent Neural Networks (RNN) |
Introduction |
| (Single) Naive Bayes/Gaussian Naive Bayes |
Introduction |
| NLTK (Natural Language Toolkit) |
Introduction |
| Nearest/most similar lyrics of a sentence to a CSV file |
Introduction |
| Natural language inference |
Introduction |
| Get directory/path/file name partially |
Introduction |
| Generate a file name, folder name. {}{}....format. Manipulation of file and folder names (rename file name and folder name): i) Create a new folder (e.g. with os.mkdir) and then copy all files from a folder to the new folder and rename the file, and then open the file. If the folder exists, then no file will be copied, but the file will still be opened. ii) Print and export the folder names and file names (with or without extensions) from a folder into a text file. iii) csv2image filename. |
Introduction |
| Count number of lines in a text file |
Introduction |
| Modify HTML webpage (e.g. with graph network by adding text/hyperlink in) |
Introduction |
| Check if a variable is a number, string or integer |
Introduction |
| Count the number of the pages in a single multi-page/frame image |
Introduction |
| Check if all the (and how many, length of a string) characters in the text are digits/numbers |
Introduction |
| Create dictionary from nested (sublist) list and get the values with keys |
Introduction |
| Skip/replace empty cells/NaN value from DataFrame/CSV file |
Introduction |
| Convert between numpy array and string |
Introduction |
Continue script execution no matter whether some try fails or not (finally)
|
Introduction |
| Get header/column names from DataFrame |
Introduction |
| Iterate over rows in a DataFrame/read and print row by row (number of columns and rows, df.shape[0]/df.shape[1]) |
Introduction |
| Remove rows if (multiple) NaN is more than a number in DataFrame |
Introduction |
| Read columns with numeric values/numbers only in dataframe |
Introduction |
| Change/rename a column name/header in a CSV file |
Introduction |
| Feature importance for Multinomial Naive Bayes algorithm |
Introduction |
| |
|
| |
|
| Get the latest/newest/most recent file in a folder within certain time/days |
Introduction |
| Check if a string is empty, NaN value or space only |
Introduction |
| Check and drop negative from dataframe pandas |
Introduction |
| y axis values are not ordered (disordered) |
Introduction |
| |
|
| Compare (pattern/ratio of) two different columns, check whether column values match in DataFrame |
Introduction |
| ◆ |
Check whether one column contains number only and another column contains letters only or mixture of numbers and letters in DataFrame |
Introduction |
| ◆ |
Check the difference between two columns in DataFrame |
Introduction |
| ◆ |
|
|
| |
|
| |
|
| Delete the column/row in a CSV file if they are empty or less than a number (or header/index only) |
Introduction |
| |
|
| Network |
|
| ◆ |
networkx for network |
Introduction |
| ◆ |
igraph for clustering and network |
Introduction |
| ◆ |
Networkit: a network/link/landscape tool |
Introduction |
| ◆ |
Pyvis: An interactive geometric graph network/link/landscape |
Introduction |
| ◆ |
|
|
| ◆ |
|
|
| |
|
na_values: NA or N/A values = missing values. optional list of strings to recognize as NaN (missing values), either in addition to or in lieu of the
default set. To completely override the default values that are recognized as missing, specify keep_default_na = False.
The default NaN recognized values are [’-1.#IND’, ’1.#QNAN’, ’1.#IND’, ’-1.#QNAN’,
’#N/A’,’N/A’, ’NA’, ’#NA’, ’NULL’, ’NaN’, ’-NaN’, ’nan’, ’-nan’].
read_csv(path, na_values=[5]) =
the default values, in addition to 5 , 5.0 when interpreted as numbers are recognized as NaN;
read_csv(path, keep_default_na=False, na_values=[""]) =
only an empty field will be NaN;
read_csv(path, keep_default_na=False, na_values=["NA", "0"]) =
only NA and 0 as strings are NaN;
read_csv(path, na_values=["Nope"]) =
the default values, in addition to the string "Nope" are recognized as NaN. |
|
| __add__, __call__, __contains__, __delitem__, __delattr__, __eq__, __enter__, __ge__, __getattribute__, __getnewargs__, __getattr__, __getitem__, __gt__, __hash__, __reduce__, __iadd__, __imul__, __init_subclass__, __index__, __int__, __invert__, __new__, __neg__, __reduce_ex__, __reversed__, __rmul__, __radd__, __rand__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __ror__, __round__,__rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, __rxor__, __dir__, __doc__, __divmod__, __iter__, __le__, __lt__, __len__, __ne__, __repr__, __setattr__, __setitem__, __sizeof__, __lshift__, __sub__, __subclasshook__, __str__ |
Introduction |
| next() |
Introduction |
| Select specific columns from a DataFrame to form a new DataFrame |
Introduction |
| Add letter/commas/numbers/characters to the end/beginning of strings in a list |
Introduction |
| Replace the lines between two lines “xx” and “yy” in a text file with new lines |
Introduction |
| Selecting only numeric/number columns, and then select two specific columns for plot |
Introduction |
| Mean (average, .mean())/.sum()/maximum(.max())/minimum(.min())/number of non-null values(.count())/.median()/variance(.var())/standard deviation(.std()/pstdev()) |
Introduction |
| Plot workflow: Create new empty column in DataFrame, Move the cells in a column to another column under certain condition, Select specific columns for scatter plot |
| Cheatsheet of numbers |
Introduction |
| First N largest and smallest numbers in a list |
Introduction |
| Search/extract all the 4-digit numbers (with and without extension) from a given text |
Introduction |
| Jupyter notebooks |
Introduction |
| Normalize data in dataframe |
Introduction |
| |
|
| names |
CSV: List of column names to use as column names. To replace header existing in file, explicitly pass
header=0. |
| |
|
| nrows |
CSV: Number of rows to read out of the file. Useful to only read a small portion of a large file |
| na_rep |
CSV: A string representation of a missing value (default ‘’) |
| names |
CSV: Names in headers of csv files. code. code. |
| nunique() |
CSV: (code) |
| neighbors |
(code) |
| Num Lock |
.press("numlock") (code). |
|
Represent one of two values: True or False. code. code. code. |
| not |
Condition must be false for the statement to be true |
| if ... not in ... |
Introduction. code. |
| if not |
(code) |
| not |
|
| if ... is None |
code. |
| .name |
(code) |
| name |
A string used for identification purposes only.
It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor. |
| on_thread_start() |
Override this method instead of start(). start() calls this method.
This method is called right before this thread is started and this object’s run() method is invoked. |
| on_thread_stop() |
Override this method instead of stop(). stop() calls this method.
This method is called immediately after the thread is signaled to stop. |
| if __name__ == '__main__' |
Introduction. The processes starts reading the current file in order to execute the function specified. Without this clause, the import would first execute more process start calls, before getting to the function execution. code. code. code. code. Application example: run the page4853main3 program (as a module) through page4853main4 program. A similar example with defined functions is page4853main5.py. (code). |
| numerator |
|
| name() |
E.g. name[0] and
name[1] prints the first and second letter of the string, respectively.
However, negative indexing, -n, in a string refers to the character present at the nth position beginning from the end. Example code |
Numbers |
Python has three types of numbers: integer, floating point and complex. |
Integer/fractions/round /decimal/digits/floating
/ceil/floor |
It does not have any fractional part. Introduction. int: Example code. |
| Floating Point |
It can store number with a fractional part |
| Complex |
It can store real and imaginary parts |
| Decimal |
Those having fixed precision |
| Rational |
Those having a numerator and a denominator |
| Sets |
Introduction, and data structures. Abstraction of a mathematical set |
| |
| scipy.linalg.norm |
This function is able to return one of eight different matrix norms, or one of an infinite number of vector norms, depending on the value of the ord parameter. For tensors with rank different from 1 or 2, only ord=None is supported. It is for an old release of SciPy (version 0.14.0). |
| from numpy.linalg import norm |
(code). |
| from scipy.linalg import norm |
code |
Norms and other numbers |
| linalg.norm(x[, ord, axis, keepdims]) |
Matrix or vector norm. |
| linalg.cond(x[, p]) |
Compute the condition number of a matrix. |
| linalg.det(a) |
Compute the determinant of an array. |
| linalg.matrix_rank(M[, tol, hermitian]) |
Return matrix rank of array using SVD method |
| linalg.slogdet(a) |
Compute the sign and (natural) logarithm of the determinant of an array. |
| trace(a[, offset, axis1, axis2, dtype, out]) |
Return the sum along diagonals of the array. |
| |
| __name__/str |
name/type: Introduction. the function name. Example code |
| cv2.namedWindow |
code. |
| nrows |
Number of rows of subplots |
| ncols |
Number of columns of subplots |
numpy |
| NumPy |
Introduction. Has advanced math functions and a rudimentary scientific computing package. It is used for all things "numbers and Python." In image processing, we are mainly making use of Numpy's array functionality. code1, code2. |
| Speed comparison with and without numpy |
Introduction |
| .ndim |
(code) |
| shape/shape[] |
(code) |
| size |
(code) |
| nbytes |
(code) |
| numpy.identity(n, dtype = None) |
Return a identity matrix i.e. a square matrix with ones on the main diagonal. code. |
| Numpy splicing |
draw a grey or color point or shape in an image. code. |
| .sort(axis=0) |
code. |
| import numpy |
Actually means import numpy as numpy. |
| import numpy as np |
Is equivalent to: (Array, code)
import numpy
np = numpy
del numpy |
| .random.permutation() |
code. |
| .randint() |
Random integer. code. |
| numpy.transpose() |
Reverse or permute the axes of an array; returns the modified array. For an array a with two axes, transpose(a) gives the matrix transpose of swapping. code. |
| .argwhere() |
(code) |
| np.linspace() |
Introduction. General, code. code. |
| np.full_like |
Return a full array with the same shape and type as a given array. Code. |
| .array |
General, number array . for image. code. in csv. |
| hist() |
(code) |
| .xlabel() |
(code) |
| .ylabel() |
(code) |
| .title() |
(code) |
| .text() |
(code) |
| .xlim() |
(code) |
| .ylim() |
(code) |
| grid() |
(code) |
| .arange() |
Introduction. Can change with an increment. General, code. code. |
| reshape() |
numpy.reshape(a, newshape, order='C', 'F', or 'A' - optional)[source]. Introduction. in csv. |
| numpy.ones(shape, dtype=None, order='C', *, like=None) |
Return a new array of given shape and type, filled with ones. General, code. |
| np.eye() |
General. |
| numpy.vander |
Create a Van der Monde matrix. |
| numpy.linalg |
This function is able to return one of eight different matrix norms, or one of an infinite number of vector norms, depending on the value of the ord parameter. |
| random.rand() |
General. |
| random.random() |
Introduction. |
| random.randn(d0, d1, ..., dn) |
Return a sample (or samples) from the “standard normal” distribution. Introduction. |
| .hstack |
General, code. |
| .vstack |
General, code. |
| random.default_rng |
Random generator: General. code. |
| Count duplicates/occurrence and show unique values in csv files |
CSV: Introduction |
| Merge two csv files |
CSV: Introduction |
| Split columns and merge in csv |
CSV: Split columns and then merge the splits in a csv file. Introduction |
| Count the number of lines (rows) and columns in a txt (and a csv) file, count different numbers in each region in a column, count missing or not available values |
CSV: Introduction. code. |
| Search/print/output the rows |
CSV: Print the rows if their cell values are greater than a specific value, in the csv file with numbers only; output the rows if the cell value is in a specific range. Introduction. code. |
| Sort a csv file with column |
CSV: Instruction: used "key = operator.itemgetter()". |
| Calculation in csv |
CSV: Compares with the ones which cannot be used for math calculation, find the maximum in a column. Introduction |
| Convert between rows and columns from csv |
CSV: Convert resulting row from CSV search into a column. Introduction |
| Replace/change to new headers in a csv file |
CSV: Introduction |
| Remove duplicate cell values from a csv file |
CSV: Introduction |
| Skip rows and/or columns in csv |
CSV: Introduction. |
| |
| Nose |
Delivers an alternate test discovery and running process for unittest. This intends to mimic py.test’s behavior as much as it can. |
| NuPIC |
The Numenta Platform for Intelligent Computing (NuPIC) is a platform which aims to implement an HTM learning algorithm and make them public source as well. It is the foundation for future machine learning algorithms based on the biology of the neocortex. |
| |
|
Comparison between spaCy and Natural Language Toolkit (NLTK)
|
Introduction |
| GetKeyState(VK_NUMLOCK) |
Turn off or on Num Lock. .press("numlock") (code). |
| hotkey('n') |
(code) |
| split() |
Introduction. Split a string by dots, split a file name by dots, split a file name from its extension. code. (code). |
Navigators |
| driver.get(“http://globalsino.com”) |
Navigate to URL |
driver.navigate().to(“http://globalsino.com”)
|
Navigate to URL |
| driver.navigate().refresh() |
Refresh page |
| driver.navigate().forward() |
Navigate forwards in browser history |
| driver.navigate().back() |
Navigate backward in browser history |
| |
| NoSuchElementException() |
(code) |
| os.path.normcase() |
(code). |
| os.path.normpath() |
(code). |
| os.path.normpath(): Normalize/format the path string into a proper string for the OS |
(code) |
| np.nan |
(code) |
| numpy.all/np.all |
Introduction |
| .DataFrame() |
Introduction. .drop(),
index,
columns,
axes,
dtypes,
size,
shape,
ndim,
empty,
T,
values |
|
| Switches simulations |
|
| Diodes simulations |
|
