Machine Learning Basics - Cheat Sheet

Machine Learning Description

• Machine Learning is the scientific study of algorithms and statistical models that computer systems use in order to perform a specific task effectively without using explicit instructions.
• From the multi fields of data, brought together as either through statistical or machine learning, data is understood and extract insights from it.
• Usually there are two types of activities in data research :
• Hypothesis Driven : Given a problem, what kind of data is needed to solve them.
• Data Driven : Given a data, what kind of problems could be solved by this.

Machine Learning Life Cycle

• Identification or Understanding of the Business Problem is very important
• Collection of Data
• Primary : Existing files, tables
• Secondary : Going for surveys
• Data Cleaning, making sure to feed right format and correct data to our ML model
• Missing values imputations
• Outliers
• Generation of required columns
• Data Transformation, converting all the data to quantitative data
• Visualization Data, easy to understand the spread and relationship between data
• input: Age, Standard, Subject, Gender, School
• output: Marks
• Model Data
• Model Selection
• Descriptive
• Inferential
• Predictive
• Model Build
• Model Evaluation
• Insights / Prediction

Data Types 

• Quantitative
• Continuous
• Ratio
• Proportions can be taken by comparing
• Zero is absolute
• Eg: Salary as 0, means no salary
• Note: so we don't need to worry on data transformation
• Interval
• Proportions cannot be taken
• Zero is not absolute
• Eg: Temperature : 0 Degrees, where it means here freezing point, but there are values below this also, we can transform this to Kelvin
• Note: Data can be transform here
• Discrete
• Nominal
• Red
• Blue
• Green
• Ordinal
• Good
• Better
• Best
• Binary
• Symmetric: equal importance
• Female
• Male
• Asymmetric: significance given
• Positive
• Negative
• Qualitative
• Text
• Images
• Audio
• Vedio

Data Cleaning

• Process of turning the raw data to analyse able data
• Note: Garbage In, Garbage Out
• Methods :
• Missing Values
• Imputation using central tendency
• Mean
• 1, 2, 2,4,6,6
• 1+2+2+4+6+6
• 45/6
• Median
• Mode
• Special Cases
• Default value: Making a reasonable guess, based on the domain knowledge
• 45,34,56,12,46,34,34 – Normal Class 
• 65,89,12,90,56,60,60 – Upper Class
• Imputation through predictions
• Before or after values: also called as nearest neighbour
• Model Estimation: or called as interpolation, using a model to predict missing values, could be either Linear Regression, KNN, etc
• Drop all the missing records, please make sure it involves risk, so be careful, this should be last option
• Outliers
• As outliers values can have an affect in the model prediction.
• Variance is important, for prediction and should be controllable
• There are two types of variance we must know :
• Good Variance
• Could be genuine, need to take steps to fix this
• Such can split two data sets
• Eg: reliance super market, where we have actual retailers and small shop owners, its important to differentiate both of them
• Bad Variance
• Could be because of wrong imputation
• Identification of Outliers
• Univariate
• IQR
• Bivariate
• Multivariate
• Regression/Residual analysis
• So here we may need to build model and predict outliers
• Handling outliers
• Identification as first step:
• Eg: Age, Salary, Gender – for prediction of policy Buy/Not 
• 32, 34, 40, 1, 80
• 10L, 5L, 15L, 10K, 100L
• 1,2,1,1,3
• Scale
• zscore
• min-max
• Replace
• Remove, as a last option

Data Transformation

• All data needs to be converted to the quantitative data
• Models are stronger when the variance are small
• Transformation is very much needed for Distance or Slope or Weight based models, such as
• Linear
• Logistic
• KNN
• Transformation methods:
• Log
• MinMax
• Xnorm = (X-Xmin)/(XMax-Xmin)
• Will always contain values between 0,1
• Scaling
• NLP
• Set of preprocessing information qualative to quantative
• Text Conversion : using DTM – Document/Data Term Matrix
• One Hot Encoding
• Male/Female
• 0,1
• Red, Green, Blue
• 1,2,3
• Red- [1,0,0]
• Blue-[0,0,1]
• Green-[0,1,0]
• Each of the m features becomes a vector of length m with containing only one 1 (e.g. [r, g, b] becomes [[1,0,0],[0,1,0],[0,0,1]])
• Label Encoding
• Good, Bad, Ugly – 1,2,3
• convert each distinct feature into a random number (e.g. [r, g, b] becomes [1,2,3])

Model Data

• Modelling is the process of incorporating information into a tool which can forecast and make predictions. 
• Usually, we are dealing with statistical modelling where we want to analyze relationships between variables. 
• Formally, we want to estimate a function f (X) such that:
• Y = f (X) + E
• where 
• X = (X 1 , X 2 , ...X p ) represents the input variables
• Y represents the output variable
• E represents random error

Model Life Cycle

• Model Selection
• Descriptive
• Visualization
• Inferential
• Statistical
• When we want to understand the relationship between X and Y. We can no longer treat f ˆ as a black box since we want to understand how Y changes with respect to X = (X 1 , X 2 , ...X n )
• Predictive
• Machine Learning
• Once we have a good estimate f ˆ (X), we can use it to make predictions on new data.
• We treat f ˆ as a black box, since we only care about the accuracy of the predictions, not why or how it works.
• Model Build
• Model Evaluation
• Supervised
• Regression
• R2 score
• Adj R2 score
• Classification
• Confussion matrix
• Classification matrix
• ROC
• AUC

Model Selection

Inferential

• Test of population parameters
• Test of means
• One Sample Test
• Two Sample Test
• Anova
• Ancova
• Manova
• Mancova
• N-Way Nanova
• Test of variance
• Levene Test
• F Test
• Test of normality
• Shapiro’s Test
• Anderson Darling
• Test of proposition
• Chi X Square

Predictive

• Supervised
• Regression
• Dependent variables are continuous
• It points estimate
• Models :
• Linear Regression
• KNN for regression
• SVM for regression
• Decision Tree for regression
• Ensemble Techniques for Regression
• Basic Ensemble Techniques
• Max Voting
• Averaging
• Weighted Average
• Advanced Ensemble Techniques
• Stacking
• Blending
• Bagging
• Boosting
• Algorithms based on Bagging and Boosting
• Bagging meta-estimator
• Random Forest
• AdaBoost
• GBM
• XGB
• Light GBM
• CatBoost

• Neural Networks for Regression
• Time Series Forcasting
• Independent variables are always fixed, that is time
• Based on time will predict the dependent variables
• Models :
• Arima model
• Controlled noise
• Also, parameteric model
• Eg: Sales forcast 
• Arc and Garc model
• Uncontrolled data
• Also, nonparametric model
• Eg: Stock market
• Classification Models :
• Logistic Regression
• Naive Bayes
• Discriminant Analysis
• KNN for classification
• SVM for classification
• Descision Tree for classification
• Neural Networks for classification

• UnSupervised
• Identification of the group but do not name them
• After discovering the results, we can try to change this further to Supervised Learning
• Clustering
• Finding similarities and dissimilarities
• Find rows – grouping
• Hierarchal
• Agoloment
• Divove
• Non Heirarical
• K Means
• DAW
• Clara
• Fuzzy Clustering
• Location Based Clustering
• Dimension Reduction
• Would be also used for preprocession
• column
• correlation
• Continous Variable
• Principal Component Analysis
• Factor Analysis
• Discreate Variable
• Multiple Correspondence Analysis
• Recommendation Systems
• Eg: Between users and products, ranking
• Models
• Popularity Based
• Market Based Analysis
• Association Analysis
• Content Based Recommendation Systems
• Collarative Recommendation Systems
• Model Based Recommendation Systems
• Hybrid Recommendation Systems

Model Evaluation

Classification

• Confusion Matrix
• An easy and popular method of diagnosing model performance.
• TN : True Negative
• TP : True Positive
• FN : False Negative - Type II 
• FP : False Positive - Type I
• Eg:

Fig: Confusion Matrix

• Always Type I and Type II models are inversely proportions, intuitively if Type I increases Type II error decreases
• Accuracy
• Percentage of correct predictions
• But in real life scenarios, we may be more keen in looking for precision and recall
• Because eg: we have a scenario where in when need to find if the there is fraud transaction, as out of all the transactions we would have only 1% of fraud transactions, we may have good results in accuracy, but our intention would be to find less False Negatives. 
• Accuracy of model: TP+TN / (TP+TN+FP+FN)
• Misclassification
• Percentage of incorrect predictions
• Formala : 
• 1 – Accuracy
• or 
• FP+FN / (TP+TN+FP+FN)
• Precision
• The precision is the ratio TP / (TP + FP) where tp is the number of true positives and FP the number of false positives. 
• The precision is intuitively the ability, indentify the Type I error, find score based on how False Positive value 0-1, can be converted to percentages.
• Eg:
• 1)
• TP = 1
• FP = 3
• Precision:
• 1 / (1+3)
• 1/4
• .25
• So here only .25 score, as there are more FP
• 2)
• TP = 6
• FP = 2
• Precission :
• 6 / (6+2)
• .75
• So here score is .75, as we have less FP
• Recall/Sensitive
• The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. 
• The recall is intuitively the ability, identify the Type II error, find score based on how False Negative value 0-1, can be converted to percentages.
• Specificity
• The specificity is the ratio tn / (tn + fn) where tn is the number of true negatives and fn the number of false negatives. 
• The recall is intuitively the ability, negative correct prediction, could also consider this as Type II error.
• F1 Score
• The harmonic mean of the precision and recall.
• F1 score reaches its best value at 1 and worst score at 0.
• Formula : (2 x Precision x Recall) / (Precision + Recall)
• ROC
• ROC stands for Receiver Operating Characteristics Curve
• Used to find performance matrix of classification model, under various Threshold parameters
• This curve is plot using two parameters:
• True Positive Rate
• TPR = TP / (TP + FN)
• Intuitively, Percentage of identifying actual true values
• False Positive Rate
• FPR = FP / (FP + TN)
• Intuitively, Percentage of identifying actual false values
• This plot is drawn at different thresholds 
• Visualization:

Fig: ROC

• AUC
• AUC stands for Area Under ROC Curve
• Also written as AUROC (Area Under the Receiver Operating Characteristics)
• AUC ranges in value from 0 to 1. A model whose predictions are 100% wrong has an AUC of 0.0; one whose predictions are 100% correct has an AUC of 1.0.
• If the value is 1, it means our model is overfitted and if the value is 0 it means it underfitted
• But Higher the AUC value, better the performance

• How AUC and ROC are used together
• Case 1
• When AUC is 1, it means the TP and TN never overlap, overfitting
• No wrong predictions

• Case 2
• When AUC is .7, it means the TP and TN have little overlap
• Probability of .3 wrong predictions

• Case 3
• When AUC is .5, it means the TP and TN have complete overlap
• Probability of .5 wrong predictions

• Case 4
• When AUC is 0, it means the TP and TN have opposite overlap
• Probability of all wrong predictions

• Observations:
• Maximum of AUC value would be our best threshold

Regression

• RSquare
• Goodness of the best fit model
• More the R^2 value, better the model
• Formula : R^2 
• = 1 – (SSRes/SSTotal)
• = 1 – ( Ʃ (yi- ŷi) ^2 / Ʃ (yi - ̅yi) ^2 ) 
• SSRes
• Sum of Residuals : Ʃ (yi- ŷi) ^2
• Residuals means errors, predictions minus actual Y value
• The Sum of Blue Lines are SSRes
• The reason we are squaring is to absolute the negative values
• SSTotal
• Sum of Total means Sum of Average Total : Ʃ (yi -  ̅yi) ^2
• Mean Line is in Aqua Blue
• Here instead of slope (best fit) we are calculating the Mean line
• And finding differences from each actual lines to Y Mean

Credits: https://www.datasciencecentral.com/wp-content/uploads/2021/10/2742052271.jpg

• Eg: Average Model
• 1- 2/4
• 1- 1/2 
• 1- .5
• 1-0.5
• 0.5
• Eg: Good Model
• 1-1/4
• 1-.25
• .75
• Eg: Bad Model
• 1-4/4
• 1-1
• 0
• Eg: Very Bad Model
• 1-8/4
• 1-2
• -1
• Adjusted R Square
• When we have new number of features added, we have an increase in R2 value
• So AdjRSquare value is used when there are comparisions between 2 or more reg models with different independent variables
• It helps us to find the new added independent variable is helpful or not in increasing the R2 square value
• Formula :
• 1 - ( (1-R^2) * (N-1) / (N-P-1) )
• N: no of rows or sample size
• P: no of predictors or independent features
• If the independent variables are correlated to the Target variable, we will have small decrease in the AdjR2.
• Else we will have higher decrease in AdjR2
• Points to remember :
• Every time we add a independent variable to a model the RSquare always increase
• Even if there is no significant correlation with Target variable, it will never decline
• Whereas Adjusted R Square increases only when independent variable is significant and affects dependent variable
• Adjusted R Square value would always be less than or equal to R Square value
• Reference : https://www.youtube.com/watch?v=WuuyD3Yr-js

Assumptions

Parametric Test

• Parametric analysis, are done when the population parameter is known or assumed as known
• Performs well even when the data is skewed or non normal
• Mean is used for the measure of tendency
• Interval and ratio are the measurement levels
• Pros :
• Proven to be highly powerful
• Cons :
• Expects population to be known
• Complicated theory

Non Parametric Test

• NonParametric analysis, are done when the population distribution is unknown.
• Median is used for the measure of tendency
• Ordinal and nominal are the measurement levels
• Pros :
• Simple and easy to understand
• No assumptions made
• Cons :
• Applicable for nominal and ordinal scale
• Consider all data groups have same spread
• Refer : https://blog.minitab.com/blog/adventures-in-statistics-2/choosing-between-a-nonparametric-test-and-a-parametric-test

Balance

Overfitting

• Overfitting refers to a model that learns the training data too well.

• Reduces Overfitting :
• Use a resampling technique to estimate model accuracy
• Hold back a validation dataset

Underfitting

• Underfitting refers to a model that can neither learn the training data nor generalize to new data.

Model Evaluation Error

• Err (X) = Biaŝ 2 + Variance + Random Error

Random Error

• It means Irreducible Error
• Usually the amount of noise in our data, which is unknown and uncontrolled

Bias error

• Bias is actually the accuracy score from the predicted values minus actual values
• Generally its ROC_AUC in classification and RMSE in regression
• Model with high bias pays very little attention to the training data and oversimplifies the model.
• High bias is underfitting of the model
• High bias in training will also lead to high bias in testing so it would be underfitted model
• Bias are the simplifying assumptions made by a model to make the target function easier to learn.
• Low bias = Less assumptions
• High bias = More assumptions
• Low bias models :
• Decision Tree, KNN, SVM
• High bias models :
• Linear, Logistic
• Usually High Bias and Low Variance are in parametric ML algorithms, as they make assumptions of the population
• Bias is also introduced by us, such as when we select the features, if we have less features or irrelevant features added to model, we add bias
• When the data is not scaled, there in also we introduce bias, as higher value data are prone to make model bias towards its value

Variance errors

• Variance is Variability of model prediction, how good the model predicts in different samples of same dataset
• High variance is overfitting the model
• High variance will win in the training set, but if different set is introduced the model accuracy score will be affected
• Low variance, small change to estimate of the target function with the changes of the training dataset
• High variance, many changes to estimate of the target function with the changes of the training dataset
• Usually High Variance and Low Bias are in non parameteric ML algorithms, as they make no assumptions of the population and have lot of flexibility
• High variance models :
• Decision Tree, KNN, SVM
• Low variance models :
• Linear, Logistic

Bias Tradeoff

• If our model is too simple we may introduce High Bias, which is Underfitting
• If our model is too complicated we may introduce High Variance, which is Overfitting
• Finding the sweet spot is important here, where there is low variance and low bias
• In reality there is no escaping the relationship between bias and variance in machine learning
• Increasing the bias will decrease the variance
• Increasing the variance will decrease the bias
• So we need to find the optimal balance

Fig: Bias Tradeoff

Problem of Imbalance

• When there is imbalance of class information, models tend to low sensitivity
• In such calls,
• Over sampling of records to bring in the balance of might work
• Ensemble methods might work
• Changing the cut off or threshold of predicted probability might also work

Cross Validation

• Due to imbalance of data, we can try to fix this using cross validation(cv)
• Usually we have data split into Train and Test.
• But using CV we can have number of splits with the train data, as train and test
• As several samples are generated using the same train data set, we can with ease have high chances of detecting over fitting
• Used when :
• When the data is noisy, (high outliers)
• When the model is a greedy algorithm, because the model never takes optimal value
• eg: decision tree, as in step 1 it does not know the further steps
• Generally there are parameters for the model
• eg: knn, the k values are used for model building, best k value would be after analysis such as it could be 3,5,11,..?
• Types:
• K-Fold Cross Validation
• Take the train data, the data then can be split to K equal parts
• Then one of the K part is considered as Test and others as Train
• Drawback: computationally expensive
• Sample Source :
• from sklearn.model_selection import cross_val_score
• print(cross_val_score(model, X, y, cv=5))
• Visualization:

• Leave P-out Cross Validation
• Leave One-out Cross Validation
• Repeated Random Sub-sampling Method
• Holdout Method

References and Credit

https://www.wordstream.com/blog/ws/2017/07/28/machine-learning-applications

Several Medium, Wikipedia and other sources.

Role of Data in Statistics

 

Statistics

Statistics as per wikipedia, is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data.

Statistics Types

• Descriptive : What ? Eg: Mean, Mode, Etc
• Diagnostics : Why ? Is all called as Inferential. Eg: Hypothesis, Conjectures 
• Predictive : What will happen ? Eg: Regression Analysis
• Prescriptive : What should we do to make this happen ? Eg: Decision Making

Statistics Terms 

• Population
• The entire universe of the data being studied. Though technical very challenging to acquire all the data for study.
• Eg: Entire nation's people vote opinions
• Parameter:
• Characteristics of a population
• Greek letters are used to represent
• Eg: for Average μ
• Sample
• Generally part or subset of the population
• Statistics
• Characteristics of sample
• Here the english letters are used for eg: x̄ for sample mean, though sometimes can be used to greek letters having ^ over it eg: ^μ
• Eg: Avg people of each state go for voting from the sample data
• Variable
• Question that is being research (ideally sample)
• Eg: Avg people from Assam go for voting
• Data
• Actual value the variable takes
• Collection of observations
• Helps us to find the relationships between two or more events
• Help us predict future
• Inputs are called as features
• Outputs are called as labels or response
• Let see with an example for the story so far.
• Conducted survey over 10000 citizens to know their voting options
• Population: Entire nation's citizens
• Sample: 10000 citizens
• Statistics : Avg of citizens going to vote from the 10000 citizens
• Variable : Will vote or not
• Data: Actually they have voted or not

Data Types

• Independent 
• Usually we can have this in X axis
• Dependent or also called as Target Data/Label Data/Class Variable
• Usually we can have this in Y axis

• Quantitative
• Numeric :
• Discrete:
• Which is countable
• Output or labels, are defined
• Eg: number of children
• Continuous:
• Keeps data getting finer and finer, infinite
• Which is measurable
• Output or labels, are not defined
• Eg: price of home, stock price, age, time, height, weight, salary
• Types :
• Ratio
• Proportions can be taken by comparing
• Zero is absolute
• Eg: Salary as 0, means no salary
• Note: so we don't need to worry on data transformation
• values have scale and a true zero point
• eg: level of measurement describes the time needed to complete a project
• Interval
• Proportions cannot be taken
• Zero is not absolute
• Eg: Temperature : 0 Degrees, where it means here freezing point, but there are values below this also, we can transform this to Kelvin
• Eg: Time, 00:00 means, not no time, its 12:00am
• Eg: Longitude and Latitude, Equator
• Note: Data can be transform here
• Categorical :
• Eg: types of car, colour of cage, gender
• Types:
• Binary
• Symmetric : equally important, example Female or Male in interview equal importance irrespective of imbalance of data.
• Asymmetric : Higher weightage to either of the value, 1 or 0, not equally. Important. Eg: Positive or Negative for disease test, so we can give more weightage Positive results
• Symmetric: equal importance
• Female
• Male
• Asymmetric: significance given
• Positive
• Negative
• One hot encoding

• Eg: name, gender
• Example :

• Nominal
• Predetermined categories
• Cannot if sorted / put data in particular order
• Eg:
• Red
• Blue
• Green
• The data further could also be represented as below:

• Ordinal
• Can be sorted / put data in particular order but space between the data do not matter
• Eg: Top 3 runner in a marathon
• 1 - 15hrs
• 2 - 17hrs
• 3 - 25hrs
• Here the different from 15 - 17 which is 2hrs and 17 - 25 which is 8hrs this is known as space and it doesn't impacts
• Lacks scale
• Label encoding (for conversion)
• Eg:
• Good
• Better
• Best
• Eg: star rating, height

• Data
• Can put data in order and space between data is important
• Zero does not have any meaning
• Eg: speed of bike
• 20km/hr
• 70km/hr
• 120km/hr
• The difference between 20 - 70 which is 50 and 70 - 120 which is 50 again has importance
• Proportion data
• Can put data in order and space between data is important
• Zero does have any meaning
• Eg: No of bikes each student has
• 5 or 0 each has value

• Qualitative
• Text
• Images
• Audio
• Video

Data Summary

• Frequency Table
• No of times a value occurs in data
• Types
• Relative Frequency
• Cumulative Relative Frequency
• Probability
• Chances of a value occurring in an experiment
• Example: Students marks

Data	Frequency	Relative Frequency (rf)	Cumulative Relative Frequency (crf)
90	300	60	60
95	100	20	80
100	100	20	100

• Sample Summary from the data
• % of students who obtained >= 90 marks in exam
• 20% + 20% = 40%
• % of students who scored 90marks or less 
• 60%
• % of students who scored more than 90% and less than 100%
• 20%
• Average Marks
• Avg = ∑x * frequency / ∑frequency (total number)
• Eg:
• 90*300 = 27000
• 95*100 = 9500
• 100*100 = 10000
• ∑x * frequency =  46500
• ∑frequency = 500
• avg = 46500/500 = 93

Data Visualization

• Easy to see the data in pictorial way
• It is very important

Credits: https://www.researchgate.net/publication/349139148/figure/fig6/AS:1005489378840612@1616738757855/Heat-map-depicting-the-distribution-of-the-population-density-in-the-250-m-250-m.png

Credits: https://neilpatel.com/wp-content/uploads/2021/03/Data-Visualization_Featured-Image-1.png

Credits: Anonymous

Common Symbols and Syntax

• x power of 5 : x*x*x*x*x; eg : 2 power 3 = 2*2*2 = 8
• x^5: same as x power of 5
• x power of -2 : 1 / (x*x); eg: 2 power -3 = 1/(2*2*2) = 1/8 = 0.125
• x power of (1/n) : 1/2 is actual square root : 
• 8 power of (1/3) = 2
• cube root of 8 = 2
• x! : x * (x-1) * (x-2) * ... 1 : eg: 3! = 3*2*1 = 6
• Ʃ power n from x=1 of x : x= 1+2+3+..+n
• Ʃ power n from x=1 of ( x sub i ) : ( x sub i ) = x1 + x2 + ..+xn
• eg: x = {5,2,1}, n=3, 
• Ʃ power 3 from x=1 = 5*1+2*1+1*1 = 8

Measures of Centre

• Arithmetic Mean : 
• The arithmetic mean, often simply called the average, is a measure used to find the central tendency of a set of numbers. It's calculated by adding up all the numbers in a dataset and then dividing the sum by the total count of numbers.
• ẋ: ( Ʃ power n from x=1 of ( x sub i ) ) / n
• Eg: x = {5,2,1}, n=3, 
• ( Ʃ power 3 from x=1 of ( x sub i ) ) / 3 = (5+2+1)/3 = 8/3 
• Mean can be influenced by the outliers
• Below are the ways to handle outliers
• Do nothing
• Trim outlier
• Wensorized
• winsorized mean is an averaging method that involves replacing the smallest and largest values of a data set with the observations closest to them, it mitigates the effects of outliers by replacing them with less extreme values - here it either provides for floor or ceiling values
• Formula:
• µ: mean of population
• ẋ: mean of sample
• Geometric Mean
• The geometric mean is another measure of central tendency, but it's specifically used when dealing with growth rates or ratios, such as investment returns over multiple periods. It's calculated by taking the nth root of the product of n numbers, where n is the total count of values.
• Arithmetic mean does not account for the compounding effect of returns over time, especially in the presence of negative returns. The geometric mean, on the other hand, accounts for this compounding effect and provides a more accurate measure of the average growth rate.
• Geometric Mean = (x₁ * x₂ * x₃ * ... * xₙ)^(1/n)
• If any one number is with -ve the entire data will be -ve, from the formula its power of ^ 1/n, which will error if we do under root of -ve number.
• Hence all these numbers has to be more than 0 i.e. >0.
• In that case +1 is hence added to all the values
• And finally before root, do -1 to the overall values.
• Eg:
• We have invested 100 dollars after 1st year it returned 200 dollars and 2nd year it returned 100 dollars
• Here in the 1st year the return is 100% and second year the returns are -50%.
• Lets try to find average return:
• First lets try with Arithmetic Mean = 1 + (-0.5%) / 2 = 0.25 = 25%
• Note: if you see we haven't got 25% return, but there the AM seems to have this stating it wrongly.
• Now using Geometric Mean (here percentage is converted)
• (((1+1) * (1-0.5)) - 1) ^ 1/2
• ((2 * 0.5) - 1) ^ 1/2
• (1 - 1) ^ 1/2
• 0
• Here if you see the actual returns are 0 and the GM average stats its correctly.
• Weighted Mean
• Arithmetic mean is a special weighted mean as here all weights are equal.
• For example:
• (1+2+3+4+5) / 5 = 3
• (1*1/5)+(2*1/5)+(3*1/5)+(4*1/5)+(5*1/5) = 0.2+0.4+0.6+0.8+1 = 3
• In weighted mean the 1/n (eg: 1/5) is different for each value, ideally used in portfolio management.
• Let's say you have three exams with different weights:  
• Input
• Exam 1: 20% weight, score of 80 
• Exam 2: 30% weight, score of 75 
• Exam 3: 50% weight, score of 90 
• To calculate the weighted mean, you first multiply each score by its respective weight, then sum up these products, and finally divide by the sum of the weights.  
• Multiply each score by its weight: 
• Exam 1: 0.20 * 80 = 16 
• Exam 2: 0.30 * 75 = 22.5 
• Exam 3: 0.50 * 90 = 45 
• Sum up these products: 16 + 22.5 + 45 = 83.5  
• Divide by the sum of the weights: 
• Total Weight = 0.20 + 0.30 + 0.50 = 1  
• Weighted Mean = 83.5 / 1 = 83.5  
• So, the weighted mean score is 83.5. This means that when taking into account the weights assigned to each exam, the overall average score is 83.5.
• Harmonic Mean
• The harmonic mean is a type of average used to determine the central tendency of a set of numbers. 
• It's calculated by dividing the number of values in the dataset by the sum of the reciprocals of the values. 
• It is the inverse of the arithmetic mean of the reciprocals.
• If you have  n values  x1, x2 , . . . , xn, 
• harmonic mean (H) is:  n / ∑xi
• One common application of the harmonic mean is in situations where rates are involved, such as calculating average speeds or average rates of return.
• It's particularly useful when dealing with ratios or rates, where outliers can heavily skew the arithmetic or geometric means.
• Example: 1, 2, 3, 4, 5, 25
• 6 / (1/1 + 1/2 + 1/3 + 1/4 ​ + 1/5 ​ + 1/25)
• 6 / 2.32
• 2.58
• Compilation of all the means:
• AM: 1+2+3+4+5+25 = 6.67
• HM: 2.58
• GM: 3.80
• AM * HM = 6.67 * 2.58 = 17.2086
• GM^2 = 14.44
• Another analogy, Arithmetic Mean * Harmonic Mean = Geometric Mean ^ 2
• Median :
• Middle value of the ordered data set
• ** must be sorted
• Is much closer to the values in the series
• Median takes in account relative position of data
• Median : (n+1)/2 : row number
• Mode :
• Most frequently occurring data value
• Sample modes
• unimode -> only one most occurred value
• bimode -> two different values but occurred with same frequency
• trimode -> three different values but occurred with same frequency
• Only mode is used to represent nominal series

Measures of Spread/Dispersal

• The spread of data.
• If the mean is the same for two sets of data, but each has a different spread of values. As mean is expected returns, the risk is how far the value from the mean or deviated from the mean.
• Variability of data is risk.
• Range:
• Diff of the highest value and lowest value of data : max-min
• The problem with range is only taking into account two extreme values into its calculation. But we want to consider all the data points to understand variability of data.
• MAD - Mean Absolute Deviation
• Calculate mean and evaluate the difference of each data points against mean.
• x - x̄
• As the sum of all the deviation will be always be zero, take the abs (x - x̄) there the sum of all and divide by n.
• MAD = ∑( | x - x̄ | ) / n
• This says on a average the spread from mean. 
• Here the problem is penalization of data away from mean is not visible.
• Variance:
• Sum of square distances from each point of the mean
• The reason its squared, because of two reasons
• The sum values would less result to 0
• The more away from the mean the square will penalize and make it further away. The higher the difference the higher the square would take away from mean.
• Formula
• population (σ²)

• sample (s power 2)

• Bessel's Corrections
• Why n-1? Loosing the degree of freedom
• Law of large numbers, as the sample size increase the sample mean approaches population mean.
• Population mean can take any value, as any value added will be change the population mean based on the data.
• Sample mean is destined to be population mean, it cannot be anything it want to be. It can take any values until n-1, but not n as it needs to be adjusted to the population mena. Hence it lose the freedom by 1.
• Standard Deviation:
• Square root of the variance
• Same units of the sample
• On the average, how far are my values from the mean, how reliable are my central values
• Formula
• population : σ= square root of (σ²) 
• sample : s = square root of (s power 2)
• Now, why we would have n-1
• When having n :
• efficient estimate
• consistent estimate
• When having n-1
• unbiased estimate
• Zscore
• Zscore, are the ways comparing values across different sets of data where the means and standard deviations are different.
• Formula : z = ( x – µ ) / ס
• µ – mean
• ס – standard deviation

• Downside Deviation:
• Standard deviation is the measure of volatility, and spread of data around the centre.
• Here the downside deviation is used ideally by risk managers where the moving downside movement is risk for the investor.
• Share Price is 100, if this goes to 120 it is not of concern but if it goes to 80 it becomes risk.
• Formula:
• DD = square root (∑((x - D) ^ 2) / n - 1)
• Eg:
• If the minimum returns needed is 3% then instead of x̄ it is D and from the example the value of D is 3.
• Input
• January: +2% 
• February: +3% 
• March: -1% 
• April: -2% 
• May: +1% 
• June: +4% 
• July: -3% 
• August: -2% 
• September: +1% 
• October: +2%
• November: +6%
• December: +1%
• Calculation
• Find mean, (2+3-1-2+1+4-3-2+1+2+6+1) / 12 = 1
• Lets take the months which are greater than equal to 3% and find the difference from the mean and square it.
• February: +3% = (3-1)^2 = 4
• June: +4% = (4-1)^2 = 9
• November: +6% = 25
• Sum of squared deviations
• 4 + 9 + 25 = 38
• Downside Deviation
• Square root ( 38 / 12 )
• 1.78%

Measures of Location

• Locates the data in the series.
• Percentile
• Divides the data into 100 equal parts
• Decilies 
• Divides the data into 10 equal parts
• Quantiles
• Divides the data into 5 equal parts 
• Quartile
• Divides the data into 4 equal parts 
• Here the values that split the data into equal chunks are known as quartiles, as they split the data into quarters. 
• Describe data, through quartiles and inter quartile range (IQR)
• Every data point is considered, as its not aggregated (median is used)
• How its calculated
• Divide series into half (median)
• 0 21 21 31 41 51 | 61 71 81 91 91 200
• Divide each sub series into further half (median)
• 0 21 21 | 31 41 51 | 61 71 81 | 91 91 200
• And now this becomes quartiles
• 1^st quartile : 26
• 2^nd quartile : 56
• 3^rd quartile : 86
• Quartile ranges would be never the same size in the real world
• IQR – tells about middle range, from 25 percentile to 75 percentile
• Outliers
• Common practice is set to a “fence”, that is 1.5 times width of the IQR
• Anything outside the fence is considered as outlliers
• This is calculated as percentiles
• Formulas
• IQR = Q3 - Q1
• Low outliers: anything less than Q 1 − 1.5(IQR)
• High outliers: anything greater than Q 3 + 1.5(IQR)
• Example :

Bivariate Data

• Compare two variables
• As said earlier,
• x axis : independent variable
• y axis : dependent variable
• Scatter plot, can be used to find the correlation between two variable
• But does’t prove causality – while is cause and affect
• As called to be spurious correlation
• https://tylervigen.com/spurious-correlations
• We may need more statistical analysis is determine causality
• Reasons of spurious correlation
• By chance
• Linked to common variable to another variable, here when one variable and another variable are related to the third variable
• When relationship emerges by dividing by some common variable
• Covariance:
• Compare two variable, to compare their variances
• How far the each variable’s mean fall
• If the values are more away from zero it means there are very well correlated
• Very difficult to interpret covariance, though it shows either positive or negative co-relationship.
• Formula :

Credits: https://www.wallstreetmojo.com/

• Correlation:
• Correlation shows the linear relationship and can be explained by line for linear data.
• Formula:
• cor (x, y) = cov(x, y) / sx sy
• cov(x, y) = covariance of x and y
• sx = standard deviation of x
• sy = standard deviation of y
• Relation between two variables
• Status of relationship
• Direction of relationship
• Magnitude of relationship
• Positive Correlation
• increase x will increase y
• Negative/Inverse Correlation
• increase x will decrease y
• Near Zero
• No linear relationship

• Pearson Correlation Coefficient:
• Compare two variables, with normalization, as not like covariance as its values are taken from variance which is squared
• Values would now fall between +1 and -1, where
• 1 = total positive linear correlation
• 0 = no linear correlation
• -1 = total negative linear correlation
• Formula :

Credits: https://www.wallstreetmojo.com/

Credits and References

https://www.bestproxyreviews.com/wp-content/uploads/2020/09/Statistical-Analysis-Methods.jpg