Thursday, 14 September 2023

Hypothesis Testing - An Introduction

 


Confidence Interval

We can use mean, median, mode, variance or proportion as point estimators to estimate the precise value of the population. But how confident are we to estimate this is completely accurate, point estimators are valuable, but they would give slight errors, as it does not deal with the complete population. If the sample is unbiased then the estimate is likely to be close to the true value of the population, but how close would be close enough. 

To solve this we have another way of estimating the population statistics, one that allows for uncertainty - that is confidence intervals. Rather than giving a precise value as an estimate for the population mean, we can specify some interval as an estimation.


Steps to find confidence intervals:
  • Choose the population statistic
    • The first step is to pick the population statistic we would want to construct a confidence interval for.
      • Eg: population mean
      • For the population statistics mean the expectation and variance from the sample is required.
  • Find its sampling distribution
    • To find the confidence limits we need to know the sampling distribution. 
    • As the normal or any particular distribution isn't a good approximation for every situation, hence we need to find the sampling distribution based on that the next steps will be taken care.
    • Normal Distribution: When the sample sizes are large, the normal distribution is ideal for finding the confidence intervals. It gives accurate results, irrespective of how the population itself is distributed.
    • t- Distribution: When the sample size is small, the population mean is unknown t-distribution is suitable for such situations. Here the curve is flatter, and have slightly fatter tails. It takes one parameter v, where v is equal to n-1, n is the size of the sample and v is called the number of degrees of freedom. This gives a wider confidence interval than the normal distribution, which makes it more appropriate for small sized samples.
  • Decide on the level of confidence
    • The level of confidence says how sure we would want to be that the confidence interval contains the population statistics. This means the confidence level is the probability of the population statistic (eg: mean) being inside the confidence interval. For a confidence level of 95% the probability is 0.95. 
    • To select the level of confidence ideally depends on the situation and how confident we need to be that the interval contains the population statistic. A 95% is most common, but some times it could be 90%, 99%.
    • The key thing to remember is that the higher the confidence level is the wider the interval becomes and the more chance there is of the confidence interval containing the population statistics. But the trouble with making the confidence interval too wide is that it can lose meaning. Ideally we need to make the interval as narrow as possible, but wide enough so we can be reasonably sure the true mean is in the interval. 
  • Find the confidence limits
    • The final step is to find 'a' and 'b', the limits of the confidence interval, which indicate the left and right borders of the range in which there's aa 95% probability falling. 
    • The exact value of 'a' and 'b' depends on the sampling distribution we need to use and the level of confidence that we need to have.
    • We can find the confidence interval for μ by rewriting the inequality in terms of μ.
      • a < μ < b
    • For normal distribution we calculate a standard score and use the standard normal probability table to help with the result we need that is 'a' and 'b'.

Common formulas for confidence intervals:

  • Confidence Interval for Population Mean (Normal Distribution, Large Sample Size): 
    • Formula
    • Where,
      •  is the sample mean.
      • Z is the critical value from the standard normal distribution corresponding to the desired confidence level. For example, for a 95% confidence level, Z=1.96.
      • σ is the population standard deviation.
      • n is the sample size.
  • Confidence Interval for Population Mean (t-Distribution, Small Sample Size): 
    • Formula
    • Where,
      •  is the sample mean.
      • t is the critical value from the t-distribution with n−1n−1degrees of freedom corresponding to the desired confidence level.
      • s is the sample standard deviation.
      • n is the sample size.
  • Confidence Interval for Population Proportion
    • Formula
    • Where,
      •   is the sample proportion.
      • is the critical value from the standard normal distribution corresponding to the desired confidence level.
      • n is the sample size.


Example: Confidence Interval for Population Mean of Normal Distribution

  • Suppose you have a sample of 50 students from a university, and you want to estimate the average number of hours they spend studying per week. From your sample, you find that the mean number of hours spent studying is 20 hours, with a standard deviation of 4 hours.
  • You want to construct a 95% confidence interval for the population mean number of hours spent studying per week.
  • Given:
    • Sample mean (xˉxˉ): 20 hours
    • Sample standard deviation (σ): 4hours
    • Sample size (n): 50 students
    • Confidence level: 95%
  • Now, we'll use the formula for the confidence interval for the population mean. 
    • Confidence Interval=xˉ±Z(nσ)
  • Since we're constructing a 95% confidence interval, we need to find the critical value Zfrom the standard normal distribution. For a 95% confidence level, Zis approximately 1.96.
  • Now, calculate the standard error:
    • Standard Error=5040.5657
  • Finally, calculate the confidence interval:
    • Confidence Interval=20±1.96×0.5657
    • Confidence Interval=20±1.1089
  • So, the 95% confidence interval for the population mean number of hours spent studying per week is approximately (18.8911,21.1089)hours.
  • This means we are 95% confident that the true population mean number of hours spent studying per week falls within this interval.

Example: Confidence Interval for Population Mean using t-Distribution

  • Suppose we have a sample of 20 students, and we want to estimate the average score they achieved on a standardized test. From our sample, we find that the mean score is 85, with a standard deviation of 10.
  • We want to construct a 95% confidence interval for the population mean score.
  • Given:
    • Sample mean (xˉ): 85
    • Sample standard deviation (s): 10
    • Sample size (n): 20 students
    • Confidence level: 95%
  • Use the formula for the confidence interval for the population mean with a t-distribution:
    • Confidence Interval=xˉ±t(ns)
  • Since we're constructing a 95% confidence interval with 20 samples, we need to find the critical value t from the t-distribution table. For a 95% confidence level and  n − 1 = 19 n−1=19 degrees of freedom,  t ≈ 2.093.
  • Now, calculate the standard error:
    • Standard Error=20102.236
  • Finally, calculate the confidence interval:
    • Confidence Interval=85±2.093×2.236
    • Confidence Interval=85±4.676
  • So, the 95% confidence interval for the population mean score is approximately (80.324, 89.676).
  • This means we are 95% confident that the true population mean score falls within this interval.
    (80.324,89.676)

Hypothesis Basics

A hypothesis is an assumption statement about a problem, idea, opinion, claim, belief or some characteristics of a population. By using estimates and confidence intervals, we can evaluate statistical claims. Hypothesis tests allow us to determine their likelihood of being true from the sample statistic representing the population. It is a supposition or proposed explanation based on limited evidence as a starting point for further investigation. Even though the sample size is small, we can still perform hypothesis tests.

Steps for Hypothesis Testing

  • Decide on the hypothesis going to be test
    • This is the claim that we're putting on trial test, and this claim is called a hypothesis.
    • The null hypothesis is the claim that will be testing, and the alternate hypothesis is what we will accept if there is sufficient evidence against the null hypothesis.
  • Choose the appropriate test statistic
    • After we know what we are going to test, we need some means of testing to test the hypothesis. 
    • The test statistic is the statistic we need to pick that best tests/calculates based on a sample, whose value is the basis for deciding the claim/hypothesis.
    • Test Statistics = Sample statistic - Hypothesized value / Standard error of the sample statistic
    • A test statistics is a random variable that changes from one sample to another.
  • Determine the critical region
    • To test whether there is sufficient evidence against the null hypothesis, we need certain level of certainty to look for evidence that contradicts Ho.
    • The critical region of the hypothesis test is the set of values that present the most extreme evidence against the null hypothesis.
    • If the number falls with the critical region, then there is sufficient evidence to reject the null hypothesis. If the number falls outside of the critical region, there we need to accept there is not sufficient evidence to reject the null hypothesis. This cut off point for the critical region is called as critical value.
    • To find the critical region, first need to decide on the significance level. The significance level is represented by α. It's a way of saying how unlikely the results to be before reject null hypothesis.
    • Just like confidence level, the significance level is given as percentage.
    • For constructing a critical region for test, another test is needed to be conducted a one-tailed or two tailed test. This helps in finding the where the critical region falls in the distribution.
  • Find the p-value of the test statistic
    • We need to see how rare our results are, assuming the claims are true.
    • The p-value is the probability of getting the results in the sample or something more extreme and including the one in sample in the direction of the critical region. 
    • This is the lowest level of significance at which we can reject the null hypothesis.
    • It's a way of taking sample and working out whether the result falls within the critical region for the hypothesis test.
    • In simple words use of p-value to say whether or not we can reject the null hypothesis.
    • The strength of the evidence against Ho increases as the p-value becomes smaller.
      • For left one-tailed test, the p-value is given by the probability that lies below the calculated test statistic.
      • For right one-tailed test, the p-value is given by the probability that lies above the calculated test statistic.
      • For two-tailed tests, the p-value is given by the sum of the probabilities in the two tails.
  • See whether the sample result is within the critical region
    • We then see if it's within our bounds of certainty. 
    • If the p-value is less than the level of significance we reject the null hypothesis.
    • The smaller the p-value, the stronger the evidence against the null hypothesis and in favor of the alternative hypothsis.
  • Make a decision regarding the hypothesis
    • Depending on the evidence accept or reject the claims of the drug company.

Null Hypothesis and Alternative Hypothesis

  • Null Hypothesis
    • The claim of testing, that represents the current state of knowledge about the population parameter that is the subject of the test is called the null hypothesis.
    • In general it is a simple statement about a population parameter that is actually tested.
    • It is represented by Ho
    • It is the claim that we will accept unless there is strong evidence against it.
    • Examples: μ = μ0, μ <= μ0, μ <= μ0 where μ is the population mean and μ0 is the hypothesis value of the population mean.
    • Note: Null hypothesis always include equal condition.  
  • Alternative Hypothesis
    • The counterclaim to the null hypothesis, where there is sufficient evidence to reject the null hypothesis is called the alternate hypothesis.
    • It is usually the alternative hypothesis the researcher is really trying to assess.
    • It is represented by Ha or H1.
    • It is the claim that we will accept if there is strong enough evidence to reject Ho.
  • When hypothesis testing, assume the null hypothesis is true. If there's sufficient evidence against it, reject null and internally accept the alternate hypothesis.
  • Note: In statistics anything can never really prove, when the null hypothesis is discredited, the implication is that the alternative hypothesis is valid. We either reject the null hypothesis or fail to reject the null hypothesis as statistically it is incorrect to say "accept" the null hypothesis. 

One Tailed or Two Tailed Hypothesis

Credits: https://arshren.medium.com/hypothesis-testing-an-intuitive-explanation-898d547db38d
  • One Tailed Tests
    • A one-tailed test, is where the critical region falls at one end or direction of the possible set of values in the test.
    • The aim is to test the possibility of a change in one direction and completely disregard the possibility of change in the other direction.
    • The tail can be at either end of the set of possible values and the end to use depends on the alternate hypothesis H1.
    • If alternate hypothesis includes a < sign, then use the lower tail or left tail, where the critical region is at the lower end of the data.
    • If alternate hypothesis includes a > sign, then use the upper tail or right tail, where the critical region is at the upper end of the data.
  • Two Tailed Tests
    • A two tailed test considers the possibility of a change it either direction and it is where the critical region is split over both ends of the set of values. 
    • Choose the level of the test α, then make sure that the overall critical region reflects this as a corresponding probability by splitting it into two.
    • Both ends contain α/2, so that the total is α.
    • If H1 contains a ≠ sign, or Ho contains a = sign, then we need to use a two tailed test. 
Credits: https://www.youtube.com/watch?v=087s7jAa6EA

Test Statistic

  • Hypothesis testing involves two statistics
    • the test statistics calculated from the sample data
    • the critical value of the test statistics
  • The value of the computed test statistic relative to the critical value is a key step in assessing the validity of a hypothesis.
  • A test statistic is calculated by comparing the point estimate of the population parameter with the hypothesized value of the parameter which is the value specified in the null hypothesis.
  • Here we concerned with the difference between the sample statistic and the hypothesized value, scaled by the standard error of the sample statistic.
    • test statistic = sample statistic - hypothesized value / standard error of the sample statistic
  • The standard error of the sample statistic is the adjusted standard deviation of the sample. 
  • When the sample statistic is the sample mean: x̄, the standard error: σx̄ of the sample statistic for the sample size n, is calculated as :
    • when the population standard deviation σ is known σx̄ = σ / root(n)
    • when the population standard deviation σ is not known, it is estimated using the standard deviation of the sample s, sx̄ = s / root(n)
  • Layman Note: In general mathematics 5 != 4 but in statistics we can say 5 = 4.
  • Test Statistic Formulas:

Credits: https://www.dummies.com/wp-content/uploads/250714.image0.jpg

Type I and Type II Errors

When we conduct a hypothesis test, we can only make a decision based on the evidence from the sample data. So if the sample is biased,  there may be wrong decision made. These errors are named as below:
  • Type I error 
    • When wrongly reject a true null hypothesis, in simple terms penalizing the innocent.
    • The significance level is the probability of making a Type I error that is rejecting the null when it is true and is designated by the Greek letter alpha α.
    • For instance, a significance level of 5% (α = 0.05) means there is a 5% of chance rejecting a true null hypothesis. 
    • When conducting hypothesis tests, a significance level must be specified to identify the critical values needed to evaluate the test statistic.
    • Multiple testing involves conducting multiple different hypothesis tests on the same dataset. The issue with multiple testing is that the alpha value, which is the threshold for statistical significance, is valid only for a single hypothesis test. As we carry out more and more tests, the alpha value for the repeated testing increases, which in turn raises the probability of making a Type I error.

  • Type II error
    • When wrongly accept a false null hypothesis, in simple terms favour the guilty.
    • Power of a test gives the probability of correcting discrediting and rejecting the null  hypothesis when it is false.
      • Power of a test = 1 - P(Type II error) = 1 - β
    • When more than one test statistic may be used, the power of the test for the competing test statistic may be useful in deciding which test statistic to use. As one would wish to use the test statistic that provides the most powerful test among all possible tests.
Credit: https://sixsigmadsi.com/wp-content/uploads/2021/06/Type-I-and-II-errors-1.jpg
  • The probability of a Type II error is determined by the sample size and the significance level chosen, which is also called the Type I error probability. However, the relationship between them is complex, and it's not easy to calculate the probability of a Type II error.
  • If we decrease the significance level, i.e. the probability of a Type I error, from 5% to 1%, for instance, we may increase the probability of failing to reject a false null (Type II error). Consequently, the power of the test may be reduced.
  • On the other hand, we can increase the power of the test for a given sample size. However, this comes at the cost of increasing the probability of rejecting a true null (Type I error).
  • To decrease the probability of a Type II error and increase the power of a test, we need to increase the sample size for a given significance level.

Hypothesis Testing Results

p-Value

  • The null hypothesis is the initial claim made about a population, while the alternative hypothesis states whether the population parameter differs from the value in the null hypothesis.
  • The p-value is a statistical measure that tests hypotheses against observed data. It calculates the probability of observed results assuming the null hypothesis is true. It is the smallest level of significance for which the null hypothesis can be rejected or can be said as more extreme than what was observed to reject null. 
  • It measures the likelihood that any observed difference between groups is due to chance.
  • A smaller p-value means stronger evidence in favor of the alternative hypothesis. The p-value serves as an alternative to rejection points to provide the smallest level of significance at which the null hypothesis would be rejected. Generally, a p-value of 0.05 or lower is considered statistically significant.
  • P-value is often used to promote credibility for studies or reports by government agencies. For example, the U.S. Census Bureau stipulates that any analysis with a p-value greater than 0.10 must be accompanied by a statement that the difference is not statistically different from zero. The Census Bureau also has standards in place stipulating which p-values are acceptable for various publications.
  • P-values are usually found using p-value tables or spreadsheets/statistical software. These calculations are based on the assumed or known probability distribution of the specific statistic tested. P-values are calculated from the deviation between the observed value and a chosen reference value, given the probability distribution of the statistic, with a greater difference between the two values corresponding to a lower p-value.
  • The calculation for a p-value varies based on the type of test performed. The three test types describe the location on the probability distribution curve: lower-tailed test, upper-tailed test, or two-tailed test. For one-tailed tests, the p-value is the probability that lies above the positive value that lies above the computed test statistic for upper tail tests or below the computed test statistic for lower tail tests. For two-tailed tests, the p-value is the probability that lies above the positive value of the computed test statistics plus the probability that lies below the negative value of the computed test statistic.
  • In practice, the significance level is stated in advance to determine how small the p-value must be to reject the null hypothesis. Because different researchers use different levels of significance when examining a question, a reader may sometimes have difficulty comparing results from two different tests. P-values provide a solution to this problem.
  • Even a low p-value is not necessarily proof of statistical significance since there is still a possibility that the observed data are the result of chance. Only repeated experiments or studies can confirm if a relationship is statistically significant.
  • In a nutshell, the greater the difference between two observed values, the less likely it is that the difference is due to simple random chance, and this is reflected by a lower p-value. The p-value approach to hypothesis testing uses the calculated probability to determine whether there is evidence to reject the null hypothesis. 
  • To avoid confusion and allow independent observers to interpret the statistical significance themselves, researchers could report the p-value of the hypothesis test. Suppose a study comparing returns from two particular assets was undertaken by different researchers who used the same data but different significance levels. In that case, the researchers might come to opposite conclusions regarding whether the assets differ. If one researcher used a confidence level of 90%, and the other required a confidence level of 95% to reject the null hypothesis, and if the p-value of the observed difference between the two returns was 0.08 (corresponding to a confidence level of 92%), then the first researcher would find that the two assets have a statistically significant difference, while the second would find no statistically significant difference between the returns.


t-Test

  • The sampling distribution of the mean when the population variance is unknown is t-distributed.
    • The sample is large (n>=30)
    • The sample is small (n<30), but the distribution of the population is normal or approximately normal
  • t-Test Formula
  • Where: 
    • x̄ is the sample mean.  
    • μ0  is the hypothesized population mean under the null hypothesis.  
    • s is the sample standard deviation.
    • n is the sample size.
  • Note: If the sample is small and the distribution is nonnormal, we have no reliable statistical test.

 

z-Test

  • The sampling distribution of the mean when the population variance is known, it is normally distributed or z-distributed.
  • z-Test Formula

  • Where: 
    • x̄ is the sample mean.  
    • μ0  is the hypothesized population mean under the null hypothesis.  
    • σ is the population standard deviation.
    • n is the sample size.
  • When the sample size is large and the population variance is unknown the z-statistics can also be used substituting the population standard deviation to sample standard deviation. This is acceptable only if the sample size is large, also the critical values for the t and z are almost identical. Although the t-statistic is the more conservative measure where the population variance is unknown.

Chi-square Test

  • A chi-square test is used to establish whether a hypothesized value of variance is equal to, less than or greater than the true population variance.
  • Unlike most distributions, the chi-square distribution is asymmetrical, and in has no negative values.

Testing the Equality of Means
  • Test the difference between two population means is zero, means of two population are equal to each other.
  • The steps to test the hypothesis that the means are equal would then follow the standard hypothesis testing procedure. The null hypothesis would be that the difference between the two is equal to zero, versus the alternative that it would be not equal to zero.
  • Two Sample t-Test
    • Formula
    • Where,
      • 1 = observed mean of 1st sample 
      • 2 = observed mean of 2nd sample
      • S1 = standard deviation of 1st sample 
      • S2 = standard deviation of 2nd sample 
      • n1 = sample size of 1st sample 
      • n2 = sample size of 2nd sample
  • Two Sample z-Test
    • Formula
    • Where,
      • 1 = observed mean of 1st sample 
      • 2 = observed mean of 2nd sample
      • σ1 = population standard deviation of 1st sample 
      • σ2 = population standard deviation of 2nd sample 
      • n1 = sample size of 1st sample 
      • n2 = sample size of 2nd sample

Example: Hypothesis Testing for a Population Mean

Suppose we work for a beverage company and we want to test whether the mean sugar content in a particular type of soft drink bottle is significantly different from the advertised value of 30 grams per bottle. We have a sample of 50 soft drink bottles, and we want to determine if there is enough evidence to conclude that the mean sugar content is different from 30 grams.

  • Step 1: Decide on the hypothesis going to be test
    • Null Hypothesis (Ho): The mean sugar content in the bottles is equal to 30 grams. H0:μ=30  
    • Alternative Hypothesis (Ha): The mean sugar content in the bottles is not equal to 30 grams. H1:μ≠30.
  • Step 2: Choose the test statistic: 
    • We have a sample of 50 soft drink bottles with their sugar content measured.  
    • Given:
      • Sample mean (xˉ): 29.5 grams
      • Sample standard deviation (s): 4 grams
      • Sample size (n): 50
    • For calculating the test statistic, use the t-test for a population mean since we're dealing with a small sample size (n=50). 
      •  (29.5 - 30) / (4/root(50))
      • t0.883
  • Step 3: Determine the Critical Region: 
    • Since we have a two-tailed test (we're testing if the mean is not equal to 30 grams), we need to find the critical values from the t-distribution for a 95% confidence level and n−1=49 degrees of freedom. 
    • Let's denote these critical values as tleft and tright.  
    • Choose the Significance Level (α), let's choose a significance level of 0.05, which is commonly used in many studies. 
    • This means we're willing to accept a 5% chance of making a Type I error (rejecting the null hypothesis when it's actually true).  
  • Step 4: Find the p-value of the test statistic
    • We calculated a test statistic t = − 0.883. 
    • To find the p-value, we would determine the probability of observing a test statistic as extreme as t=−0.883 or more extreme in either tail of the t-distribution.  
    • Since we have a two-tailed test (testing if the mean is not equal to 30 grams), we need to consider both tails of the t-distribution.  
    • If we have access to statistical software or tables, we can directly find the p-value corresponding to our test statistic. Alternatively, we can calculate the p-value by finding the area under the t-distribution curve beyond t=−0.883 in both tails.  
    • Given the test statistic t=−0.883 and the degrees of freedom ( df = 50 − 1 = 49 ), we'd find the probability associated with t=−0.883 in the t-distribution table or using statistical software.  Let's assume we use software and find that the p-value corresponding to t=−0.883 with df=49 is approximately 0.140 (this is a hypothetical value for illustration purposes).  
    • Interpreting this p-value: If the p-value is greater than the chosen significance level (0.05 in this case), we fail to reject the null hypothesis.  If the p-value is less than or equal to the significance level, we reject the null hypothesis.  
  • Step 5: See whether the sample result is within the critical region
    • Let's assume we calculate our test statistic t to be -0.883. 
    • We would compare this value to our critical values from the t-distribution. 
    • If t falls outside the range defined by tleft and tright, we would reject the null hypothesis. 
    • Since 0.140>0.05, we would fail to reject the null hypothesis. Thus, we do not have sufficient evidence to conclude that the mean sugar content in the bottles is different from 30 grams at the 5% significance level.
    • Otherwise, we would fail to reject the null hypothesis.  
    • Proceed assuming tleft=−2.01 and tright=2.01(approximate values for a 95% confidence level with 49 degrees of freedom).  
    • Since t=−0.883 falls within the range defined by tleftt and tright, we fail to reject the null hypothesis. 
  • Step 6: Make a Decision regarding the hypothesis: 
    • If the test statistic falls outside the critical region, we reject the null hypothesis. Otherwise, we fail to reject the null hypothesis. 
    • If we reject the null hypothesis, we conclude that there is enough evidence to suggest that the mean sugar content in the bottles is different from 30 grams. 
    • Otherwise, we fail to reject the null hypothesis, indicating that we don't have sufficient evidence to conclude that the mean sugar content is different from 30 grams.  
    • Thus, we do not have sufficient evidence to conclude that the mean sugar content in the bottles is different from 30 grams at the 5% significance level. 


Confidence Level vs Significance Level

Confidence level and significance level are both important concepts in statistics, particularly in hypothesis testing and constructing confidence intervals, but they have different interpretations and applications.

Confidence Level:

  • The confidence level is associated with confidence intervals.
  • It represents the proportion of intervals, calculated from repeated samples, that would contain the true population parameter.
  • For example, if you construct 95% confidence intervals for a population parameter (e.g., population mean), the interpretation is that if you were to take many samples and construct confidence intervals for each sample, approximately 95% of those intervals would contain the true population parameter.
  • Commonly used confidence levels include 90%, 95%, and 99%.
  • The expression for the confidence interval can be manipulated and restated as :
    • -critical value <= test statistic <= +critical value
  • This is the range within which we fail to reject the null for a two-tailed hypothesis test at a given level of significance. 

Significance Level:

  • The significance level (often denoted by α) is associated with hypothesis testing.
  • It represents the probability of rejecting the null hypothesis when it is actually true (i.e., making a Type I error).
  • It is often set before conducting the hypothesis test and represents the threshold for deciding whether to reject the null hypothesis.
  • Commonly used significance levels include 0.05 (corresponding to a 5% chance of making a Type I error) and 0.01 (corresponding to a 1% chance of making a Type I error).

  • goog_1323895243The significance level (often denoted by 
  • α
  • ) is associated with hypothesis testing.
  • It represents the probability of rejecting the null hypothesis when it is actually true (i.e., making a Type I error).
  • It is often set before conducting the hypothesis test and represents the threshold for deciding whether to reject the null hypothesis.
  • Commonly used significance levels include 0.05 (corresponding to a 5% chance of making a Type I error) and 0.01 (corresponding to a 1% chance of making a Type I error).

Relationship between Confidence Level and Significance Level:

  • There's an inverse relationship between confidence level and significance level. As one increases, the other decreases.
  • For example, if you construct a 95% confidence interval, you're implicitly stating that you're willing to accept a 5% chance of making a Type I error (rejecting the null hypothesis when it's actually true) in hypothesis testing.
  • Similarly, if you set a significance level of 0.05 in hypothesis testing, you're implicitly stating that you want to be 95% confident in the result if you reject the null hypothesis.

Parametric Test vs Nonparametric Test

Parametric and Nonparametric test describe situations where each is the more appropriate type of the test.

Parametric Test
  • Parametric Test are statistical tests in which we make assumptions regarding the distribution of the population.
Nonparametric Test
  • Nonparametric Test are also called as distribution free tests that do not make any assumption regarding the distribution of the parameter under study.
  • Used when either of scenarios:
    • The data do not meet distributional assumptions
    • There are outliers
    • The data under analysis is ordinal
    • Hypothesis test does not concern a parameter
Credits: https://miro.medium.com/v2/resize:fit:1200/1*s3ygduFb-WST4BSmvMi4Ng.jpeg



    Conclusion

    One way of estimating population statistics is through point estimators. These estimators provide a means of approximating the exact value of the population statistics. They represent the most informed guess we can make based on the available sample data. However, confidence intervals are usually preferred as they give a range of values for the population statistic that can be trusted with a high level of confidence, instead of a highly precise estimate.

    Statistical significance does not always imply practical significance. In reality, several other factors must be considered. Even though statistical tests may show highly significant results, a very large sample size can lead to small effects in absolute terms.


    Credits and References

    https://editor.analyticsvidhya.com/uploads/37660critical.jpg
    https://www.youtube.com/watch?v=087s7jAa6EA
    https://chat.openai.com/ - Big help with examples while understanding these concepts
    FRM and Head First Statistics Books

    No comments:

    Post a Comment

    Scarcity Brings Efficiency: Python RAM Optimization

      In today’s world, with the abundance of RAM available, we rarely think about optimizing our code. But sooner or later, we hit the limits a...