1, What is Hypothesis Testing?

1, What is Hypothesis Testing?

**is a method of statistical inference. Based on data collected from a survey or an experiment, you calculate what is the probability (p-value) of observing the statistics from your data given the null hypothesis is true. Then you decide whether to reject the null hypothesis comparing p-value and significance level. It is widely used to test the existence of an effect.**

Hypothesis Testing

Hypothesis Testing

2, What is the p-value?

2, What is the p-value?

**is the probability of observing the data if the null hypothesis is true. A smaller p-value means a higher chance of rejecting the null hypothesis.**

P-value

P-value

3, What is the confidence level?

3, What is the confidence level?

The

**confidence level**in hypothesis testing is the probability of rejecting the null hypothesis when the null hypothesis is True:

P(Not Rejecting H0|H0 is True) = 1 - P(Rejecting H0|H0 is True)

The default statistical power is set at 95%.

4, What is the confidence interval?

4, What is the confidence interval?

In contrast to point estimation, a

**confidence interval**is an interval estimation of a parameter obtained through statistical inference. It is calculated by:

[point_estimation - cv*sd, point_estimation + cv*sd]

where cv is the critical value based on the sample distribution, and sd is the standard deviation of the sample.

It is essential to interpret confidence in the confidence interval. For example, if I say the 95% confidence interval for the bus waiting time is [5 min,10 min], what am I actually saying? Please check out my article here for more details:

5, What is the statistical power?

5, What is the statistical power?

**measures the probability of rejecting the null hypothesis when the null hypothesis is False:**

Statistical power

Statistical power

P(Reject H0|H0 is False) = 1- P(Not Rejecting H0|H0 is False)

The default statistical power is set at 80%.

6, What is Type I error, and what is Type II error?

6, What is Type I error, and what is Type II error?

**is P(Rejecting H0|H0 is True), is False Positive (Thank Koushal Sharma for capturing the typo here), is ⍺, is one minus confidence level;**

Type I error

Type I error

**is P(Not Rejecting H0|H0 is False), is False Negative, is β, is one minus statistical power.**

Type II error

Type II error

There is a trade-off between Type I error and Type II error, meaning that if everything else stays the same, to decrease Type I error, we need to increase Type II error.

If you are interested in connecting Type I error and Type II error with the classification metrics in machine learning models, read my article for more details:

7, What is the Central Limit Theorem (CLM)?

7, What is the Central Limit Theorem (CLM)?

The

**Central Limit Theorem**states that no matter what is the population’s original distribution, when taking random samples from the population, the distribution of the means or sums from the random samples approaches a normal distribution, with mean equals to the population mean, as the random sample size gets larger:

image from Wikipedia

8, What is the Law of Large Numbers?

8, What is the Law of Large Numbers?

**states that as the number of trials gets large enough, the average result of the trials will become closer to the expected value. For example, when you toss a fair coin for 1000 times, you are more likely to see Heads half of the time than tossing a fair coin only 100 times.**

Law of Large Numbers

Law of Large Numbers

9, What is the standard error? What is the standard error of mean?

9, What is the standard error? What is the standard error of mean?

**of a statistic is the standard deviation of its sampling distribution or an estimate of that standard deviation.**

Standard error

Standard error

Using CLM, we can estimate the standard error of mean by using population standard deviation divided by the square root of sample size n. If the population standard deviation is unknown, we can use the sample standard deviation as an estimation.

10, How to choose the sample size for an experiment?

10, How to choose the sample size for an experiment?

The

**sample size**is closely related to the sample’s standard error, the desired confidence level, power, and effect size. Sample size increases as the sample’s standard error, confidence level, and power increases, or as the sample’s effect size decreases. Please check out this article for the intuition behind:

11, What is bootstrapping?

11, What is bootstrapping?

**is one of the re-sampling techniques. Given a sample, you repeatedly take other random samples from it**

Bootstrapping

Bootstrapping

**. Bootstrapping is useful when the sample size is small, and when you need to estimate the empirical distribution. We can estimate the standard error of the median using bootstrapping. Please read the article below for more details:**

*with replacement***How to estimate the standard error of the median: The Bootstrap Strategy**In this article, I will discuss the method we can use to estimate the standard error of the median. I will first…towardsdatascience.com

12, What is sample bias?

12, What is sample bias?

**is the sample taken to statistic inference is not a great representation of the entire population. It is because of several reasons:**

Sample bias

Sample bias

1,

**sampling bias**: non-random sampling;

2,

**selection bias**: the sample doesn’t represent the entire population. For example, passing a survey in universities when you want to estimate the average income for all adults;

3,

**response bias**: either because of too few responses or because only certain types of subjects will respond to the survey. For example, a survey for a professor’s teaching skills may only respond by students like or really hate the professor;

4,

**survivorship bias**: bias from overlooking subjects that did not make it past the selection process.

13, How to detect outliers?

13, How to detect outliers?

**are observations that differ significantly from other observations. Detecting outliers is the same as defining the difference. The most straightforward way is to plot the variable and find the data points that are far away from others. To quantify the difference, we can use the quartiles and**

Outliers

Outliers

**Interquartile Range**(IQR). IQR is the third quartile minus the first quartile (Q3-Q1). The outliers are any data points that are less than Q1–1.5*IQR, or larger than Q3+1.5*IQR.

If the data follows a normal distribution, the outliers are the points with a Z score larger than 3 or smaller than -3.

What is mean/median/mode, when is median better than mean in measuring central tendency?

14, What is Bayesian inference?

14, What is Bayesian inference?

**is a method of statistical inference in which**

Bayesian inference

Bayesian inference

**Bayes’ theorem**is used to update the probability for a hypothesis as more evidence or information becomes available. Bayes’ theorem is stated below:

where P(A) is a prior belief, P(B) is the evidence, and P(B|A) is the conditional probability of event B occurs when A occurs.

15, What is Maximum Likelihood Estimation (MLE)?

15, What is Maximum Likelihood Estimation (MLE)?

**is estimating the parameter by maximizing the likelihood function using Bayes’ theorem. According to Bayes’ theorem:**

Maximum Likelihood Estimation

Maximum Likelihood Estimation

where P(θ) is the prior distribution for the parameter; P(y|θ) is the likelihood function describing the likelihood of observing the data points y when we have the parameter θ; P(y) is the evidence, which is usually used to normalize the probability. Maximizing P(θ|y) is the goal of finding the optimal θ, where we maximize the conditional probability of having θ given all the data points y. In practice, we can easily calculate P(y|θ) once we know the distribution. Thus, we solve the optimizing problem by maximizing the likelihood function P(y|θ) with respect to θ.

1, 50% of all people who receive a first interview receive a second interview

2, 95% of your friends that got a second interview felt they had a good first interview

3, 75% of your friends that DID NOT get a second interview felt they had a good first interview

If you feel that you had a good first interview, what is the probability you will receive a second interview?

The key to solving problems like this is to define the events carefully. Suppose your friends are a good representation of the entire population:

- Let's define feel good about the first interview as event A, and define receive the second interview event B;
- According to 1, P(B)=0.5, thus P(not B) is one minus P(B), which is 0.5 as well;
- According to 2, P(A|B) =0.95;
- According to 3, P(A|not B) = 0.75.
- Given P(B), P(A|B), P(A|not B), what is P(B|A)?

According to Bayes’ theorem:

thus:

17, What is the difference between correlation and causation?

17, What is the difference between correlation and causation?

**is the relationship between two variables, it can be positive, negative, zero, or little correlation depending on the sign and size of the following equation:**

Correlation

Correlation

Correlation between X and Y

where:

Covariance of X and Y

Cov(X, Y) is the

**Covariance**of the two variables, and Cor(X, Y) is normalized by the standard deviation of X and Y (Sx, Sy) so that correlation can be between -1 and 1. When correlation equals to -1, X, Y have a perfect negative correlation and when it equals to 1, they have a perfect positive correlation. When the absolute value of correlation is close to zero, X, Y have little correlation with each other.

**is much more difficult to capture, it is the relationship between X and Y such that X has caused Y to happen, or vice versa. For example, in a study, you may observe people eat more vegetables a day are healthier, there is clearly a positive correlation between eating vegetables and healthy level. However, if only based on this information, you cannot claim that eat more vegetable causes you to be healthier, which is stating a causal relationship. You may observe this relationship because the subjects in your study have other healthy lifestyles (omitted variables) that could improve their health level, and eating vegetables is just one of the healthy habits they have. Finding a causal relationship requires additional information and careful modeling.**

Causation

Causation

18, What is Simpson’s Paradox?

18, What is Simpson’s Paradox?

**refers to the situations in which a trend or relationship that is observed within multiple groups disappears or reverses when the groups are combined. The quick answer to why there is Simpson’s paradox is the existence of confounding variables. I have an article that explains Simpson’s paradox with an example:**

Simpson’s paradox

Simpson’s paradox

19, What is the confounding variable?

19, What is the confounding variable?

A

**confounding variable**is a variable that is correlated with both the dependent variable and the independent variable. For example, when checking the causal relationship between smoking and death rate, age is a confounding variable because as age goes up, the death rate increases, and the smoking rate decreases. Failing to control age can cause Simpson’s Paradox in statistical inference.

20, What is A/B testing, when can we use it, and when can we not use it?

20, What is A/B testing, when can we use it, and when can we not use it?

**is conducting a randomized experiment with two variants, A and B. Through statistical hypothesis testing or “two-sample” hypothesis testing, A/B testing is a way to compare two versions of a single variable, typically by testing a subject’s response to variant A against variant B, and determining which of the two variants is more effective. It is commonly used to improve and optimize the user experience and marketing strategies.**

A/B testing

A/B testing

Not every experiment can be conducted by A/B testing:

- A/B testing is not good for testing long-term effect
- A/B testing can only compare two versions, but cannot tell you what you are missing
- A/B testing cannot be used when there is a network effect in the market. For example, you cannot increase some consumer’s prices while decreasing others in the same market because it will twist the market demand.

21, What is PMF/PDF?

21, What is PMF/PDF?

A

**probability mass function**(

**PMF**) is a function that gives the probability that a discrete random variable is exactly equal to some value. The PMF does not work for continuous random variables, because for a continuous random variable P(X=x)=0 for all x∈R. Instead, we can usually define the

**probability density function (PDF)**. The PDF is the

**density**of probability rather than the probability mass:

PDF at x0

22, Summarize the important distributions.

22, Summarize the important distributions.

I have an article that summarizes the most important distributions, including their assumptions, PDF/PMF, simulations, etc. Please check it out here:

Here are all the 22 fundamental statistical questions. Help this article helps you prepare for your interviews or refreshing your memories in the stats class. Thank you for reading!

If you feel ready to interview and apply for a job as a data scientist, please go to the following link.