Suppose that you work for a pharmaceutical company that wants to put a weigh-loss treatment on the market. Their researchers came up with two treatments (each one developed by a different team), but only one of them can be manufactured in industrial-scale and they are counting on your statistics skills to tell them which treatment is more effective.
You have performed clinical trials for both treatments with the following results:
|
|
||||||||||||||||
You might be tempted to say that treatment B is better given that people lost in average 20 Pounds compared to the 10 Pounds for treatment B, but does this hypothesis hold water?
The first assumption we need to make is that weight loss is a random variable that follows some distribution. Assuming a Normal or Gaussian distribution is a reasonable choice given how ubiquitous this distribution is in nature: You will find it on people's heights and weights, IQ scores, errors in measurements, lifetimes of electrical devices (such as light bulbs, computer microchips), etc.
Normal distributions are this common because of an important theorem in statistics named "The Central Limit Theorem". Follow this link to learn more about it on Khan Academy.
Normal distributions are "bell-shaped":
Figure 1 - Gaussian distribution with $\mu=10$ and $\sigma=10$
Figure 2 - Gaussian distribution with $\mu=20$ and $\sigma=20$
Figures 1 and 2 were plotted using the same horizontal scale, so it is clear how these distributions differ. The obvious difference is that due to the means not being the same (the first one is 10 and second one is 20) these graphs are out of alignment, but the most important difference is that the bell-shaped graph in figure 2 is "fatter". Remember that the standard deviation is equal to the square root of the variance. The variance being the arithmetic average of the squared distances of each data point from the distribution's mean:
$\sigma^2 = \frac{1}{N} \cdot \sum\limits_{i=1}^N (x_i - \mu)^2$
The standard deviation gives you a metric of how much the data spreads-out, thus figure 2 is wider due to its greater standard deviation. This means that even though the mean is greater for the second distribution (more weight lost in average), there is a greater uncertainty due to the data being more spread-out.
But other than using visual inspection as we did, could we compare these distributions in a more precise way? That is, could we calculate some number that reflects how much these distributions differ? Hypothesis test comes to the rescue.
How the weight loss treatment affects a given patient (more specifically how much weight is lost) is a random variable that follows its respective probability distribution:
$W_A$: How much weight is lost for a given patient subjected to treatment A which is sampled at random.
$W_B$: How much weight is lost for a given patient subjected to treatment B which sampled at random.
A question we could make is: If we sample two patients, one subjected to treatment A and the other for treatment B, could we get that both lost the same amount of weight? If so, what's the probability of this happening?
$W_A = W_B$
This is the same as:
$W_B - W_A = 0$
We see that these two random variables can be combined into a single new variable $W_{B-A} = W_B - W_A$. This trick makes possible to go around the complication of working with two independent random variables. We need to analyze the distribution of a single random variable now, specifically we want to determine if it's possible that $W_{B-A} = 0$.
Given that both $W_A$ and $W_B$ follow Gaussian distributions, the new variable $W_{B-A} = W_B - W_A$ follows a Gaussian distribution as well. We only need to find the mean and the standard deviation of the new variable:
$\mu_{B-A} = \mu_B - \mu_A$
$\sigma_{B-A}^2 = \sigma_B^2 + \sigma_A^2$
$\mu_{B-A} = 20 - 10 = 10$
$\sigma_{B-A} = \sqrt{{20}^2 + {10}^2} = 10 \sqrt{5}$
Thus, our hypothesis test boils down to the following then:
$H0: W_{B-A} = 0$
$H1: W_{B-A} \neq 0$
We know that the radius of the confidence interval is given by:
$x = \alpha \cdot \frac{\sigma}{\sqrt{N}}$
The usual confidence for hypothesis test is 95%. Given that the number of patients (size of the sample) is 10,000, which is greater than 30, we may use a Gaussian approximation and take $\alpha$ to be equal to 1.96. Otherwise, we would require a T-Table to determine $\alpha$ for a two-tailed distribution and 95% confidence:
Source: Stanford MIPS BIOE222
Note that the table also gives an $\alpha$ of 1.96 for the following parameters: Two-tailed, 95% confidence and degrees of freedom equal to large (column at last row and fourth column). The label "0,05 (2-tails)" on the column means that there is a 5% chance that our sample will fall outside the confidence interval, which is the same as saying that there is a 95% chance that our sample will fall inside the confidence interval. These are only two different ways to say the same thing: We choose a confidence level of 95%.
Figure 3 - 95% confidence interval
For the new random variable $W_{B-A}$, we have an estimate for the mean and standard deviation given by the following:
$\mu_{B-A} = 10$
$\sigma_{B-A} = 10\sqrt{5}$
We want to determine what's the true mean $\mu^{true}_{B-A}$. Given the confidence interval formula:
$x = \frac{\sigma}{\sqrt{N}}$
$x = \frac{10\sqrt{5}}{\sqrt{10000}} = \frac{10\sqrt{5}}{100} = \frac{\sqrt{5}}{100}$
This means that our confidence interval is given by:
$(\mu_{B-A}-x) \le \mu^{true}_{B-A} \le (\mu_{B-A} + x)$
$(10 - \frac{\sqrt{5}}{100}) \le \mu^{true}_{B-A} \le (10 + \frac{\sqrt{5}}{100})$
In other words, we know with 95% confidence that the true mean is inside this interval.
Since we know with 95% confidence that $\mu^{true}_{B-A}$ can not be zero. Treatment A and B clearly can not be equally effective. As a matter of fact, we know with 95% confidence that $\mu^{true}_{B-A}$ is at least equal to $10-\frac{\sqrt{5}}{100}$ or approximately 9.98.
Being $W_{B-A} = W_B-W_A \ge 9.98$, this means that treatment B is more effective than treatment A.
No comments:
Post a Comment