two populations are independent. What happens when the two populations are not independent?

Transcription

1 PHP2500: Introduction to Biostatistics Lecture XI: Hypothesis Testing (cont.) 1

2 In previous lectures we have seen the procedures for hypothesis testing for population mean, or difference between two population means, when we have normal distributions to start with and the two populations are independent. What happens when the two populations are not independent? What happens when we do not start with normal distributions? 2

3 difference of means of paired observations X N(µX, σ x) 2 Y N(µY, σ y) 2 where Xk is paired with Yk (before/after, left-hand/right-hand, paired treated/control...) H0 : µx = µy, H1 : µx µy But since X and Y are not independent, X and Ȳ are not independent. We do not have the simple result X Ȳ N(µX µy, σ X 2 /n + σ2 Y /n) (why not?) 3

4 Solution: For each pair, we form the difference Dk = Xk Yk. Thus D1,...,Dk N(µX µy, σ D 2 ) and we estimate σ2 D with the sample variance S D 2. The test statistic for testing D = d0 is then D d0 Sd/ student t with n degree of freedom n 1. 4

5 When we do not start with normal distributions, Central limit theorem is our friend again, as long as we have large samples. 5

6 Example: Bernoulli Suppose the incidence rate for children at 5 for disease W is.0137 (137 per 10,000) in We want to know if the incidence rate in Providence is the same as the national rate. A sample of 2000 children were randomly selected and their medical record queried to see if they had caught the disease in of them had the disease. 6

7 1. select a probability model: Bernoulli(p) 2. Set up the hypotheses: H0 : p = H1 : p determine a test statistic: 2000 is a large sample, by CLT, X. N(p, p(1 p)/n), thus X p N(0, 1) p(1 p)/n 4. Under H0, X.0137 N(0, 1).0137(1.0137)/2000 We observe X = 30/2000 =.015, thus the test statistic is (1.0137)/2000 =

8 % % For two-sided test, the critical regions for α =.05 are (, 1.96) and (1.96, ). We do not reject H0 at significance level 0.05 (thus we certainly do not reject H0 at any more significant level, such as.01). OR, we compute the p-value 2P(Z >.519) = 2.30 = >.05 and we do not reject H0 at significance level.05 8

9 Example: Suppose we want to compare the average daily visit for two emergency rooms, A and B. For each we record the daily visit number for a year. On average, there is 15.4 visits to ER A a day, and 14.8 visits to ER B a day. Do these two ERs have the same daily visit rates? 1. Probability model: Poisson, for events randomly happen over time 2. H0 : λa = λb, H1 : λa λb 3. Test statistic: For either ER we observe 365 days, by CLT, XA. N(λA, λa/n), XB. N(λB, λb/n), XA XB. N(λA λb, λa/n + λb/n) 9

10 ( XA XB) (λa λb). N(0, 1) λ A/n + λb/n 4. Under H0, (λa λb) = 0, ( XA XB). N(0, 1) 2λ/n We can pool estimate λ from the two samples and get ˆλ = ( )/( ) = 15.1 Test statistic: ( ) = 2.09 > Z.025 = /365 OR,p value = 2P(Z > 2.09) = =.036 <.05 Reject H0. 10

11 Review: The logic for hypothesis testing: If the null hypothesis, instead of the alternative hypothesis, is true, should I be surprised by the data? I am surprised, if the the probability of observing such or more extreme result is small based on H0. This probability is called the p-value. By convention, most people reject the null hypothesis if p-value is smaller than The smaller the p-value, the stronger my doubt is about H0, thus the more significant the result is against H0. 11

12 Review: The procedure for hypothesis testing 1. Select the probability model 2. Set up the null and alternative hypothesis 3. determine a test statistic 4. compute p-value reject H0 if p-value less than significance level OR find critical regions at specific significance levels reject H0 if the statistic is in the critical regions 12

13 Possible results from hypothesis testing: Decision H0 True Reject, Type I error (α) Not reject H1 True Reject Not reject, Type II error β Type I error: Rejecting H0 when H0 is true. Type II error: Fail to reject H0 when H0 is false (H1 is true) Power: The ability to reject H0 when H0 is false P0( Reject H0) = P(Reject H0 H0 true) = α this probability is the significance level (Type I error rate) P1( Reject H0) = P(reject H0 H1 true) = 1 β this probability is the power of a hypothesis test 13

14 14 type I error is the area of the area of red shaded region. For any decision rule reject H0 if test statistic is greater than c, the 2 0 c critical region, reject H0 sample size of 100, we form the test statistic T = X 5 10/ = X Under H0, we know T follows Z distribution. Suppose we have a normal model and know the variance is For a H0 : µ = µ0 = 5 H1 : µ = µ1 = 8 > µ0. Consider a one-sided test first:

15 Demo 1: type I error and choice of critical region 15

16 16 We can try to increase power by using a smaller cutoff, but this would increase the type I error. We can try to reduce type I error by using a larger cutoff, but this would increase type II error (reducing power) β α H0 H1 critical region, reject H0 What about type II error? X 8 Under H1, 10/ = X 8 N(0, 1) 100 Under H1, X 5 = ( X 8) + 3 N(3, 1)

17 Demo: the trade-off between type I and type II error. 17

18 critical region, reject H0 critical region, reject H0 Two-sided test:

19 We have seen that for a hypothesis test, there is a trade off between type I and II errors. For the same study design, we cannot simultaneously reduce both of them. The common practice is to fix the type I error at a small level, such as.05 or.01, so that we know that at least we are not rejecting H0 too often when we should not. What other factors affect power, for a given type I value? 19

20 β α H0 H1 H1 critical region, reject H0 If H1 is true, the larger the difference between µ0(nullvalue) and µ1(alternative), the higher the power. (1. Type I error) 2. effect size

21 3. Sample size We know that X. N(µ, σ 2 /n). This means as sample size increases, the sample mean gets more concentrated near the true mean. Thus the null and alternative hypothesis becomes easier to separate. H0 H1 H0 H1 21

22 demo3 22

23 Computation of power 1. Write down the two hypothesis H0 and H1 2. Write down the probability models based on each hypothesis 3. determine the test statistic 4. For given type I error rate (α), effect size and sample size, determine the critical region (rejection region) 5. computer power 1-β 23

24 Computation of power : one sided test Example: Suppose we want to test whether the mean of a population is 12 or less than 12. We assume normal distribution with known variance 36. What is the power of this test if the true mean is 10, and we have a sample size of 25, and set significance level.05? 1. H0 : µ = 12;H1 : µ < Under H0: X N(12, 36) X N(12, 36/n) Truth:X N(10, 36) X N(10, 36/n) 3. Test statistic: X Since we have a one sided test with H1 : µ < 12, we will reject H0 when the test statistic is less than a cutoff. X 12 α = 0.05 = P0(Reject H0) = P0( 36/ 25 < C) = P(Z < C) From the Z-table we know C = Under H1 24

25 X 12 POWER = P1( < 1.645) 36/ 25 = P1( X /5 = P1( X 10 6/5 = P1( X 10 6/ /5 < < 1.645) < 1.645) /5 ) = P(Z <.0217) = 1 P(Z >.0217) =.52 25

26 Example: Suppose we want to test whether the mean of a population is 12 or greater than 12. We assume normal distribution with known variance 36. What is the power of this test if the true mean is 10, and we have a sample size of 25, and set significance level.05? 1. H0 : µ = 12;H1 : µ > Under H0: X N(12, 36) X N(12, 36/n) Truth : X N(10, 36) X N(10, 36/n) 3. Test statistic: X Since we have a one sided test with H1 : µ > 12, we will reject H0 when the test statistic is greater than a cutoff. α = 0.05 = P0(Reject H0) = P0( From the Z-table we know C = X 12 36/ 25 > C) = P(Z > C) 5. Under H1 26

27 X 12 POWER = P1( > 1.645) 36/ 25 = P1( X /5 = P1( X 10 6/5 = P1( X 10 6/5 + = P(Z > 3.31) /5 > > 1.645) > 1.645) /5 ) When truth is µ = 10, the probability that you will be able to reject H0 in a test for µ = 12 versus µ > 12, is nearly 0. 27

28 Example: Suppose we want to test whether the mean of a population is 12 or not 12. We assume normal distribution with known variance 36. What is the power of this test if the true mean is 10, and we have a sample size of 25, and set significance level.05? 1. H0 : µ = 12;H1 : µ Under H0: X N(12, 36) X N(12, 36/n) Truth X N(10, 36) X N(10, 36/n) 3. Test statistic: X Since we have a two-sided test, we will reject H0 when the absolute value of test statistic is greater than a cutoff. X 12 α = 0.05 = P0(Reject H0) = P0( 36/ 25 > C) = P( Z > C) From the Z-table we know C = Z.025 = Under H1 28

29 X 12 P1( > 1.96) 36/ 25 = P1( X 12 6/5 > 1.96) + P1( X 12 6/5 < 1.96) = P1( X /5 = P1( X 10 6/5 + = P(Z > /5 = P(Z > 3.63) + P(Z <.29) > 1.96) + P1( X /5 > 1.96) + P1( X 10 6/ / ) + P(Z < /5 6/5 ) = P(Z > 3.63) + P(Z >.29) = < 1.96) < 1.96)

30 In general, for X1, X2,...,Xn N(µ, σ 2 ), the power for test H0 : µ = µ0 versus H1 : µ < µ0 is P(Z < µ 0 µ1 σ/ n Z α) the power for test H0 : µ = µ0 versus H1 : µ > µ0 is P(Z > µ 0 µ1 σ/ n + Z α) the power for test H0 : µ = µ0 versus H1 : µ µ0 is P(Z < µ 0 µ1 σ/ n Z α/2) + P(Z > µ 0 µ1 σ/ n + Z α/2) 30

31 Exercise: Now if I give you the same set up, just different numberes: different H0 mean (not 12, but 200), different truth (not 10, but 180), different variance( not 36, but 64), different significance level (not.05, but.01), different sample size (not 25, but 49), can you compute the power for the test H0 : µ = 200 versus H1 : µ > 200? 31

32 Next topic: We now know that type I error, Type II error (1-power), effect size and sample size are all connected. Can we determine a necessary sample size when we need to meet certain requirements of error rates? For one sample two-sided test:h0 : µ = µ0 versus µ µ0 n = (Z α/2 + Zβ) 2 σ 2 (µ1 µ0) 2 For one sample one-sided test:h0 : µ = µ0 versus µ µ0 n = (Z α + Zβ) 2 σ 2 (µ1 µ0) 2 the smaller the error rate, the larger the sample size the larger the variance, the larger the sample size the larger the effect size, the smaller the sample size 32