On the methods of testing the homogeneity of two independent samples

Methods for testing the homogeneity of two independent samples refer to a classic area of mathematical statistics. Various criteria for testing the statistical hypothesis of homogeneity in different statements have been developed and their properties have been studied for more than 110 years since publication of the fundamental Student’s article. Nowadays, the streamlining of the totality of gained scientific results has become an urgent problem. It is necessary to analyze the whole variety of problem statements for testing the statistical hypotheses of the homogeneity of two independent samples, as well as the corresponding statistical criteria. Such an analysis is the goal of the article. We summarize the main results regarding the methods for testing the homogeneity of two independent samples and their comparative study which allows system analysis of the diversity of such methods in order to select the most appropriate for processing specific data. The main statements of the problem of testing the homogeneity of two independent samples are formulated using the basic probabilistic-statistical model. A comparative analysis of the Student and Cramer — Welch criteria designed to test the homogeneity of mathematical expectations is presented along with substantiation of the recommendation on the widespread use of the Cramer – Welch criterion. The criteria of Wilcoxon, Smirnov, Lehmann – Rosenblatt are considered among nonparametric methods for testing homogeneity. Two myths about the Wilcoxon criteria are dismantled. Analysis of the publications of the founders revealed the incorrectness of the term «Kolmogorov – Smirnov criterion». To verify the absolute homogeneity, i.e. coincidence of the distribution functions of samples, it is recommended to use the Lehmann – Rosenblatt criterion. The current problems of the development and application of nonparametric criteria are discussed, including the difference between nominal and real significance levels, which complicates comparison of the criteria in power. The necessity of taking into account the coincidence of the sample values (from the view point of the classical theory of mathematical statistics, the probability of coincidences is 0) is marked.

1. Student. The probable error of a mean / Biometrika. 1908. N 6(1). P. 1 – 25.

2. Bol’shev L. N., Smirnov N. V. Tables of mathematical statistics. — Moscow: Nauka, 1983. — 416 p. [in Russian].

3. Orlov A. I. Distributions of real statistical data are not normal / Nauch. Zh. KubGAU. 2016. N 117. P. 71 – 90 [in Russian].

4. Orlov A. I. Applied statistics. — Moscow: Йkzamen, 2006. — 671 p. [in Russian].

5. Orlov A. I. Statistical hypothesis testing of homogeneity of mathematical expectations of two independent samples: the Cramer – Welch instead of the Student’s test / Nauch. Zh. KubGAU. 2015. N 110. P. 197 – 218 [in Russian].

6. Borovkov A. A. Mathematical statistics. 4th edition. — Moscow: Lan’, 2010. — 704 p. [in Russian].

7. Kramer G. Mathematical methods of statistics. 3rd edition. — Moscow – Izhevsk: NIC «Regulyarnaya i khaoticheskaya dinamika», 2003. — 648 p. [in Russian].

8. Orlov A. I. On the application of statistical methods in biomedical research / Vestn. AMN SSSR. 1987. N 2. P. 88 – 94 [in Russian].

9. Orlov A. I. Real and nominal significance levels in statistical hypothesis testing / Nauch. Zh. KubGAU. 2015. N 114. P. 42 – 54 [in Russian].

10. Orlov A. I. Statistical simulations method in applied statistics / Zavod. Lab. Diagn. Mater. 2019. Vol. 85. N 5. P. 67 – 79 [in Russian].

11. Orlov A. I. Current status of nonparametric statistics / Nauch. Zh. KubGAU. 2015. N 106. P. 239 – 269 [in Russian].

12. Gaek Ya., Shidak Z. Theory of rank tests. — Moscow: Nauka, 1971. — 376 p. [Russian translation].

13. Khollender M., Vul’f D. Nonparametric statistical methods. — Moscow: Finansy i statistika, 1983. —- 518 p. [in Russian].

14. Orlov A. I. Two-sample Wilcoxon test — analysis of two myths / Nauch. Zh. KubGAU. 2014. N 104. P. 91 – 111 [in Russian].

15. Orlov A. I. Consistent tests of absolute homogeneity of independent samplings / Zavod. Lab. Diagn. Mater. 2012. Vol. 78. N 11. P. 66 – 70 [in Russian].

16. Guidelines. Testing the homogeneity of two samples of product parameters when estimating its technical level and quality / Orlov A. I., Mironova N. G., Fomin V. N., Chernomordik O. M. — Moscow: VNIIStandartizatsii, 1987. — 116 p.

17. Orlov A. I. Nonparametric goodness-of-fit Kolmogorov, Smirnov, omega-square tests and the errors in their application / Nauch. Zh. KubGAU. 2014. N 97. P. 32 – 45 [in Russian].

18. Billingsli P. Convergence of probability measures. — Moscow: Nauka, 1977. — 353 p. [in Russian].

19. Orlov A. I. Limit Theory of Nonparametric Statistics / Nauch. Zh. KubGAU. 2014. N 100. P. 31 – 52 [in Russian].

20. Orlov A. I. The Model of Coincidence Analysis in the Calculation of Nonparametric Rank Statistics / Zavod. Lab. Diagn. Mater. 2017. Vol. 83. N 11. S. 66 – 72 [in Russian].

21. Orlov A. I. Testing of homogeneity of paired samples / Nauch. Zh. KubGAU. 2016. N 123. P. 708 – 726 [in Russian].