Statistics of nonnumeric data in forty years (review)

Forty years ago, the statistics of nonnumeric data was singled out into an independent area of mathematical research methods. First the term «statistics of objects of nonnumeric nature» was used. The textbook on nonnumeric statistics is entitled «Nonnumeric Statistics». Statistics of nonnumeric data is one of the four main fields of applied statistics (along with the statistics of numbers, multivariate statistical analysis, statistics of time series and random processes). Statistics of nonnumeric data is divided into statistics in spaces of general nature and sections devoted to specific types of nonnumeric data (statistics of interval data, statistics of fuzzy sets, statistics of binary relations, etc.). Currently, statistics in spaces of general nature is the central part of applied statistics, and the statistics of nonnumeric data that includes it is the main area of applied statistics. This statement is confirmed by analysis of publications in the section «Mathematical Methods of Research» of our journal. This article is devoted to analysis of the main ideas of statistics of nonnumeric data against the background of the development of applied statistics on the basis of the new paradigm of the methods of mathematical research. Different types of nonnumeric data are considered along with the history of development of the statistics of nonnumeric data and statistical science. The basic ideas of statistics in spaces of general nature are analyzed: mean values, laws of large numbers, extreme statistical problems, nonparametric estimators of the probability distribution density, classification methods (diagnostics and cluster analysis), and statistics of the integral type. Some statistical methods for data analysis in specific nonnumeric spaces are briefly discussed: nonparametric statistics (in most cases, real distributions significantly differ from normal), statistics of fuzzy sets, expert estimation theory (Kemeny median as a sample average of expert orderings), etc. Some unsolved problems in statistics on nonnumeric data are also discussed.

1. Orlov A. I. Development of the Methods of Mathematical Research (2006 – 2015) / Zavod. Lab. Diagn. Mater. 2017. Vol. 83. N 1. Part 1. P. 78 – 86 [in Russian].

2. Orlov A. I. Stability in socio-economic models. — Moscow: Nauka, 1979. — 296 p. [in Russian].

3. Orlov A. I. Statistics of Non-Numeric Objects and Expert Estimation / Expert estimates. Questions of cybernetics. Vol. 58. — Moscow: Nauch Sovet AN SSSR po kompleksnoi probleme «Kibernetika», 1979. P. 17 – 33 [in Russian].

4. Tyurin Yu. N., Litvak B. G., Orlov A. I., Satarov G. A., Shmerling D. S. Analysis of non-numeric information / Zavod. Lab. 1980. Vol. 46. N 10. P. 931 – 935 [in Russian].

5. Orlov A. I. Statistics of Non-Numeric Objects (review) / Zavod. Lab. 1990. Vol. 56. N 3. P. 76 – 83 [in Russian].

6. Orlov A. I. Thirty Years of the Statistics of Non-Numeric Objects (review) / Zavod. Lab. Diagn. Mater. 2009. Vol. 75. N 5. P. 55 – 64 [in Russian].

7. Orlov A. I. Organizational-economic modeling: Textbook. Part 1. Nonnumeric statistics. — Moscow: Izd. MGTU im. N. Й. Baumana, 2009. — 542 p. [in Russian].

8. Kuhn T. The Structure of Scientific Revolutions. — Moscow: AST, 2003. — 605 p. [in Russian].

9. Orlov A. I. The New Paradigm of Applied Statistics / Zavod. Lab. Diagn. Mater. 2012. Vol. 78. N 1. Part I. P. 87 – 93 [in Russian].

10. Orlov A. I. The New Paradigm of Mathematical Methods of Research / Zavod. Lab. Diagn. Mater. 2015. Vol. 81. N 7. P. 5 [in Russian].

11. Orlov A. I. About the New Paradigm of Mathematical Methods of Research / Nauch. Zh. KubGAU. 2016. N 122. P. 807 – 832 [in Russian].

12. Orlov A. I. Objects of non-numeric nature / Zavod. Lab. Diagn. Mater. 1995. Vol. 61. N 3. P. 43 – 52 [in Russian].

13. Bernshtein S. N. The current state of probability theory and its applications / Proc. of the All-Russian Congress of Mathematicians in Moscow, April 27 – May 4, 1927. — Moscow – Leningrad: GIZ, 1928. P. 50 – 63 [in Russian].

14. Novikov A. M., Novikov D. A. Methodology. — Moscow: SINTEG, 2007. — 668 p. [in Russian].

15. Discussion on the analysis of interval data / Zavod. Lab. 1990. Vol. 56. N 7. P. 75 – 95 [in Russian].

16. Voshchinin A. P. The method of optimization of objects by interval models of the objective function. — Moscow: MÉI, 1987. — 109 p. [in Russian].

17. Voshchinin A. P., Sotirov G. R. Optimization under uncertainty. — Moscow: MÉI, Sofiya: Tekhnika, 1989. — 224 p. [in Russian].

18. Voshchinin A. P., Akmatbekov R. A. Optimization of regression models and design of experiments. — Bishkek: Ilim, 1991. — 164 p. [in Russian].

19. Voshchinin A. P. Method of data analysis with interval errors in problems of hypothesis testing and estimation of parameters of implicit linearly parameterized functions / Zavod. Lab. Diagn. Mater. 2000. Vol. 66. N 3. P. 51 – 64 [in Russian].

20. Voshchinin A. P. Interval analysis of data: development and prospects / Zavod. Lab. Diagn. Mater. 2002. Vol. 68. N 1. P. 118 – 126 [in Russian].

21. Voshchinin A. P., Bronz P. V. Build-up of analytical models according to simulating experiment in problems of sensitivity analysis and economic risk estimation / Zavod. Lab. Diagn. Mater. 2007. Vol. 73. N 1. P. 101 – 109 [in Russian].

22. Voshchinin A. P., Skibitskii N. V. Interval Approach to the Uncertainty of Measurements and Calibration of Digital Measurement Systems / Zavod. Lab. Diagn. Mater. 2007. Vol. 73. N 11. P. 66 – 71 [in Russian].

23. Gus’kova E. A., Orlov A. I. Interval Linear Pair Regression (generalizing paper) / Zavod. Lab. Diagn. Mater. 2005. Vol. 71. N 3. P. 57 – 63 [in Russian].

24. Orlov A. I. Applied statistics. — Moscow: Ékzamen, 2006. — 671 p. [in Russian].

25. Orlov A. I. Decision theory. — Moscow: Ékzamen, 2006. — 576 p. [in Russian].

26. Orlov A. I., Lutsenko E. V. System fuzzy interval mathematics. — Krasnodar: KubGAU, 2014. — 600 p. [in Russian].

27. Orlov A. I. Statistics of Interval Data (generalizing paper) / Zavod. Lab. Diagn. Mater. 2015. Vol. 81. N 3. P. 61 – 69 [in Russian].

28. Skibitskii N. V., Seval’nev N. V. Interval Model in Optimal Control Problems / Zavod. Lab. Diagn. Mater. 2015. Vol. 81. N 11. P. 73 – 80 [in Russian].

29. Skibitskii N. V. Construction of Direct and Inverse Static Characteristics of the Objects by Interval Data / Zavod. Lab. Diagn. Mater. 2017. Vol. 83. N 1. Part 1. P. 87 – 93 [in Russian].

30. Tarantsev A. A. On the relationship of interval analysis with probability theory / Zavod. Lab. Diagn. Mater. 2004. Vol. 70. N 3. P. 60 – 65 [in Russian].

31. Orlov A. I. Econometrics. — Moscow: Ékzamen, 2002. — 576 p. [in Russian].

32. Orlov A. I. Fuzzy Sets Theory as the Part of Probability Theory / Nauch. Zh. KubGAU. 2013. N 92. P. 51 – 60 [in Russian].

33. Orlov A. I., Lutsenko E. V., Loiko V. I. Organizational-economic methods, mathematical modeling and software for controlling, innovation and management: monograph. — Krasnodar: KubGAU, 2016. — 600 p. [in Russian].

34. Kramer G. Mathematical methods of statistics. — Moscow: Mir, 1975. — 648 p. [in Russian].

35. Smirnov N. V., Dunin-Barkovskii I. V. Course of probability theory and mathematical statistics for technical applications. 3rd Edition. — Moscow: Nauka, 1969. — 512 p. [in Russian].

36. Bol’shev L. N., Smirnov N. V. Tables of mathematical statistics. 3rd edition. — Moscow: Nauka, 1983. — 416 p. [in Russian].

37. Kagan A. M., Linnik Yu. V., Rao P. R. Characterization problems of mathematical statistics. — Moscow: Nauka, 1972. — 656 p. [in Russian].

38. Actual problems of cybernetics (applied statistics) — Moscow: Znanie, 1981. — 64 p. [in Russian].

39. Orlov A. I. On the restructuring of statistical science and its applications / Vestn. Statist. 1990. N 1. P. 65 – 71 [in Russian].

40. Orlov A. I. Probabilistic models of specific types of objects of non-numeric nature / Zavod. Lab. Diagn. Mater. 1995. Vol. 61. N 5. P. 43 – 51 [in Russian].

41. Tyurin Yu. N., Litvak B. G., Orlov A. I., Satarov G. A., Shmerling D. S. Analysis of non-numeric information. — Moscow: Nauch. Sovet AN SSSR po kompleksnoi probleme «Kibernetika», 1981. — 80 p. [in Russian].

42. Andreenkov V. G., Orlov A. I., Tolstova Yu. N., Eds. Analysis of non-numerical information in sociological research. — Moscow: Nauka, 1985. — 220 p. [in Russian].

43. Orlov A. I. The criteria for selection of indicators of efficiency of scientific activity / Kontrolling. 2013. N 3(49). P. 72 – 78 [in Russian].

44. Loiko V. I., Lutsenko E. V., Orlov A. I. Actual approaches in scientometrics. — Krasnodar: KubGAU, 2017. — 532 p. [in Russian].

45. Orlov A. I. Characterization of average values by means of measurement scales / Nauch. Zh. KubGAU. 2017. N 134. P. 877 – 907 [in Russian].

46. Psychological measurements. Digest of article. — Moscow: Mir, 1967. — 196 p. [in Russian].

47. Pfanzagl I. Measurement Theory. — Moscow: Mir, 1976. — 248 p. [Russian translation].

48. Tolstova Yu. N. A brief history of the development of representative measurement theory / Zavod. Lab. Diagn. Mater. 1999. Vol. 65. N 3. P. 49 – 56 [in Russian].

49. Orlov A. I. Representative Theory of Measurement and Its Applications / Zavod. Lab. Diagn. Mater. 1999. Vol. 65. N 3. P. 57 – 62 [in Russian].

50. Barskii B. V., Sokolov M. V. Medial values, invariant relative to permissible transformations of measuring scale / Zavod. Lab. Diagn. Mater. 2006. Vol. 72. N 1. P. 59 – 66 [in Russian].

51. Orlov A. I. Mathematical testing methods and theory of measurings / Zavod. Lab. Diagn. Mater. 2006. Vol. 72. N 1. P. 67 – 70 [in Russian].

52. Kemeni G., Snell G. Cybernetic modeling: Some applications. — Moscow: Sovetskoe radio, 1972. — 192 p. [in Russian].

53. Zhukov M. S., Orlov A. I. Using expert rankings in calculating credit risk in a bank / Innovats. Menedzhm. 2017. N 1. P. 18 – 25 [in Russian].

54. Zhukov M. S., Orlov A. I. The problem of research of final ranking for group of experts by means of Kemeny median / Nauch. Zh. KubGAU. 2016. N 122. P. 785 – 806 [in Russian].

55. Zhukov M. S., Orlov A. I., Fal’ko S. G. Expert assessment in risks / Kontrolling. 2017. N 4(66). P. 24 – 27 [in Russian].

56. Orlov A. I. Average values and rules of large numbers in the spaces of arbitrary origin / Nauch. Zh. KubGAU. 2013. N 89. P. 556 – 586 [in Russian].

57. Orlov A. I. On mean values / Large systems management. Vol. 46. — Moscow: IPU RAN, 2013. P. 88 – 117 [in Russian].

58. Orlov A. I. The limit theory of the solutions of extremal statistical problems / Nauch. Zh. KubGAU. 2017. N 133. P. 579 – 600 [in Russian].

59. Orlov A. I. Methods for finding the most informative sets of characteristics in regression analysis / Zavod. Lab. Diagn. Mater. 1995. Vol. 61. N 1. P. 56 – 58 [in Russian].

60. Orlov A. I. Asymptotic behavior of solutions of extremal statistical problems / Zavod. Lab. Diagn. Mater. 1996. Vol. 62. N 10. P. 45 – 46 [in Russian].

61. Tyrsin A. N. Robust Construction of Linear Regressive Models by Experimental Data / Zavod. Lab. Diagn. Mater. 2005. Vol. 71. N 11. P. 53 – 58 [in Russian].

62. Prokhorov Yu. V., Ed. Probability and Mathematical Statistics: Encyclopedia. — Moscow: Bol’shaya Rossiiskaya Éntsiklopediya, 1999. — 910 p. [in Russian].

63. Orlov A. I. Density estimates in spaces of arbitrary nature / Statistical methods for evaluating and testing hypotheses: an inter-university collection of scientific papers. Vol. 25. — Perm: Perm. Gos. Nats. Issl. Univ., 2013. P. 21 – 33 [in Russian].

64. Orlov A. I. Estimates of probability density function in spaces of arbitrary nature / Nauch. Zh. KubGAU. 2014. N 99. P. 15 – 32 [in Russian].

65. Orlov A. I. Nuclear density estimates in spaces of arbitrary nature / Statistical methods for evaluating and testing hypotheses: an inter-university collection of scientific papers. Vol. 26. — Perm: Perm. Gos. Nats. Issl. Univ., 2015. P. 43 – 57 [in Russian].

66. Orlov A. I. Limit theorems for kernel density estimators in spaces of arbitrary / Nauch. Zh. KubGAU. 2015. N 108. P. 316 – 333 [in Russian].

67. Orlov A. I. Nonparametric kernel estimators of probability density in the discrete spaces / Nauch. Zh. KubGAU. 2016. N 122. P. 833 – 855 [in Russian].

68. Orlov A. I. Nuclear density estimates in finite spaces / Statistical methods for evaluating and testing hypotheses: an inter-university collection of scientific papers. Vol. 27. — Perm: Perm. Gos. Nats. Issl. Univ., 2016. P. 24 – 37 [in Russian].

69. Orlov A. I. Statistics of Non-Numeric Objects / Teor. Veroyatn. Primen. 1980. Vol. XXV. N 3. P. 655 – 656 [in Russian].

70. Orlov A. I. Nonparametric density estimates in topological spaces / Applied statistics. Scientists notes on statistics. Vol. 45. — Moscow: Nauka, 1983. P. 12 – 40 [in Russian].

71. Rosenblatt M. Remarks on some nonparametric estimates of a density function / Ann. Math. Statist. 1956. Vol. 27. N 5. P. 832 – 837.

72. Parzen E. On estimation of a probability density function and mode / Ann. Math. Statist. 1962. Vol. 33. N 6. P. 1065 – 1076.

73. Orlov A. I. Asymptotics of estimates of probability density function / Nauch. Zh. KubGAU. 2017. N 131. P. 845 – 873 [in Russian].

74. Orlov A. I. The rate of convergence of kernel density estimates in spaces of arbitrary nature / Statistical methods for evaluating and testing hypotheses: an inter-university collection of scientific papers. Vol. 28. — Perm: Perm. Gos. Nats. Issl. Univ., 2018. P. 35 – 45 [in Russian].

75. Ibragimov I. A., Khas’minskii R. Z. Asymptotic estimation theory. — Moscow: Nauka, 1979. — 528 p. [in Russian].

76. Orlov A. I. On the Development of Mathematical Methods in the Theory of Classification (review) / Zavod. Lab. Diagn. Mater. 2009. Vol. 75. N 7. P. 51 – 63 [in Russian].

77. Orlov A. I., Tolcheev V. O. On the Use of Nonparametric Statistical Tests to Assess the Accuracy of Classification Methods (Summarizing Article) / Zavod. Lab. Diagn. Mater. 2011. Vol. 77. N 3. P. 58 – 66 [in Russian].

78. Orlov A. I. Three Main Results of the Mathematical Theory of Classification / Zavod. Lab. Diagn. Mater. 2016. Vol. 82. N 5. P. 63 – 70 [in Russian].

79. Perekrest V. T. Nonlinear typological analysis of socio-economic information: Mathematical and computational methods. — Leningrad: Nauka, 1983. — 176 p. [in Russian].

80. Terekhina A. Yu. Data analysis using multidimensional scaling methods. — Moscow: Nauka, 1986. — 168 p. [in Russian].

81. Lagutin M. B. Visual representation of tight connections / Zavod. Lab. Diagn. Mater. 2005. Vol. 71. N 7. P. 53 – 57 [in Russian].

82. Lutsenko E. V., Orlov A. I. Methods of reducing space dimension of statistical data / Nauch. Zh. KubGAU. 2016. N 119. P. 92 – 107 [in Russian].

83. Orlov A. I. Mathematical methods of research and diagnosis materials (generalizing paper) / Zavod. Lab. Diagn. Mater. 2003. Vol. 69. N 3. P. 53 – 64 [in Russian].

84. Orlov A. I. Predictive power — the best indicator of the quality of the diagnostic algorithm / Nauch. Zh. KubGAU. 2014. N 99. P. 33 – 49 [in Russian].

85. Orlov A. I. Stability of Classification Regarding the Choice of the Method of Cluster Analysis / Zavod. Lab. Diagn. Mater. 2013. Vol. 79. N 1. P. 68 – 71 [in Russian].

86. Orlov A. I. Asymptotic behavior of integral type statistics / Dokl. AN SSSR. 1974. Vol. 219. N 4. P. 808 – 811 [in Russian].

87. Orlov A. I. About the testing of homogeneity for two independent samples / Zavod. Lab. Diagn. Mater. 2003. Vol. 69. N 1. P. 55 – 60 [in Russian].

88. Orlov A. I. Methods for testing the homogeneity of the associated samples / Zavod. Lab. Diagn. Mater. 2004. Vol. 70. N 7. P. 57 – 61 [in Russian].

89. Orlov A. I. Asymptotic behavior of integral type statistics / Probabilistic processes and their applications. Interuniversity collection of scientific papers. — Moscow: MIEHM, 1989. P. 118 – 123 [in Russian].

90. Orlov A. I. Limit Theory of Nonparametric Statistics / Nauch. Zh. KubGAU. 2014. N 100. P. 31 – 52 [in Russian].

91. Orlov A. I. Organizational-economic modeling: Textbook. Part 3: Statistical data analysis methods. — Moscow: Izd. MGTU im. N. Й. Baumana, 2012. — 624 p. [in Russian].

92. Nalimov V. V. The Application of Mathematical Statistics to Chemical Analysis. — Moscow: Fizmatgiz, 1960. — 430 p. [in Russian].

93. Novitskii P. V., Zograf I. A. Estimation of measurement errors. — Leningrad: Énergoatomizdat, 1985. — 248 p. [in Russian].

94. Orlov A. I. How often the distribution of the results of observations is normal? / Zavod. Lab. 1991. Vol. 57. N 7. P. 64 – 66 [in Russian].

95. Orlov A. I. Distributions of real statistical data are not normal / Nauch. Zh. KubGAU. 2016. N 117. P. 71 – 90 [in Russian].

96. Seleznev V. D., Denisov K. P. Study of the properties of consent criterion of data distribution function with the Gaussian by Monte-Carlo method for small samples / Zavod. Lab. Diagn. Mater. 2005. Vol. 71. N 1. P. 68 – 73 [in Russian].

97. Orlov A. I. Nonparametric point and interval estimation of the characteristics of distribution/ Zavod. Lab. Diagn. Mater. 2004. Vol. 70. N 5. P. 65 – 70 [in Russian].

98. Orlov A. I. Statistical hypothesis testing of homogeneity of mathematical expectations of two independent samples: the Cramer – Welch instead of the Student’s test / Nauch. Zh. KubGAU. 2015. N 110. P. 197 – 218 [in Russian].

99. Orlov A. I. Two-sample Wilcoxon test — analysis of two myths / Nauch. Zh. KubGAU. 2014. N 104. P. 91 – 111 [in Russian].

100. Orlov A. I. Consistent Tests of Absolute Homogeneity of Independent Samplings / Zavod. Lab. Diagn. Mater. 2012. Vol. 78. N 11. P. 66 – 70 [in Russian].

101. Orlov A. I. Structure of Nonparametric Statistics (generalizing paper) / Zavod. Lab. Diagn. Mater. 2015. Vol. 81. N 7. P. 62 – 72 [in Russian].

102. Orlov A. I. Current status of nonparametric statistics / Nauch. Zh. KubGAU. 2015. N 106. P. 239 – 269 [in Russian].

103. Orlov A. I. Real and nominal significance levels in statistical hypothesis testing / Nauch. Zh. KubGAU. 2015. N 114. P. 42 – 54 [in Russian].

104. Orlov A. I. The Model of Coincidence Analysis in the Calculation of Nonparametric Rank Statistics / Zavod. Lab. Diagn. Mater. 2017. Vol. 83. N 11. P. 66 – 72 [in Russian].

105. Murav’eva V. S., Orlov A. I. Nonparametric estimation of the point of intersection of the regression lines / Zavod. Lab. Diagn. Mater. 2008. Vol. 74. N 1. P. 63 – 68 [in Russian].

106. Orlov A. I. Nonparametric Method of Least Squares with the Periodic Component (generalizing article) / Zavod. Lab. Diagn. Mater. 2014. Vol. 80. N 1. P. 65 – 75 [in Russian].

107. Orlov A. I. Statistics of fuzzy data / Nauch. Zh. KubGAU. 2016. N 119. P. 75 – 91 [in Russian].

108. Orlov A. I. Theory of Lusians / Nauch. Zh. KubGAU. 2014. N 101. P. 275 – 304 [in Russian].

109. Orlov A. I. Distances in the spaces of statistical data / Nauch. Zh. KubGAU. 2014. N 101. P. 227 – 252 [in Russian].

110. Orlov A. I. Statistical control of two alternative variables and a method for verifying their independence from a set of small samples / Zavod. Lab. Diagn. Mater. 2000. Vol. 66. N 1. P. 58 – 62 [in Russian].

111. Orlov A. I. Main problems of controlling of the quality / Nauch. Zh. KubGAU. 2015. N 111. P. 20 – 52 [in Russian].

112. Orlov A. I. Limit theorems in statistical control / Nauch. Zh. KubGAU. 2016. N 116. P. 462 – 483 [in Russian].

113. Orlov A. I. Organizational-economic modeling: Textbook. Part 2: Expert estimators. — Moscow: Izd. MGTU im. N. Й. Baumana, 2009. — 486 p. [in Russian].

114. Statistical methods for analyzing expert estimators / Scientists notes on statistics. Vol. 29. — Moscow: Nauka, 1977. — 385 p. [in Russian].

115. Expert estimators / Voprosy kibernetiki. — Vyp. 58. — Moscow: Nauch. Sovet AN SSSR po kompleksnoi probleme «Kibernetika», 1979. — 200 p. [in Russian].

116. Expert estimates in system studies. Collection of works. Vol. 4. — Moscow: VNIISI, 1979. — 120 p. [in Russian].

117. Expert estimates in management. Collection of works. — Moscow: Inst. Probl. Upravl., 1982. — 106 p. [in Russian].

118. Orlov A. I. Expert estimation / Zavod. Lab. Diagn. Mater. 1996. Vol. 62. N 1. P. 54 – 60 [in Russian].

119. Orlov A. I. On the Development of Expert Technology in Our Country (review) / Zavod. Lab. Diagn. Mater. 2010. Vol. 76. N 11. P. 64 – 70 [in Russian].

120. Orlov A. I. Theory of expert estimates in our country / Nauch. Zh. KubGAU. 2013. N 93. P. 1 – 11 [in Russian].

121. Gnedenko B. V., Orlov A. I. The role of mathematical research methods in the fundamental acceleration of scientific and technological progress / Zavod. Lab. 1988. Vol. 54. N 1. P. 1 – 4 [in Russian].

122. Orlov A. I. On modern problems of implementing applied statistics and other statistical methods / Zavod. Lab. 1992. Vol. 58. N 1. P. 67 – 74 [in Russian].

123. Orlov A. I. Certification and statistical methods (generalizing paper) / Zavod. Lab. Diagn. Mater. 1997. Vol. 63. N 3. P. 55 – 62 [in Russian].

124. Gorskii V. G., Orlov A. I. Mathematical methods of research: results and prospects / Zavod. Lab. Diagn. Mater. 2002. Vol. 68. N 1. P. 108 – 112 [in Russian].

125. Orlov A. I. High statistical technologies / Zavod. Lab. Diagn. Mater. 2003. Vol. 69. N 11. P. 55 – 60 [in Russian].

126. Orlov A. I. About high statistical technologies / Nauch. Zh. KubGAU. 2015. N 105. P. 14 – 38 [in Russian].

127. Orlov A. I. «Six sigmas» — new system of introduction of mathematical research methods / Zavod. Lab. Diagn. Mater. 2006. Vol. 72. N 5. P. 50 – 53 [in Russian].

128. Orlov A. I. Mathematical software certification: a comparative analysis of dialogue systems for statistical control / Zavod. Lab. Diagn. Mater. 1996. Vol. 62. N 7. P. 46 – 49 [in Russian].

129. Smirnova O. S. Software for Statistic Analysis / Zavod. Lab. Diagn. Mater. 2008. Vol. 74. N 5. P. 68 – 75 [in Russian].

130. Orlov A. I. Statistic Packages: Tools of the Researcher / Zavod. Lab. Diagn. Mater. 2008. Vol. 74. N 5. P. 76 – 78 [in Russian].

131. Zhukov M. S. On the Algorithms for Kemeny Median Calculation / Zavod. Lab. Diagn. Mater. 2017. Vol. 83. N 7. P. 72 – 78 [in Russian].

132. Zagoruiko N. G., Orlov A. I. Some unsolved mathematical problems of applied statistics / Modern problems of cybernetics (applied statistics). — Moscow: Znanie, 1981. P. 53 – 63 [in Russian].

133. Orlov A. I. Some outstanding problems in the field of mathematical research methods / Zavod. Lab. Diagn. Mater. 2002. Vol. 68. N 3. P. 52 – 56 [in Russian].

134. Orlov A. I. Limit Theorems and Monte Carlo Method / Zavod. Lab. Diagn. Mater. 2016. Vol. 82. N 7. P. 67 – 72 [in Russian].

135. Grigor’ev Yu. D. Monte Carlo Method: Accuracy of Asymptotic Solutions and Quality of Pseudorandom Number Generators / Zavod. Lab. Diagn. Mater. 2016. Vol. 82. N 7. P. 72 – 84 [in Russian].

136. Nikitin Ya. Yu. Asymptotic efficiency of nonparametric criteria. — Moscow: Nauka, 1995. — 240 p. [in Russian].

137. Orlov A. I. The problem of multiple tests of statistical hypotheses / Zavod. Lab. Diagn. Mater. 1996. Vol. 62. N 5. P. 51 – 54 [in Russian].

138. Orlov A. I. About the methodology of statistical methods / Nauch. Zh. KubGAU. 2014. N 104. P. 53 – 80 [in Russian].