Towards the nonparametric estimation of measures of complex dependencies in dynamic systems

Nonparametric methods of estimation in stochastic systems are certainly more general than parametric estimation methods. The latter assume the presence of a priori information, which ultimately assumes the knowledge of the number of estimated parameters and the structure of the system itself. In nonparametric methods, such information is not required. The paper objective: to develop methods for nonparametric estimation of dependence measures for complexly organized random processes. The article introduces a measure of dependence that links k pairs of random processes. Such a measure, based on the use of conditional mathematical expectations of processes, can be considered as a further generalization of dispersion functions. Convergence with probability 1 of nonparametric estimates of such a measure is derived using sample data. These estimates are used to construct sample analogues of some nonlinear measures of stochastic dependence of random processes, in particular, to obtain a consistent measure of dependence in the sense of Kolmogorov, i.e., a measure that vanishes if and only if the given random processes are stochastically independent. As a direct consequence, the consistency of the measure of dependence in the sense of Rényi, i.e., a measure that satisfies the corresponding Rényi axioms, will immediately follow from the obtained results. The developed estimation algorithms converging with probability 1 do not require any a priori information about the system and can be used to construct input-output mappings of nonlinear systems without any special requirements for the system.

1. Risuleo R. S., Bottegal G., Hjalmarsson H. A nonparametric kernel-based approach to Hammerstein system identification / Automatica. 2017. Vol. 85. P. 234 – 247. DOI: 10.1016/j.automatica.2017.07.055

2. Orlov A. I. Asymptotics of the probability distribution / Sci. J. KubSAU. 2017. No. 131. P. 845 – 873 [in Russian].

3. Zhang E., Pintelon R. Nonparametric identification of linear dynamic errors-in-variables systems / Automatica. 2018. Vol. 94. P. 416 – 425. DOI: 10.1016/j.automatica.2018.04.039

4. Darwish M. A. H., Cox P. B., Proimadis I., et al. Prediction-error identification of LPV systems: a nonparametric Gaussian regression approach / Automatica. 2018. Vol. 97. P. 92 – 103. DOI: 10.1016/j.automatica.2018.07.032

5. Tyrsin A. N. Scalar measure of interdependence between random vectors / Zavod. Lab. Mater. Diagn. 2018. Vol. 84. No. 7. P. 76 – 82 [in Russian]. DOI: 10.26896/1028-6861-2018-84-7-76-82

6. Risuleo R. S., Lindsten F., Hjalmarsson H. Bayesian nonparametric identification of Wiener systems / Automatica. 2019. Vol. 108. Art. 108480. DOI: 10.1016/j.automatica.2019.06.032

7. Khosravi M., Smith R. S. Convex nonparametric formulation for identification of gradient flows / IEEE Control Syst. Lett. 2021. Vol. 5. No. 3. P. 1097 – 1102. DOI: 10.1109/lcsys.2020.3000176

8. Wenxiao Zhao, Weyer E., Yin G. A general framework for nonparametric identification of nonlinear stochastic systems / IEEE Trans. Autom. Control. 2021. Vol. 66. No. 6. P. 2449 – 2464. DOI: 10.1109/tac.2020.3007569

9. Changming Cheng, Er-Wei Bai. Variable selection according to goodness of fit in nonparametric nonlinear system identification / IEEE Trans. Autom. Control. 2021. Vol. 66. No. 7. P. 3184 – 3196. DOI: 10.1109/tac.2020.3015744

10. Qing Sun, Siting Zou, Dajun Du, Minrui Fei. Nonparametric identification of linear dynamic output-errors systems / Int. J. Control Autom. Syst. 2022. Vol. 20. P. 3932 – 3939. DOI: 10.1007/s12555-020-0401-1

11. Tyrsin A. N. Scalar measure of interdependence between several random vectors / Zavod. Lab. Mater. Diagn. 2022. Vol. 88. No. 3. P. 73 – 80 [in Russian]. DOI: 10.26896/1028-6861-2022-88-3-73-80

12. Lijie Wang, Zuhua Xu, Jun Zhao, Zhijiang Shao. Nonparametric identification based on Gaussian process regression for distributed parameter systems / Int. J. Syst. Sci. 2023. Vol. 54. No. 6. P. 1229 – 1242. DOI: 10.1080/00207721.2023.2169058

13. Sun B., Cai Q. Y., Peng Z. K., et al. Variable selection and identification of high-dimensional nonparametric nonlinear systems by directional regression / Nonlin. Dyn. 2023. Vol. 111. P. 12101 – 12112. DOI: 10.1007/s11071-023-08488-6

14. Shuyu Wang, Zuhua Xu, Minghao Chen, et al. Two-stage transfer learning-based nonparametric system identification with Gaussian process regression / Comput. Chem. Eng. 2024. Vol. 189. Art. 108799. DOI: 10.1016/j.compchemeng.2024.108799

15. Ye Zhao, Bin Xu, Genda Chen. Nonparametric identification of multi-degree-of-freedom nonlinear systems from partially measured responses under uncertain dynamic excitations / Int. J. Non-Lin. Mech. 2024. Vol. 167. Art. 104903. DOI: 10.1016/j.ijnonlinmec.2024.104903

16. González R. A., van Haren M., Oomen T., Rojas C. R. Sampling in parametric and nonparametric system identification: aliasing, input conditions, and consistency / IEEE Control Syst. Lett. 2024. Vol. 8. P. 2415 – 2420. DOI: 10.1109/lcsys.2024.3487501

17. Rolin J. M. Nonparametric competing risks models: identification and strong consistency / J. P. Florens, M. Mouchart (eds.). Nonparametric Bayesian inference. — Cham: Springer, 2024. P. 219 – 245. DOI: 10.1007/978-3-031-61329-6_10

18. Xiaotao Ren, Wenxiao Zhao, Han Zhang. Recursive identification of nonlinear nonparametric systems under event-triggered observations / Syst. Control Lett. 2025. Vol. 196. Art. 106013. DOI: 10.1016/j.sysconle.2024.106013

19. Rajbman N. S. Extensions to nonlinear and minimax approaches / P. Eykhoff (ed.) Trends and progress in system identification. — Oxford, Pergamon Press. 1981. P. 185 – 237. DOI: 10.1016/B978-0-08-025683-2.50012-0

20. Varlaki P., Seidl L. On the estimation of variance functions of ergodic random processes / Autom. Remote Control. 1983. No. 6. P. 62 – 65 [in Russian].

21. Sarmanov O. V., Zakharov E. K. Measures if dependence between random values and spectra of stochastic kernels and matrices / Matem. Sb. 1960. Vol. 52(94). P. 953 – 990 [in Russian].

22. Rényi A. On measures of dependence / Acta Acad. Sci. Math. Hung. 1959. Vol. 10. Nos. 3 – 4. P. 441 – 451. DOI: 10.1007/bf02024507

23. Ovsepyan F. A., Lepsky D. M. On identification of nonlinear nonparametric control systems / IFAC Proc. Vol. 20. No. 5. Part 10. P. 239 – 243. DOI: 10.1016/s1474-6670(17)55507-x