Development and application of Computer Vision-based algorithm for digitizing archival monitoring observations of powerful cosmic radio source fluxes on the RT URAN-4 (IRA NASU)
Рубрика:
| 1Zabora, DA, 2Ryabov, MI, Sukharev, AI, 3Strakhov, Ye.M, 4Shymbarova, OO 1Institute of Radio Astronomy of the National Academy of Sciences of Ukraine, Kharkiv, Ukraine; Odesa I. I. Mechnikov National University, Odesa, 65082 Ukraine 2Institute of Radio Astronomy of the National Academy of Sciences of Ukraine, Kharkiv, Ukraine 3Odesa I. I. Mechnikov National University, Odesa, 65082 Ukraine 4Odesa I. I. Mechnikov National University, Odesa, 65082 Ukraine |
| Space Sci. & Technol. 2025, 31 ;(6):123-133 |
| https://doi.org/10.15407/knit2025.06.123 |
| Язык публикации: English |
Аннотация: This study presents the development and preliminary application of a computer vision-based algorithm for digitizing archival paperbased
records (1987—1992) from the URAN-4 radio telescope. The input data consists of JPEG scans at 300 dpi, with the quality of the digitized results primarily depending on the quality of the original paper medium. The developed algorithm encompasses several key stages: identification of the recorder’s curve and grid lines of medium by color; filtering and clustering of grid lines; determination and correction of scan tilt introduced by imperfect paper medium placement in the scanner; stitching together multiple scans of a single extended record by detecting and matching time markers on adjacent scans; establishing vertical (intensity) and horizontal (time) scales; filtering and removing noise components from the signal for subsequent digital processing. The research also demonstrates the feasibility and quality of subsequent time-frequency analysis of the obtained signal. This approach to digitization enables the application of modern digital processing methods to historical monitoring records, allowing for acurcomprehensive analysis of the frequency characteristics of radio flux variability in time intervals of approximately 1 second and, inparticular, for detailed investigations of ionospheric scintillations. The proposed stitching algorithm facilitates the construction of long-term time series and the digital identification of significant events, offering a new perspective on space weather conditions during historical ionospheric and magnetic storms, the effects of solar and lunar eclipses, and ionospheric tidal lensing. The initial application of this algorithm will focus on analyzing the impact of ionospheric and magnetic storms within the Odesa Magnetic Anomaly zone during the 22nd solar activity cycle. |
| Ключевые слова: archival radio telescope data digitization, automated chart recognition, computer vision, ionospheric scintillation, machine learning, radio astronomy, time-frequency analysis |
References:
1. Bradski G. (2000). The OpenCV Library. Dr. Dobb’s J. Software Tools, 25(1), 120—126.
2. Ester M., Kriegel H. P., Sander J., Xu X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proc. Second Int. Conf. on Knowledge Discovery and Data Mining (KDD’96). AAAI Press, 226—231. https://doi.
org/10.5555/3001460.3001507
3. Galanin V. V., Inyutin G. A., Kvasha I. M., Panishko S. K., Pisarenko Ya. V., Rashkovskij S. L., Ryabov M. I., Serokurova N. G., Tsesevich V. P., Sharykin N. K. (1989). URAN-4 radio telescope as an element of VLBI system. Kinematics and Physics
of Celestial Bodies, 5(5), 87—90.
4. Grossmann A., Morlet J. (1984). Decomposition of Hardy Functions into Square Integrable Wavelets of Constant Shape SIAM J. Mathematical Analysis, 15(4), 723—736. https://doi.org/10.1137/0515056
5. Halko N., Martinsson P.-G., Tropp J. A. (2009). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. e-print. https://doi.org/10.48550/arXiv.0909.4061
6. Harris C. R., Millman K. J., van der Walt S. J., et al. (2020). Array programming with NumPy. Nature, 585, 357—362.
https://doi.org/10.1038/s41586-020-2649-2
7. Huang T., Yang G., Tang G. (1979). A fast two-dimensional median filtering algorithm. IEEE Trans. Acoust., Speech. Signal Process., 27(1), 13—18. https://doi.org/10.1109/TASSP.1979.1163188
8. Hunter J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering (CiSE), 9(3), 90—95. https://doi.org/10.1109/MCSE.2007.55
9. Lee G. R., Gommers R., Wasilewski F., Wohlfahrt K., O’Leary A. (2019). PyWavelets: A Python package for wavelet analysis. J. Open Source Software (JOSS), 4(36), 1237. https://doi.org/10.21105/joss.01237
10. OpenCV team, OpenCV. (2025). OpenCV modules. URL: https://docs.opencv.org/4.12.0/ (Last accessed: 16.10.2025).
11. Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg
V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay É. (2011). Scikit-learn: Machine Learning in Python. J. Machine Learning Res., 12, 2825—2830.
12. Ryabov M. I., Panishko S. K., Guglya L. I. (2011). Space weather impacts on the Earth’s upper atmosphere according to the monitoring of powerful radio sources fluxes at the URAN-4 radio telescope (Institute of Radio Astronomy, NAS of
Ukraine). Odessa Astron. Publ., 24, 159—161. https://doi.org/10.18524/1810-4215.2011.24.85118
13. Ryabov M. I., Sobitnyak L. I. (2019). Study of effects of lunar tidal wave passage in upper atmosphere of Earth according to monitoring data at radio telescope «URAN-4» RI NANU. Odessa Astron. Publ., 32, 132—136. https://doi.org/10.18524/1810-
4215.2019.32.182539
14. Ryabov M. I., Sobitniak L. I. (2022). Tidal phenomena in the Earth’s upper atmosphere. Astron. and Astrophys. Trans., 33(1), 37—44. https://doi.org/10.17184/eac.6468
15. Savitzky A., Golay M. J. E. (1964). Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Anal. Chem., 36(8), 1627—1639. https://doi.org/10.1021/ac60214a047
16. Schubert E., Sander J., Ester M., Kriegel H. P., Xu X. (2017). DBSCAN Revisited, Revisited: Why and How You Should (Still) Use DBSCAN. ACM Transactions on Database Systems (TODS), 42(3), 1—21. https://doi.org/10.1145/3068335
17. scikit-learn developers, scikit-learn. (2025). API Reference. URL: https://scikit-learn.org/stable/api/index.html (Last accessed: 16.10.2025).
18. scikit-learn developers, scikit-learn. (2025). User Guide. URL: https://scikit-learn.org/stable/user_guide.html (Las accessed: 16.10.2025).
19. Sculley D. (2010). Web-scale k-means clustering. Proc. 19th Int. conf. on World wide web, WWW ’10, 1177—1178. https://doi.org/10.1145/1772690.1772862
20. Serra J. (1982). Image Analysis and Mathematical Morphology. London: Acad. Press, 610 p.
21. Sobitnyak L. I., Ryabov M. I., Orlyuk M. I., Sukharev A. L., Romenets A. O., Sumaruk Yu. P., Pilipenko A. A. (2020). Analysis of the Magnetic Storms Catalog for Monitoring Radio Source Fluxe Data with the URAN-4 Radio Telescope in
the Odesa Magnetic Anomaly Zone. Radio phys. and radio astron., 25(4), 324—330. https://doi.org/10.15407/rpra25.04.324
22. The pandas development team, Pandas. (2025). pandas-dev/pandas: Pandas (v2.3.3). https://doi.org/10.5281/zenodo.3509134
23. The PyWavelets Developers, PyWavelets. (2025). Continuous Wavelet Transform (CWT), PyWavelets Documentation. URL: https://pywavelets.readthedocs.io/en/latest/ref/cwt.html (Last accessed: 16.10.2025).
24. The SciPy community, SciPy. (2025). SciPy API. URL: https://docs.scipy.org/doc/scipy/reference/index.html (Last accessed: 16.10.2025).
25. The SciPy community, SciPy. (2025). SciPy User Guide. URL: https://docs.scipy.org/doc/scipy/tutorial/index.html (Last accessed: 16.10.2025).
26. Tukey J. W. (1977). Exploratory Data Analysis. Boston: Addison-Wesley Publ. Company, 688 p.
27. Virtanen P., Gommers R., Oliphant T. E., Haberland M., Reddy T., Cournapeau D., et al. (2020). SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261—272. https://doi.
org/10.1038/s41592-019-0686-2
2. Ester M., Kriegel H. P., Sander J., Xu X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proc. Second Int. Conf. on Knowledge Discovery and Data Mining (KDD’96). AAAI Press, 226—231. https://doi.
org/10.5555/3001460.3001507
3. Galanin V. V., Inyutin G. A., Kvasha I. M., Panishko S. K., Pisarenko Ya. V., Rashkovskij S. L., Ryabov M. I., Serokurova N. G., Tsesevich V. P., Sharykin N. K. (1989). URAN-4 radio telescope as an element of VLBI system. Kinematics and Physics
of Celestial Bodies, 5(5), 87—90.
4. Grossmann A., Morlet J. (1984). Decomposition of Hardy Functions into Square Integrable Wavelets of Constant Shape SIAM J. Mathematical Analysis, 15(4), 723—736. https://doi.org/10.1137/0515056
5. Halko N., Martinsson P.-G., Tropp J. A. (2009). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. e-print. https://doi.org/10.48550/arXiv.0909.4061
6. Harris C. R., Millman K. J., van der Walt S. J., et al. (2020). Array programming with NumPy. Nature, 585, 357—362.
https://doi.org/10.1038/s41586-020-2649-2
7. Huang T., Yang G., Tang G. (1979). A fast two-dimensional median filtering algorithm. IEEE Trans. Acoust., Speech. Signal Process., 27(1), 13—18. https://doi.org/10.1109/TASSP.1979.1163188
8. Hunter J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering (CiSE), 9(3), 90—95. https://doi.org/10.1109/MCSE.2007.55
9. Lee G. R., Gommers R., Wasilewski F., Wohlfahrt K., O’Leary A. (2019). PyWavelets: A Python package for wavelet analysis. J. Open Source Software (JOSS), 4(36), 1237. https://doi.org/10.21105/joss.01237
10. OpenCV team, OpenCV. (2025). OpenCV modules. URL: https://docs.opencv.org/4.12.0/ (Last accessed: 16.10.2025).
11. Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg
V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay É. (2011). Scikit-learn: Machine Learning in Python. J. Machine Learning Res., 12, 2825—2830.
12. Ryabov M. I., Panishko S. K., Guglya L. I. (2011). Space weather impacts on the Earth’s upper atmosphere according to the monitoring of powerful radio sources fluxes at the URAN-4 radio telescope (Institute of Radio Astronomy, NAS of
Ukraine). Odessa Astron. Publ., 24, 159—161. https://doi.org/10.18524/1810-4215.2011.24.85118
13. Ryabov M. I., Sobitnyak L. I. (2019). Study of effects of lunar tidal wave passage in upper atmosphere of Earth according to monitoring data at radio telescope «URAN-4» RI NANU. Odessa Astron. Publ., 32, 132—136. https://doi.org/10.18524/1810-
4215.2019.32.182539
14. Ryabov M. I., Sobitniak L. I. (2022). Tidal phenomena in the Earth’s upper atmosphere. Astron. and Astrophys. Trans., 33(1), 37—44. https://doi.org/10.17184/eac.6468
15. Savitzky A., Golay M. J. E. (1964). Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Anal. Chem., 36(8), 1627—1639. https://doi.org/10.1021/ac60214a047
16. Schubert E., Sander J., Ester M., Kriegel H. P., Xu X. (2017). DBSCAN Revisited, Revisited: Why and How You Should (Still) Use DBSCAN. ACM Transactions on Database Systems (TODS), 42(3), 1—21. https://doi.org/10.1145/3068335
17. scikit-learn developers, scikit-learn. (2025). API Reference. URL: https://scikit-learn.org/stable/api/index.html (Last accessed: 16.10.2025).
18. scikit-learn developers, scikit-learn. (2025). User Guide. URL: https://scikit-learn.org/stable/user_guide.html (Las accessed: 16.10.2025).
19. Sculley D. (2010). Web-scale k-means clustering. Proc. 19th Int. conf. on World wide web, WWW ’10, 1177—1178. https://doi.org/10.1145/1772690.1772862
20. Serra J. (1982). Image Analysis and Mathematical Morphology. London: Acad. Press, 610 p.
21. Sobitnyak L. I., Ryabov M. I., Orlyuk M. I., Sukharev A. L., Romenets A. O., Sumaruk Yu. P., Pilipenko A. A. (2020). Analysis of the Magnetic Storms Catalog for Monitoring Radio Source Fluxe Data with the URAN-4 Radio Telescope in
the Odesa Magnetic Anomaly Zone. Radio phys. and radio astron., 25(4), 324—330. https://doi.org/10.15407/rpra25.04.324
22. The pandas development team, Pandas. (2025). pandas-dev/pandas: Pandas (v2.3.3). https://doi.org/10.5281/zenodo.3509134
23. The PyWavelets Developers, PyWavelets. (2025). Continuous Wavelet Transform (CWT), PyWavelets Documentation. URL: https://pywavelets.readthedocs.io/en/latest/ref/cwt.html (Last accessed: 16.10.2025).
24. The SciPy community, SciPy. (2025). SciPy API. URL: https://docs.scipy.org/doc/scipy/reference/index.html (Last accessed: 16.10.2025).
25. The SciPy community, SciPy. (2025). SciPy User Guide. URL: https://docs.scipy.org/doc/scipy/tutorial/index.html (Last accessed: 16.10.2025).
26. Tukey J. W. (1977). Exploratory Data Analysis. Boston: Addison-Wesley Publ. Company, 688 p.
27. Virtanen P., Gommers R., Oliphant T. E., Haberland M., Reddy T., Cournapeau D., et al. (2020). SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261—272. https://doi.
org/10.1038/s41592-019-0686-2