其他知识库

中国明安图超宽频谱射电日像仪(Mingantu Ultrawide Spectral Radioheliograph,MUSER)进入实际观测后,每3 ms产生一帧100 k B左右的数据,每天的原始观测数据约3.5 TB。由于射电日像仪的原始数据采用自定义格式,为了后续数据分析和共享的需要,有必要根据数据存储需求把这些原始数据转换成天文常用的文件格式。在前期工作中已经实现了原始数据格式到UVFITS文件的转换,在此基础上研究了基于MPI的集群并行环境下UVFITS合成系统的性能优化。通过实验验证,在改进后的并行环境下,UVFITS合成系统的性能达到了需求的2.5倍,可以有效处理当前及未来一定时间内射电日像仪的海量观测数据。同时,改进后的系统具有良好的横向扩展能力,能够为相关项目的数据处理提供借鉴和参考。

Mingantu Ultrawide Spectral Radioheliograph (MUSER) generates 100 kilobytes raw observational data in every 3 milliseconds and more than 3. 5 Terabytes data per day. For further data analysis and sharing, it is necessary to convert the raw data stored in self-defined format to standard format usually used in the field of radio astronomy. In previous work,we have analyzed the format of UVFITS and converted the raw data to UVFITS file successfully. However, the efficiency of the format converting system needs to be further improved. This paper presents a parallel UVFITS file assembly system based on cluster parallel environment. Experiments show that the system can reduce the execution time of assembling a UVFITS file to about 1.2 milliseconds, 2.5 times faster than that of the data acquisition, which is very promising to meet the data processing requirements in the project. Moreover, the parallel system can be used for reference in other systems. The implementation of this parallel data format converting system can also provide a good reference to similar data processing systems.