其他知识库

天文观测数据是天文研究的基础,但传统的集中式数据检索方法已难以满足日益增长的海量天文数据的高性能检索和查询需求.提出了一种基于Elastic Search分布式搜索引擎,通过River机制对现有的海量FITS(Flexible Image Transport System)数据进行索引构建,从而实现海量FITS数据高效检索的方法,并讨论了其中的近实时检索和查询的关键技术.实测结果表明,在百万到千万级的天文数据量下,该方法可获得极高的检索性能,并能够很方便地集成到现有的天文数据归档系统中,完全可以满足当前国内各类望远镜系统天文数据的归档要求.

Astronomical observational data are the fundamental element for modern astronomical researches. However, with the rapid increase of astronomical data, the traditional centralized retrieval methods are hard to meet the requirements of highperformance data retrieval. In the study, we present a novel method which is based on the Elastic Search distributed retrieval engine and River mechanism to create data indexes, and provide high performance data retrieval for massive FITS（Flexible Image Transport System） data. We discuss the key technologies of the nearly real-time retrieval and query. The experimental results show that the method is capable of obtaining high retrieval performance especially for the cases in which the number of the FITS data exceeds millions or even tens of millions. Meanwhile, the method can be easily integrated into the current astronomical data archiving systems, and completely meet the archive requirements of all kinds of astronomical telescope systems.