The architecture of a multi-tiered virtual observatory (original) (raw)
Related papers
Intelligent Archive Visionary Use Case: Virtual Observatories
2000
Astronomy in the next twenty years will be powered by diverse new observing capabilities. Huge gains in instrument sensitivity covering a broad range of wavelengths for ground and space-based observatories promise abundant data for making unique discoveries, finding solutions to fundamental astrophysical problems, and expanding the frontiers of astrophysical knowledge [1]. These anticipated advances in astronomy will also depend largely on new capabilities for archiving, discovering, accessing, analyzing, and using observational data. Advances in data processing infrastructures will be required to transform accumulated and expected volumes of data into information and knowledge.
Open SkyQuery--VO Compliant Dynamic Federation of Astronomical Archives
2004
Abstract. We discuss the redesign of the SkyQuery architecture, originally built as a simple proof of concept for dynamic federation of astronomical archives. In keeping with the Virtual Observatory philosophy of hierarchical services, the design of Open SkyQuery is based upon higher level services extending the basic functionality of the current VO standard, the ConeSearch. Open SkyQuery implements the VO specifications for data access, retrieval and spatial join.
Data and Metadata Management at the Keck Observatory Archive
2015
A collaboration between the W. M. Keck Observatory (WMKO) in Hawaii and the NASA Exoplanet Science Institute (NExScI) in California, the Keck Observatory Archive (KOA) was commissioned in 2004 to archive data from WMKO, which operates two classically scheduled 10 m ground-based telescopes. The data from Keck are not suitable for direct ingestion into the archive since the metadata contained in the original FITS headers lack the information necessary for proper archiving. The data pose a number of challenges for KOA: different instrument builders used different standards, and the nature of classical observing, where observers have complete control of the instruments and their observations, lead to heterogeneous data sets. For example, it is often difficult to determine if an observation is a science target, a sky frame, or a sky flat. It is also necessary to assign the data to the correct owners and observing programs, which can be a challenge for time-domain and target-of-opportunit...
Interoperability tools for the Virtual Observatory
Astronomical Data Analysis, 2001
Interoperability is one of the key issues in the current efforts to build the Virtual Observatory. We present here some of the tools which already contribute to the efficient exchange of information between archives, databases, and journals.
2006
We present a summary of the major contributions to the Special Session on Data Management held at the IAU General Assembly in Prague in 2006. While recent years have seen enormous improvements in access to astronomical data, and the Virtual Observatory aims to provide astronomers with seamless access to on-line resources, more attention needs to be paid to ensuring the quality and completeness of those resources. For example, data produced by telescopes are not always made available to the astronomical community, and new instruments are sometimes designed and built with insufficient planning for data management, while older but valuable legacy data often remain undigitised. Data and results published in journals do not always appear in the data centres, and astronomers in developing countries sometimes have inadequate access to on-line resources. To address these issues, an 'Astronomers Data Manifesto' has been formulated with the aim of initiating a discussion that will lead to the development of a 'code of best practice' in astronomical data management.
Communications of the Byurakan Astrophysical Observatory, 2020
The Virtual Observatory (VO) represents a successful international enterprise providing interoperability of data collections, thus allowing the possibility of multi-frequency and multi-messenger research. The Big Data era, that astrophysics has stepped into, is forcing scientists to perform data-intensive research. This new concept requires an evolution of the VO concept to provide additional services, in order to transform the VO: from sets of interoperable data to an integrated system of services capable of supporting data-intensive science.
Planetary Science Virtual Observatory architecture
Astronomy and Computing, 2014
(2) , J. Berthier (3) , F. Henry (1) , C. Chauvin (2) , N. André (4) , V. Génot (4) , C. Jacquey (4) , M. Gangloff (4) , N. Bourrel (4) , B. Schmitt (5) , M. T. Capria (6) , G. Chanteur (7)
Italian center for astronomical archives publishing solution: modular and distributed
Software and Cyberinfrastructure for Astronomy V, 2018
The Italian center for Astronomical Archives has, among its goals, to provide astronomical data resources as interoperable services based on IVOA standards. It did so for part of its archives (mainly raw telescope data from LBT, TNG and Italian national telescopes) and continued on with hosted data collections and providing expertise to national and international research projects (like WINGS, VIPERS, VIALACTEA). Its expertise and knowledge of the VO comes from active participation within IVOA and VO at European and international level, with a double-fold goal: learn from the collaboration (acquiring skills and technical knowledge) and provide inputs (implementations and feedback) to the VO community. In this scenario the first solution to build an easy to configure and maintain resource publisher conformant to VO standards proved to be too optimistic, not considering the complexity the IVOA architecture could have reached in a short while. For this reason it has been necessary to rethink the architecture with a modular system. This latter is now partially in place and will gradually replace the previous solution allowing for an easier to extend and rework if major changes will happen at the level of VO standards. The solution chosen for the architecture orbits around the messaging concept, where each modular component speaks to the other interested parties through a system of broker-managed queues (currently using AMQP with RabbitMQ as the broker). The messaging system lets us free to choose the development language for the business logic components, not only for the front-end, web interfacing solutions (needed to expose VO HTTP based protocols), but also on the archives and database access components, the logging systems and any other tool or component that may be needed in the future. The first implementation covered the simplest VO protocol, the Simple Cone Search, were the messaging task architecture connects the parametric HTTP interface to the database backend access module, the logging module, and allows multiple cone search resources to be managed together through a configuration manager module. Even if it has been initially used as a test for the new architecture, it already proved the flexibility required by the overall system when the database backend needed to be changed from a MySQL to a PostgreSQL+PgSphere solution. Another implementation test has been made to leverage task distribution over multiple servers dedicated to computation to allow for a single HTTP interface to serve simultaneously: FITS cubes direct linking, cubes cutout (using an AST and cfitsio engine) and cubes positional merging (using a Montage based solution). The solution proved also to be a quick answer to load distribution (although not really efficient). Alongside these the implementation of the SIA-2.0 standard protocol is ongoing, following the scheme used for the Simple Cone Search, while for the TAP protocol implementation we will be re-using and adapting the already available TAPlib library. Alongside this production, message-driven, publishers, a first administration tool (TASMAN) has been developed to ease the build up and maintenance of the TAP SCHEMA component of TAP services including also ObsCore maintenance capability. Future work will be devoted at widening the range of VO protocols covered by the set of available modules, improve the configuration management and develop specific purpose modules common to all the service components.
The Virtual Observatory Data, Standards and Tools a technical - user point of view
Proceedings of the International Astronomical Union, 2010
In 2002 the International Virtual Observatory Alliance (IVOA) has been created in order to gather efforts on data standardization and dissemination. Since then, the virtual Observatory allowed to spread validated data all over the world and to use data from everywhere from earth. From the standards definitions to development of tools, developers have set up a technical infrastructure used by astronomers to easily search for data and make science with all available products, more tools and more confidence on the quality of data. The goal of this review is to present the state of the art of the VO data, standards and tools. This review focuses on basic astronomer's questions : what kind of data are accessible, how to deal with these data and how to use them.