DART, a New Solution to Deploy and Access Astronomical Data (original) (raw)
Related papers
SkyPortal: An Astronomical Data Platform
Journal of Open Source Software
SkyPortal is a web application that stores and interactively displays astronomical datasets for annotation, analysis, and discovery. It is designed to be modular and extensible, so it can be customized for various scientific use cases. It is released under the Modified BSD license. SkyPortal was designed with time-series survey data from the Zwicky Transient Facility, and eventually The Large Synoptic Survey Telescope, in mind. By default, it aims to provide a useful, rich user experience, including light curves of named astronomical events/transients, spectra, live chat, and links to other surveys. But the intent, ultimately, is for the frontend to be modified to best suit the specific scientific problem at hand. The current UI/UX was inspired by that developed for the Palomar Transient Factory (PTF) Marshal (Law et al., 2009).
Making Access to Astronomical Software More Efficient
2010
Access to astronomical data through archives and VO is essential but does not solve all problems. Availability of appropriate software for analyzing the data is often equally important for the efficiency with which a researcher can publish results. A number of legacy systems (e.g. IRAF, MIDAS, Starlink, AIPS, Gipsy), as well as others now coming online are available but have very different user interfaces and may no longer be fully supported. Users may need multiple systems or stand-alone packages to complete the full analysis which introduces significant overhead. The OPTICON Network on 'Future Astronomical Software Environments' and the USVAO have discussed these issues and have outlined a general architectural concept that solves many of the current problems in accessing software packages. It foresees a layered structure with clear separation of astronomical code and IT infrastructure. By relying on modern IT concepts for messaging and distributed execution, it provides full scalability from desktops to clusters of computers. A generic parameter passing mechanism and common interfaces will offer easy access to a wide range of astronomical software, including legacy packages, through a single scripting language such as Python. A prototype based upon a proposed standard architecture is being developed as a proof-ofconcept. It will be followed by definition of standard interfaces as well as a reference implementation which can be evaluated by the user community. For the long-term success of such an environment, stable interface specifications and adoption by major astronomical institutions as well as a reasonable level of support for the infrastructure are mandatory. Development and maintenance of astronomical packages would follow an open-source, Internet concept.
2006
We present a summary of the major contributions to the Special Session on Data Management held at the IAU General Assembly in Prague in 2006. While recent years have seen enormous improvements in access to astronomical data, and the Virtual Observatory aims to provide astronomers with seamless access to on-line resources, more attention needs to be paid to ensuring the quality and completeness of those resources. For example, data produced by telescopes are not always made available to the astronomical community, and new instruments are sometimes designed and built with insufficient planning for data management, while older but valuable legacy data often remain undigitised. Data and results published in journals do not always appear in the data centres, and astronomers in developing countries sometimes have inadequate access to on-line resources. To address these issues, an 'Astronomers Data Manifesto' has been formulated with the aim of initiating a discussion that will lead to the development of a 'code of best practice' in astronomical data management.
INES: The Next Generation Astronomical Data Distribution System
2001
The IUE Archive was the first astronomical archive to be made accessible on-line, back in 1985, when the World Wide Web didn't even exist. The archive stores more than 110000 spectra which span nearly two decades of Ultraviolet Astronomy. The IUE Newly Extracted Spectra System (INES), a complete astronomical archive and its associated data distribution system, was developed with the goal of delivering IUE data to the scientific community in a simple and efficient form. Data distribution is structured into three levels: a Principal Centre at LAEFF (Laboratory for Space Astronomy and Theoretical Physics, owned by the Spanish National Institute for Aerospace Technology) and its Mirror at CADC, a number of National Hosts (currently 22), and an unlimited number of end users. The INES Principal Centre can be reached at http://ines.vilspa.esa.es.
Enabling Access to Astronomical Databases through the Grid: a Case Study
In recent years great efforts have been spent in the astronomical community to allow a global and seamless electronic access to distributed astronomical data repositories, and to enable scientific analysis on them. A system capable of providing these services over distributed databases and computer resources is called a Virtual Observatory (VO). VOs are still in the study phase, with many projects around the world now delivering their first prototype implementations. In the meanwhile grid technologies have started to emerge and consolidate and are now expected to play a fundamental role in the development of Astrophysical Virtual Observatories. We report here about our activities aimed at integrating in the grid environment a system, developed in Padova, specifically designed for accessing very large astronomical catalogues. We evaluated several possible solutions, including the use of a tool for accessing databases developed by the European Data Grid (EDG) project, but finally we d...
Astronomical database related applications in the Grid. it project
… and Systems XIV, 2005
We describe the activity done in the contest of the Grid.it project to access Astronomical Catalogues and Archives through grid environment. Different approaches are tested: job oriented, web service oriented. The crucial aspect of this work is the development and deployment of an IVOA compliant Data Source Engine model in the grid middleware.
Japanese Virtual Observatory (JVO) as an advanced astronomical research environment
2006
We present the design and implementation of the Japanese Virtual Observatory (JVO) system. JVO is a portal site to various kinds of astronomical resources distributed all over the world. We have developed five components for constructing the portal: (1) registry, (2) data service, (3) workflow system, (4) data analysis service (5) portal GUI. Registry services are used for publishing and searching data services in the VO, and they are constructed using an OAI-PMH metadata harvesting protocol and a SOAP web service protocol so that VO standard architecture is applied. Data services are developed based on the Astronomical Data Query Language (ADQL) which is an international VO standard and an extension of the standard SQL. The toolkit for building the ADQL-based service is released to the public on the JVO web site. The toolkit also provides the protocol translation from a Simple Image Access Protocol (SIAP) to ADQL protocol, so that both the VO standard service can be constructed using our toolkit. In order to federate the distributed databases and analysis services, we have designed a workflow language which is described in XML and developed execution system of the workflow. We have succeeded to connect to a hundred of data resources of the world as of April 2006. We have applied this system to the study of QSO environment by federating a QSO database, a Subaru Suprim-Cam database, and some analysis services such a SExtractor and HyperZ web services. These experiences are described is this paper.
Japanese Virtual Observatory (JVO) as an advanced astronomical research environment
Advanced Software and Control for Astronomy, 2006
We present the design and implementation of the Japanese Virtual Observatory (JVO) system. JVO is a portal site to various kinds of astronomical resources distributed all over the world. We have developed five components for constructing the portal: (1) registry, (2) data service, (3) workflow system, (4) data analysis service (5) portal GUI. Registry services are used for publishing and searching data services in the VO, and they are constructed using an OAI-PMH metadata harvesting protocol and a SOAP web service protocol so that VO standard architecture is applied. Data services are developed based on the Astronomical Data Query Language (ADQL) which is an international VO standard and an extension of the standard SQL. The toolkit for building the ADQL-based service is released to the public on the JVO web site. The toolkit also provides the protocol translation from a Simple Image Access Protocol (SIAP) to ADQL protocol, so that both the VO standard service can be constructed using our toolkit. In order to federate the distributed databases and analysis services, we have designed a workflow language which is described in XML and developed execution system of the workflow. We have succeeded to connect to a hundred of data resources of the world as of April 2006. We have applied this system to the study of QSO environment by federating a QSO database, a Subaru Suprim-Cam database, and some analysis services such a SExtractor and HyperZ web services. These experiences are described is this paper.