TOPCAT (original) (raw)
Tool for OPerations on CataloguesAndTables
Does what you want with tables
Latest (see Version History for details)
Version 4.10 released 7 August 2024
I/O handler enhancements, bugfixes, build/source modernisation
Version 4.9-1 released 29 February 2024
New: TAP window improvements
New ADQL tab displays all service functions, UDFs and optional features;
Choose either service-defined or alphabetical order for schema/table tree listing;
Tables selected in service search show up immediately in metadata panel.
New: HAPI Load Dialogue
Query and download data fromHeliophysics Data API time series services.
See introductionslides/video if you know about HAPI but not TOPCAT.
New: Cube Plot coordinate extensions
New Coordinates selector lets you plot 3-vectors or spherical polars as well as X, Y, Z components.
- What is TOPCAT?
- Features
- Documentation
- Frequently Asked Questions
- Mailing Lists
- Tutorials and Examples
- Screenshots
- Downloads
- Version history — Version 4.10 released 7 August 2024
- Further information
What is TOPCAT?
TOPCAT is an interactive graphical viewer and editor for tabular data. Its aim is to provide most of the facilities that astronomers need for analysis and manipulation of source catalogues and other tables, though it can be used for non-astronomical data as well. It understands a number of different astronomically important formats (including FITS, VOTable and CDF) and more formats can be added. It is especially good at interactive exploration of large (multi-million row, lots of columns) tables.
It offers a variety of ways to view and analyse tables, including a browser for the cell data themselves, viewers for information about table and column metadata, and facilities for sophisticated interactive 1-, 2-, 3- and higher-dimensional visualisation, calculating statistics and joining tables using flexible matching algorithms. Using a powerful and extensible Java-based expression language new columns can be defined and row subsets selected for separate analysis. Table data and metadata can be edited and the resulting modified table can be written out in a wide range of output formats.
It is a stand-alone application which works quite happily with no network connection. However, because it uses Virtual Observatory (VO) standards, it can cooperate smoothly with other tools, services and datasets in the VO world and beyond.
The program is written in pure Java and available under the GNU General Public Licence, though some of the library code is LGPL. Source code is available athttps://github.com/Starlink/starjava. It has been developed mostly in the UK within various UK and Euro-VO projects (Starlink, AstroGrid, VOTech, AIDA, GAVO, GENIUS, DPAC) and under PPARC and STFC grants. Its underlying table processing facilities are provided by the related packagesSTIL andSTILTS.
Features
The following is a list of the program's main capabilities. The hyperlinks are to the relevant parts of the user document.
- Fast access to large datasets (millions of rows/hundreds of columns)
- View/edit table data in a scrollable browser
- View/edit table and column metadata
- Re-order and hide/reveal columns
- Insert 'synthetic' columns defined by algebraic expression
- Sort rows on the values in a given column
- Define row subsets in various ways
- View interactive and configurableplots of column-based quantities against each other distinguishing different data sets:
- Plot types arehistogram,plane,sky,cube,sphere,time
- Features includevariable transparency,error bars,point labelling,colour axes,all-sky plots,configurable density shading,vectors,ellipses,areas,polygons,lines,contours,density maps,KDEs,analytic functions,plain text/LaTeX axis annotation, ...
- Plots can be exported in bitmapped or vector formats, and a command to script the same plot is displayed
- Calculate statistics on each column for some or all rows
- Perform flexible and fast matching of rows in the same or different tables
- Cause various things to happen when a row or plotted point is selected.
- Communicate with other applications using SAMP
- Acquire tables from afile, URL, orSQL query
- Communicate with external VO and non-VO data services, possibly withsingle-sign on, includingTAP,cone search,VizieR,CDS X-Match,CDS Hips2fits,SIA,SSA,DataLink,HAPI.
- Perform multiple (per-row)cone search,SIA orSSA queries, to join a local to a remote catalogue or image/spectrum archive.
- Concatenate the rows of existing tables to create new ones
- Write modified tables out in original or different format tofile or an SQL database
- Comprehensive documentation supplied within the application or off-line
Supported table input formats include:
- FITS BINTABLE (binary table) or TABLE (ASCII table) extensions
- VOTables in any of the format variants (TABLEDATA, FITS, BINARY, BINARY2) or versions
- ASCII tables in a number of variations
- CDF files
- Feather files
- Comma-Separated Values (CSV)
- Enhanced Character-Separated Values (ECSV)
- Results of SQL queries on relational databases
- IPAC format
- AAS Machine-Readable Tables
- Apache Parquet
- HAPI streams
- GBIN files and supported output formats include:
- FITS BINTABLE (binary table)
- VOTables in any of the format variants (TABLEDATA, FITS, BINARY, BINARY2) or versions
- Plain ASCII text
- Comma-Separated Values (CSV)
- Enhanced Character-Separated Values (ECSV)
- New table exported to an SQL-compatible relational database
- IPAC format
- Apache Parquet
- HTML TABLE element
- LaTeX
tabular
environment
Documentation
Full tutorial and reference documentation for TOPCAT is provided bySUN/253, the user document. This is available within the program at runtime via the context-sensitive and searchable help system, or in the following forms within the distribution or on the web:
Frequently Asked Questions
A list of frequently asked questions is available. If you want to suggest additional questions and/or answers, please get in touch.
Mailing Lists
Two mailing lists are in use for TOPCAT and related software:
topcat-user@jiscmail.ac.uk
: Public list for questions, answers, discussions, bugs, comments, ...topcat-announce@bristol.ac.uk
: Low-volume read-only list for release announcements and news items. See themailing list page for more information.
If you have queries or support enquiries you can send them totopcat-user
(especially if the answer might be of interest to other users), or email me if you prefer that.
Tutorials and Examples
Demos/tutorials of using TOPCAT with Gaia DR3 data:
- TOPCAT and Gaia DR3:MP4 Video (52mins, 155Mb) and accompanyingslides (2022, TOPCAT >=v4.8-6)
- TOPCAT and Gaia XP SpectraMP4 Video (16mins, 72Mb) and accompanyingslides (2022, TOPCAT >=v4.8-7)
Tutorial script exercising TOPCAT and STILTS with Gaia DR2/EDR3 data (built from repository - you can adapt it):
- Script from ESCAPE VO School #1/#2, using Gaia EDR3 (2021/2)
- Script from ASTERICS VO School #4, using Gaia DR2 (2018)
- Script from ASTERICS VO School #1, using Gaia DR1 (2016)
Introduction/Demo tutorial covering some similar material as presented toShristi Astronomy/AstroSprint courses:
- Introduction:Youtube video andaccompanying slides (2020)
- Crossmatching:Youtube video andaccompanying materials (2021)
TOPCAT as HAPI client, intended as TOPCAT introduction for HAPI users:
Español: Tutorial para crear el diagrama H-R (Hertzsprung-Russell) de las Pléyades (M45) utilizando TOPCAT
- Youtube Video (21 mins) preparado porHelios GTTP
Some older tutorials are available too:
- Demo video by Niall Deacon (Hawaii), originally posted onAstroBetter (2010).
- Tutorial script used for a seminar at ARI Heidelberg (2011)
- Tutorials on v4.1 plotting and matching from a Gaia Brown Dwarfs meeting (2014)
And some more general items may also be helpful:
- Virtual Observatory Text Treasures from GAVO contains many nice tutorials, including several relating to TOPCAT
- Beginner's Guide to Working with Astronomical Data by Markus Pössel (MPIA); Chapter 4 is a nice undergraduate-level introduction to TOPCAT (2019)
Screenshots
You can see screenshots of TOPCAT in action in the following places:
- SUN/253's windows appendix details all of TOPCAT's windows with illustrations
- [somewhat outdated] The TOPCAT gallery is a page containing lots of window screenshots
- [somewhat outdated] The TOPCAT V4 graphics page showcases some features of the new-style plotting windows introduced in version 4 (March 2013)
- [very outdated] The TOPCAT V3 screenshots page showcases some of the features introduced at version 3 (August 2007)
- There's a montage of some of the windows TOPCAT provides below; click on it for a full size version
Downloads
TOPCAT is written in the Java language using the Java 2 Standard Edition version 8, and should run on any Java SE 8 or more recent system. This means it can be run on a wide range of platforms, without requiring any recompilation - you just need to ensure that you have a suitable Java Runtime Environment (JRE) - though that's not necessary for the MacOS DMG download, see below. If you don't have Java installed, or have an unsuitable version, you can obtain the Java SE for Linux, Mac OS X, MS Windows and Solaris fromOracle's web site(you only need the "JRE" rather than the "JDK" download, unless you will be doing development work). Java SE Runtime Enviroments (sometimes called JVMs or Java Virtual Machines) for other platforms may be available from operating system vendors. OpenJDK is also suitable.
If starting TOPCAT fails with an error likejava.lang.UnsupportedClassVersionError
, you have a version of Java that is too old, and you should upgrade to Java 8 or later. If you're unable or unwilling to do that and you only have the rather ancient Java 6, you can still use TOPCAT versionv4.6-3 or older.
Having got Java, There are several ways to download TOPCAT, described in rough order of advisability in the following subsections. More information on how to run the program having obtained it can be found in SUN/253's section on Invoking TOPCAT.
Standalone Jar File
The generally recommended form for downloading is to pick up a single Jar file containing the required classes:
- <topcat-full.jar> (30.7M) - standard version
- <topcat-extra.jar> (51.0M) - bloated version including Parquet file format support
On Unix-like operating systems, download one or other of these jar files and the startup script into the same directory, then "chmod +x topcat
", and you can just run the command:
topcat
On non-Unix systems the script won't work, and you can use a command like:
java -jar topcat-*.jar
or invoke it in some other system-dependent way such as by clicking on it.
MacOS
If you have an Apple Mac with homebrew installed you can just do
brew install --cask topcat --no-quarantine
which will install the TOPCAT application and the stilts
and topcat
scripts, along with all dependencies. Without homebrew, you can pick up the DMG file manually:
<topcat-all.dmg> (114.6M)
You are advised to download it using curl
(curl -OL http://www.starlink.ac.uk/topcat/topcat-all.dmg
) rather than the browser otherwise it will need to be unquarantined — see the FAQ entryproblem with "damaged" dmg file.
The application DMG file contains a bundled Java Runtime Environment, user manuals for TOPCAT and STILTS, and the<topcat-extra.jar> file (see above). The FAQ contains advice onhow to set flags for memory usage etc. It ought to work for both Intel and Apple Silicon hardware, and does not require an external Java installation. It is therefore quite large; if you want something more compact, you can use an existing java installation with one of thetopcat-*.jar
Standalone Jar Files listed in the previous section. The main disadvantage is that you don't get to click on a yellow cat to start it.
This DMG file was built usingInstall4j, a multi-platform installer builder, which has saved me a lot of Mac-specific wrangling.
Full Starjava Installation
If you want the most comprehensive installation then download and unpack the full starjava tree in one of the following forms:
- starjava compressed tar archive (233.5M)
- starjava zip archive (233.5M)
These archives include related applications such asSPLAT,SoG andTreeview. You can run TOPCAT using the starjava/bin/topcat
script (Unix) or by running java -jar starjava/lib/topcat/topcat.jar
.
From other distributions
TOPCAT is also packaged as part of some other larger software distributions:
- The Debian Astro bundle (see the FAQ)
- The Starlink Software Collection
- others?
Note however these may not provide the most up to date version.
Version History
The most recent public release of TOPCAT is version 4.10, released 7 August 2024.
For a detailed history of the changes in this and previous releases, see the
section of SUN/253.
Further Information
STILTS
TOPCAT's sister package is STILTS, the STIL Tool Set. STILTS offers many of the same facilities as TOPCAT (and some additional ones) in the form of command-line tools, which can be invoked from the Unix/Windows command-line prompt, or from Jython.
Other Resources relating to the code
- STIL web page for the table handling classes
- Javadocs for TOPCAT classes (though this API is not intended for public use)
- Source code zip file for the TOPCAT classes (see FAQ)
- github page for git source code repository (see FAQ)
- Archive containing old, and perhapspre-release versions of TOPCAT
Acknowledgements
- The code relies on many bits of third-party software, and advice and contributions from many users; credit for some of these can be found in theAcknowledgements section of the manual.
- If you use this software in published work, then citing paper2005ASPC..347...29T would be appreciated.
TOPCAT is currently (2024) under active development, and I'm very open to user feedback. Any comments, questions, requests, bugs etc, please either post to the topcat-user list or contact me direct:
- Author: Mark Taylor,Astrophysics Group,Physics Department,University of Bristol
- Email: m.b.taylor@bristol.ac.uk
- TOPCAT WWW page: http://www.starlink.ac.uk/topcat/