STILTS (original) (raw)

StarlinkTablesInfrastructureLibraryToolSet

Latest (see Version History)

Version 3.5-1 released 6 November 2024

Bugfixes, minor enhancements

Version 3.5 released 7 August 2024

New: TAP service authentication

Commandstapquery,tapskymatch,taplint andtapresume now allow login to TAP services with optional authentication (auth=true)

I/O handler enhancements, bugfixes, build/source modernisation

• What is STILTS?
• Commands
• Documentation
• Mailing Lists
• Downloads
• Further Information

What is STILTS?

The STIL Tool Set is a set of command-line tools based on STIL, the Starlink Tables Infrastructure Library. It deals with the processing of tabular data; the package has been designed for, but is not restricted to, astronomical tables such as source catalogues. Some of the tools are generic and can work with multiple formats (including FITS, VOTable, CDF, ECSV, CSV, PDS4, Parquet, MRT, Feather, GBIN, SQL and ASCII), and others are specific to the VOTable format. STILTS is the command-line counterpart of the GUI table analysis toolTOPCAT. The package is robust, fully documented, and designed for efficiency, especially with very large datasets.

Facilities offered include:

• format conversion
• crossmatching
• plotting
• column calculation and rearrangement
• row selections
• data and metadata manipulation and display
• sorting
• statistical calculations
• histogram calculation and aggregate functions
• data and service validation
• access to remote data services including Virtual Observatory A powerful and extensible expression language is used for specifying data calculations. These facilities can be put together in very flexible and efficient ways.

For tasks in which the data can be streamed, the size of table STILTS can process is effectively unlimited. For other tasks, tables of a few million rows and a few hundred columns usually do not present a problem.

STILTS is written in pure Java (Java SE 8 or later; versions prior to STILTS 3.2 were Java SE 6), and can be run from the command line or from Jython, or embedded into java applications. The STILTS application is released under the GNUGPL, though much of the library code is licensed under the LGPL or other less restrictive licenses - see <LICENSE.txt>.

Commands

STILTS currently consists of generic table processing commands:

• [tcopy](sun256/tcopy.html) - Table format converter
• [tpipe](sun256/tpipe.html) - Generic table pipeline processing utility
• [tmatch2](sun256/tmatch2.html),[tskymatch2](sun256/tskymatch2.html) - Two-table crossmatchers
• [tmatch1](sun256/tmatch1.html) - Intra-table crossmatcher
• [tmatchn](sun256/tmatchn.html) - Multi-table crossmatcher
• [tjoin](sun256/tjoin.html) - Trivial side-by-side multiple-table joiner
• [arrayjoin](sun256/arrayjoin.html) - Adds table-per-row data as array-valued columns
• [tgridmap](sun256/tgridmap.html) - N-dimensional histogram calculator with table output
• [tcube](sun256/tcube.html) - N-dimensional histogram calculator with FITS array output
• [tgroup](sun256/tgroup.html) - Grouped aggregate function calculator
• [tcat](sun256/tcat.html),[tcatn](sun256/tcatn.html) - Multiple-table concatenaters
• [tmulti](sun256/tmulti.html),[tmultin](sun256/tmultin.html) - Multiple-table container writers
• [tloop](sun256/tloop.html) - Test table creation utility plotting commands (as well as three deprecated ones):
• [plot2plane](sun256/plot2plane.html) - Plots on 2d Cartesian axes
• [plot2sky](sun256/plot2sky.html) - Plots on celestial axes
• [plot2cube](sun256/plot2cube.html) - Plots on 3d Cartesian axes
• [plot2sphere](sun256/plot2sphere.html) - Plots in spherical polar space
• [plot2corner](sun256/plot2corner.html) - Plots a Corner Plot, a grid of scatter plots and histograms
• [plot2time](sun256/plot2time.html) - Plots 2-d axes with Time horizontally VOTable-specific commands:
• [votcopy](sun256/votcopy.html) - VOTable encoding translator
• [votlint](sun256/votlint.html) - VOTable validity checker Virtual Observatory/external data service access commands:
• [cone](sun256/cone.html) - Cone-search like queries (including SIA and SSA)
• [tapquery](sun256/tapquery.html),[tapresume](sun256/tapresume.html),[tapskymatch](sun256/tapskymatch.html) - TAP service clients
• [cdsskymatch](sun256/cdsskymatch.html) - match local table against VizieR/SIMBAD using CDS X-Match service
• [taplint](sun256/taplint.html) - TAP service test suite
• [datalinklint](sun256/datalinklint.html) - DataLink validator
• [regquery](sun256/regquery.html) - Registry Query
• [coneskymatch](sun256/coneskymatch.html) (formerly multicone, now somewhat deprecated) - Match local table with one behind a Cone Search/SIA/SSA service sky pixel-related commands:
• [tskymap](sun256/tskymap.html) - Generate HEALPix sky density maps
• [pixfoot](sun256/pixfoot.html) - Generate Multi-Order Coverage (MOC) maps
• [pixsample](sun256/pixsample.html) - Sample from a HEALPix pixel data file SQL-specific commands:
• [sqlskymatch](sun256/sqlskymatch.html) (formerly sqlcone) - Match local table with one in an SQL database
• [sqlclient](sun256/sqlclient.html) - JDBC-based SQL command-line client
• [sqlupdate](sun256/sqlupdate.html) - Updates data in existing cells of an RDBMS table and some miscellaneous items:
• [server](sun256/server.html) - HTTP server which executes STILTS commands
• [calc](sun256/calc.html) - Quick expression evaluator
• [funcs](sun256/funcs.html) - Documentation browser for expression language functions
• [xsdvalidate](sun256/xsdvalidate.html) - Validates against XML schema

See also the Commands by Categorysection of the manual.

More commands and facilities may be added in the future.

Documentation

The STILTS documentatation is provided in the user document, SUN/256. This is available in the following formats:

• Multi-page HTML document
• Single-page HTML document (2.4M)
• PDF document (4.0M)

Mailing Lists

Two mailing lists exist for STILTS, TOPCAT and related software:

• topcat-user@jiscmail.ac.uk: Public list for questions, answers, discussions, bugs, comments, ...
• topcat-announce@bristol.ac.uk: Low-volume read-only list for release announcements and news items. See themailing list page for more information.

If you have queries or support enquiries you can send them to topcat-user. However, I'm still happy to answer mail directly tome if you prefer that.

Downloads

You can download STILTS from one of the following links:

• Main jar file <stilts.jar> (16.1M)
• Invocation script
• Standalone JyStilts jar file <jystilts.jar> (57.8M) to run JyStilts without an independent Jython installation
• Documentation zip file (6.5M)
• Full source code repository athttp://github.com/Starlink/starjava/ (see ttools subdirectory)
• Source code zip file (3.7M)
• Zip archive of individual jar files<stilts%5Fjars.zip> (14.8M) (may be useful for those using STILTS classes as a library)
• <LICENSE.txt> file discussing licensing
• Archive of previous, and perhapspre-release versions The <stilts.jar> file contains all the classes required to run STILTS. If you run "java -jar stilts.jar" you will see a help message; on Unix you also have the choice of downloading the script into the same directory and using that for convenience. For more details on invoking the tools, read SUN/256.

It is also possible to run STILTS commands against an installation of the full Starjava set or of TOPCAT(run "topcat -stilts ... or "java -jar topcat-full.jar -stilts ...") since the relevant classes are present in each case.

If you have MacOS and homebrew, then installing TOPCAT with "brew install --cask topcat --no-quarantine" will install the stilts script along with the TOPCAT application.

The current release of STILTS is v3.5-1, released 6 November 2024. The full Version History is available in the user document.

Further Information

Products related to STILTS are:

STIL

Public API underlying the functions of STILTS

TOPCAT

Graphical user tool for interactive table manipulation

If you have any comments, questions, requests, bugs etc, please either post to the topcat-user list or contact me direct:

• Author: Mark Taylor,Astrophysics Group,School of Physics,University of Bristol
• Email: m.b.taylor@bristol.ac.uk
• STILTS WWW page: http://www.starlink.ac.uk/stilts/

If you use this software in published work, then citing paper2006ASPC..351..666Twould be appreciated.

And finally...

Q: What do you get if you combine all the Starlink Java table handling applications?
A: TOPCAT STILTS.
Thanks to Proporta and Malcolm Currie for that.