Lightning Imaging Sensor (LIS) (original) (raw)

Lightning Imaging Sensor (LIS)

Table of Contents

Summary

This document provides basic information on the LIS instrument and the software that is used to extract the LIS data from the files. The software can be downloaded here: ftp://ghrc.nsstc.nasa.gov/pub/doc/lis/LISOTD_1.1.tar.gz To install the software, you need to follow these steps (on a Unix system):

>gunzip LISOTD_1.1.tar.gz
>tar -xvf LISOTD_1.1.tar

The first step uncompresses the file and then the second actually extracts the file. If you are using a Windows operating system and have Winzip©, it can extract these files for you.
Specific information about the HDF file structure and tools useful for data extraction are comprehensively discussed in the LIS/OTD Software Package . We strongly recommend that the user read the LISOTD_UserGuide before jumping into the data.

Note: As of October 28, 2013, the LIS data going forward will be proceesed with an updated version 4.2 algorithm.

The updated v4.2 software corrected the following two issues:

1. A problem with the clustering of events near the International Dateline (+-180 Longitude). 2. Near 00 UTC the LIS events ephemeris was tagged as missing, resulting in lost events and clustering/location issues.

Changes to data with version 4.1 release:

1. Orbit numbering scheme now matches that of the other TRMM satellite data. In the past, LIS used a different method of determining the orbit number that was inconsistent with that used by TRMM. Over time, orbit numbers didn't match. The v4.1 LIS code corrects this problem. 2. Subsecond timing is correct in all orbits. In v4.0, the subsecond timing was incorrect for some orbits (usually less than 100 ms or so). This has been fixed with the v4.1 code. This will make is easier to compare the LIS results with ground based lightning measurements. 3. Several orbits that could not be processed by the v4.0 code have been successfully processed using v4.1 code. This amounts to less than 10f the total orbits. 4. The v4.0 code sometimes used 'predicted' ephemeris files for processing. The v4.1 code uses the actual ephemeris data (Note: this has produced only minor differences in data). 5. Due to the sub second timing difference and new processing, the LIS lightning numbers changed slightly. Overall, the number of flashes in the v4.1 files vary less than 1% than those produced in the v4.0 code.

The LIS Instrument and Its Data

LIS was launched on 28 November 1997 aboard the Tropical Rainfall Measuring Mission (TRMM) Observatory into a nearly circular orbit inclined 35 degrees with an altitude of approximately 350km. TRMM will study mesoscale phenomena such as storm convection, dynamics, and microphysics. These will be related to global rates and amounts and distribution of convective precipitation, as well as to the release and transport of latent heat, which are all influenced by global scale processes.

The LIS instrument was designed by the GHCC Lightning Team and was manufactured at the Marshall Space Flight Center in Huntsville, Alabama. LIS will contribute significantly to several TRMM mission objectives by providing a global lightning and thunderstorm climatology from which changes (even subtle temperature variations) might be easily detected.

The LIS sensor contains a staring imager which is optimized to locate and detect lightning with storm-scale resolution of 3-6 km (3 at nadir, 6 at limb) over a large region (550-550 km) of the Earth's surface. The field of view (FOV) is sufficient to observe a point on the Earth or a cloud for 80 seconds, adequate to estimate the flashing rate of many storms. The instrument records the time of occurrence of a lightning event, measures the radiant energy, and estimates the location.

The calibrated lightning sensor uses a wide FOV expanded optics lens with a narrow-band filter (centered at 777 nanometers) in conjunction with a high speed charge-coupled device detection array. A real-time event processor (RTEP) is used to determine when a lightning flash occurs, even in the presence of bright sunlit clouds. Weak lightning signals that occur during the day are hard to detect because of background illumination. The RTEP will remove the background signal, thus enabling the system to detect weak lightning and achieve a 90% detection efficiency.

LIS Geolocation and Event Intercomparison

LIS geolocation of lightning events and background images involves many facets of the LIS program testing process. The orientation of the Charge Coupled Device (CCD) with respect to the LIS alignment cube was determined from an Euler angle analyses of precise yaw and pitch maneuvers of the LIS sensor head assembly during radiometric calibration of LIS. Then the orientation of the LIS alignment cube to the spacecraft-based attitude reference frame was determined. The alignment correction is simply a constant angular measure applied to spacecraft attitude. Given real-time updates of spacecraft ephemeris and attitude data, extremely accurate LIS geolocation is determined.

One form of intercomparison involves using the LIS background image and basic knowledge of geography. Because the radiant properties from land and water differ, LIS pointing can be verified by coastline discrimination of background images. In addition, LIS background cloud-field images are matched to appropriate visible and near-infrared satellite images.

Ground truth stations have been established for event intercomparison with data from LIS. Data from the NASA Kennedy Space Center Lightning Detection and Ranging (LDAR) system is to be a primary means for assessing event location errors. The 7-antenna LDAR time-of-arrival system maps lightning with high spatial resolution for sources within 100 km of the antenna network. This location accuracy is sufficient when compared to the storm-scale spatial resolution of LIS.

In addition, data from the National Lightning Detection Network (Vaisala), long range sferics systems, time-of-arrival (TOA) systems, and other lightning detection systems (e.g., interferometers) and networks (e.g., local networks operated at TRMM ground truth sites) are being used to verify LIS pointing accuracy.

Quality Control of the LIS Data

LIS exists in a noisy space environment. It also responds to a number of optical signals, not all of which are necessarily lightning-related. A significant amount of software filtering has gone into the production of science data distributed to the science community. The filters maximize both detection efficiency and confidence level so that each datum is a lightning signal and not noise.

Each LIS lightning event in a LIS file is tagged with four low-level quality indicators, while each LIS data file is assigned four high-level flags that were designed to notify potential users of possible irregularities in the data file. An automated process is used to tag each optical event in the LIS data file with a set of four numbers that indicate the relative likelihood that the event was produced by lightning, as opposed to solar glint, energetic particles in the Van Allen radiation belts, or electronic noise. These low-level tags are as follows:

1. Non-noise Probability (the probability that the event is not caused by random noise or energetic particles). 2. Solar Glint Factor (a number that indicates the likelihood that the event was caused by direct reflected solar radiation). 3. Event Rate Ratio (a number that represents the ratio of "accepted" events to the raw detected events during a one-second period at the time of the event). 4. Probability Density (a number that indicates whether the event is geolocated in the vicinity of other events that are likely to be lightning).

In addition, a LIS data file is manually inspected for irregularities in the data set. The data files that fail specific quality assurance are flagged. The high-level quality flags assigned to each LIS HDF data file (included as part of the HDF file) are as follows:

1. Instrument Alert Flag 2. Platform Alert Flag 3. External Alert Flag 4. Processing and Algorithm Alert Flag

Orbit file Varieties

The orbit files from LIS can come in 5 varieties, or classes:

Class 1- Good files - these files contain good data - be forewarned that occasionally the instrument/platform fatal flags may be intermittantly set in some of these orbits. In these orbits, about 50 of the one second data flags are set to fatal or warning. Unless these flags are contiguous, the data is considered good. The vast majority of the LIS files are in this category.

Class 2- Good files containing 0 events - These are a subset of the good files, except that no events were observed. This subset is broken listed separately, because even though they contain no events, there is a dummy vdata set of length 1 inserted into these files to prevent problems in reading the files. All fields in the dummy point data sets are set to 0. The viewtimes data are good and are necessary when computing climatological lightning rates. These files are not listed separately anywhere, it is up to the user to determine how to work with them. There are only about 10 of these files a year.

Class 3- Files unreadable with the idl code: These files contain good orbit data, but the LIS instrument wasn't working because it was turned off for some reason. The one second data vdata can be read in, but the lightning data has a length of 0 that causes some software to crash. It should be noted that there is no lightning information in these files since the instrument was turned off. A listing of these orbit numbers will be maintained on the web site.

Class 4- Files with known anomalies - These files have been observed to have some sort of anomaly, such that lightning data are available for only part of the orbit. The one second data flags are set correctly in these files. These files are documented on the LIS data anomalies web site (https://ghrc.nsstc.nasa.gov/lightning/data/data_lis_trmm_anomalies.html) as a courtesy. Note however, that not all the files anomalies may be listed on the web site. It is up to the user to check the one second data to verify that the data are good. In particular, LIS buffer overflows may not be listed due to the short duration of the data outage. In addition, files that occur immediately before and after files of type Class 3 will probably be in this category and will not be listed on the anomalies page.

Class 5- Missing files - Some files are simply not produced. These are the same as class 3. above, except no files are produced. The causes vary, but are mainly due to instrument outages due to sun acquisition manuevers, Leonid meteor stream, etc.

Because they contain no useful science data, files of type Class 3 and Class 5 will not be distributed.

The Format of the LIS Data and the NCSA HDF Libraries

There are three products generated from the raw LIS data. They are:

LIS Backgrounds

Lightning Imaging Sensor Background Images. These background images created approximately one to two seconds apart provide the scene on which lightning can be plotted. When using the LIS/OTD Read Software, an entire orbits worth of background images can be displayed in a simple animation to allow a quick way to see if interesting cloud systems (hurricanes, MCSs, Frontal systems, etc.) were in the field of view.

LIS Science Data

Lightning Imaging Sensor Science Data. These data are stored in a Hierarchical Data Format (HDF), the standard format for Earth Observing System (EOS) projects. HDF is a platform independent data format used for the storage and exchange of scientific data. The LIS/SCF (Science Computing Facility) has spent considerable time and effort in designing and constructing a software package which greatly facilitates the extraction of meaningful information from the LIS HDF datasets. This software (See paragraph 3 below) is included with each order from the GHRC, and is available on line at https://ghrc.nsstc.nasa.gov/lightning/dataset-info.html under the LIS dataset collection. Additional information can be found in the charts from the links below.

* Orbit * One Second Data * Orbit Summary Image * Point Data * Area * Flash * Group * Event * Viewtime Granules * Background Image Summary

LIS Browse

Lightning Imaging Sensor Browse Images. Daily browse images are created showing the ascending and descending orbits, location of lightning and statistical data, as shown below. All browse images are available on line at: https://ghrc.nsstc.nasa.gov/lightning/dataset-info.html. Selectable from a table are both Quality Controlled (QC Browse) and Non-Quality Controlled (NQC) images. As LIS data is QCed, the NQC images will be deleted as the QC images appear on the page.

Orbit Data in Browse

It is significant to note that the 'day' in the browse imagery and the 'day' in the science data are defined differently.

Click here to display a sample browse image in a new browser window to go along with the discussion below.

Data contained in the browse imagery begins at 00Z, and ends 24 hours later- that is, from midnight to midnight. If the satellite happens to be at 30 degrees North latitude on the descending portion of the orbit at 00Z, then that is the point at which the browse imagery begins the day, and consequently begins the data swath.

In the above example browse image, orbital swaths are easily recognizable as blue 's' shaped curves in each of the panels. All ascending passes (that portion of the orbit from the southernmost point at 35S to the northernmost point at 35N) are in the top panel, and descending passes are in the bottom panel. Along the bottom of each panel is posted a time in Universal Coordinated Time also called UTC, Greenwich Mean Time (GMT) or Zulu (Z) time. These times correspond to the equatorial crossing time (ECT)located in the image as a tic mark on the equator directly above the time.

Along the top of the image are posted local times. These are to the local solar time (LST) of the equatorial crossing. LST along with latitude and longitude is essential to determine the angle of the suns rays at the surface. Local solar time is determined from the longitude of the point in question. This time differs from local standard time with which we are all familiar. Time zones are ideally 15 degrees in longitude wide, and in that time zone all clocks are set to the same time. Solar time is unique for each longitude. If you are at 83W, for example, local solar noon occurs when the sun passes through your longitude; 83W, which is four minutes later than was solar noon at 82W. Local solar midnight for 83W occurs when the sun passes through 97E, when the sun has traveled 180 degrees around the globe. 0600LST occurs when the sun passes through 7E (90 degrees before 83W), and 1800LST when the sun passes through 173W (90 degrees after 83W).

In the top panel, the ECT posted on the far left is at 1443Z, which corresponds to a local solar time of 0224. (Note the disparity between UTC and LST: the minutes in each of the times is different. Remember: local solar time is determined with respect to the equatorial crossing longitude, and not with respect to local time zones.) By examining sequential ECTs (using the bottom times) one can determine the approximate orbital period of the TRMM observatory, in this case between 91 and 92 minutes. Knowing that an orbit takes about 91 minutes, means that half an orbit takes about 45 minutes and a quarter of an orbit (from the equator to 35N and one quarter of the distance around the Earth) takes ~22.5 minutes.

If one examines the above example browse image carefully, one can see that in the ascending orbit (top) graphic just off of the west coast of India (circled), the swath simply stops. This is 00Z at the end of the day. Following that orbit backwards (to the left) until it intersects the equator and dropping down to the corresponding time indicates that ECT was at 2351UTC: nine minutes before the end of the GMT day. Subsequent to this ECT, the satellite traveled about halfway toward the top of the orbit, or somewhere around 10 more minutes before the day ended. (This eyeballing of the data gives a sanity check to the image). The beginning of the day is a bit tougher to find, but can be seen in the descending (bottom) image about 40 degrees to the east of Japan at about 30 degrees North latitude: again this point is circled. Applying the sanity check to this orbit shows that the subsequent ECT occurred at 0016UTC, which makes the start time of the orbit look about right.

Data contained in the science data files begin and end differently from that of the browse products. LIS orbits themselves are defined to start at the southernmost latitude (35 degrees South) which corresponds to the beginning of the ascending part of the orbit. You can see that in the top graphic as the start point for any swath, which is located at 35S. The first orbit of the GMT day is defined as that orbit containing 00Z for the day no matter where in the orbit that time occurs. In other words, for all but a very few days (when 00Z happens to coincide with the southernmost point of the orbit) the first orbit of the science data actually starts on the preceding day. For instance, the science data for 5 Jan 98 (whose browse product is shown above) contains data from 4 Jan 98 because the beginning of the orbit containing 00Z on the 5th, began somewhere near 35S 20W (extrapolating an orbit backwards from 00Z in the above image). Note that this portion of the swath is not plotted on the browse image. The science data files also end prior to 00Z on the 6th. In fact, the last point on the science data from the 5th would be near 35S 50W just off of the coast of Argentina. The rest of that orbit is contained in the science data for 6 Jan 98, but that in the browse image the swath continues until 00Z.

LIS/OTD Read Software

NOTE: LIS HDF data can be read with "C" libs or with IDL. The "C" libs require NCSA HDF libs. A new software package has been written to read LIS HDF files. The following is from the introductory chapter of the users manual for the software package which gives the philosophy of the software paradigm:

This document serves as a guide to the software intended for use with satellite data from the Optical Transient Detector (OTD) and Lightning Imaging Sensor (LIS). The software suite consists of both fully featured GUI (Graphical User Interface) driven applications, and collections of high- and low-level APIs (Application Programming Interfaces). The software is designed to simplify, as much as possible, user access to the OTD and LIS lightning data sets, which are currently distributed in HDF (Hierarchical Data Format) files. The suite is designed with four goals in mind: simplicity, reusability, compatibility and deployment. By providing software strongly tailored to these goals, we hope to minimize each user's time spent accessing and managing the datasets, and maximize the time spent actually analyzing them.

NCSA HDF Libraries

To read files written in a HDF format, it is necessary to obtain software from The National Center for Supercomputing Applications (NCSA) which is available via the Internet at URL http://www.hdfgroup.org/. NCSA provides a public domain library supporting HDF on a wide variety of computer platforms.

In addition to the NCSA software libraries, a LIS-specific application software library is available for accessing and managing data from the LIS HDF files. This set of C library functions is available to users running SGI IRIX 5.3 and NCSA HDF 3.3r4 or higher, and can be downloaded by following the instructions given in the section entitled "LIS/OTD Software Package" in paragraph 4.1 below.

Obtaining Software

The LIS/OTD software package has been developed by the LIS team to allow users relatively pain free access to the LIS data. Specific system and platform requirements are spelled out in Chapter 3 of the LIS/OTD software manual. For more information, please review Chapter 3 of the Software manual.

The LIS/OTD Software Package

Go to URL https://ghrc.nsstc.nasa.gov/lightning/dataset-info.html. There, in the LIS information column you will find the links to the software manual (which is a PDF file), the program files, release notes and Quick Start guide.

The NCSA HDF Library

See http://www.hdfgroup.org/.

References

Christian, H. J., R. J. Blakeslee, and S. J. Goodman, The Detection of Lightning from Geostationary Orbit, J. Geophys. Res., Vol. 94, pp. 13329-13337, 1989.

Christian, H. J., R. J. Blakeslee, S. J. Goodman, and D. M. Mach, Algorithm Theoretical Basis Document (ATBD) For the Lightning Imaging Sensor (LIS), Earth Observing System (EOS) Instrument Product.

Christian, H.J., R.J. Blakeslee, and S.J. Goodman, Lightning Imaging Sensor (LIS) for the Earth Observing System, NASA Technical Memorandum 4350, MSFC, Huntsville, AL, February, 1992.

Contact Information

To order these data or for further information, please contact:

> Global Hydrology Resource Center > User Services > 320 Sparkman Drive > Huntsville, AL 35805 > Phone: 256-961-7932 > E-mail: support-ghrc@earthdata.nasa.gov > Web: http://ghrc.nsstc.nasa.gov/