Masking not working properly with _Unsigned · Issue #794 · Unidata/netcdf4-python (original) (raw)

I have a Level 2 QPE product from GOES-16 that caused some support issues. The relevant CDL is:

netcdf satellite/goes16/GOES16/Products/RainRateQPE/FullDisk/current/OR_ABI-L2-RRQPEF-M3_G16_s20181072300427_e20181072311194_c20181072311280.nc {
  dimensions:
    y = 5424;
    x = 5424;
    number_of_time_bounds = 2;
    band = 1;
    number_of_image_bounds = 2;
    number_of_sunglint_angle_bounds = 2;
    number_of_LZA_bounds = 2;
    number_of_SZA_bounds = 2;
    number_of_lat_bounds = 2;
    number_of_rainfall_rate_bounds = 2;
  variables:
    short RRQPE(y=5424, x=5424);
      :_FillValue = -1S; // short
      :long_name = "ABI L2+ Rainfall Rate - Quantitative Prediction Estimate";
      :standard_name = "rainfall_rate";
      :_Unsigned = "true";
      :valid_range = 0S, -6S; // short
      :scale_factor = 0.00152602f; // float
      :add_offset = 0.0f; // float
      :units = "mm h-1";
      :resolution = "y: 0.000056 rad x: 0.000056 rad";
      :coordinates = "latitude retrieval_local_zenith_angle quantitative_local_zenith_angle solar_zenith_angle t y x";
      :grid_mapping = "goes_imager_projection";
      :cell_methods = "latitude: point (good quality pixel produced) retrieval_local_zenith_angle: point (good or degraded quality pixel produced) quantitative_local_zenith_angle: sum (good quality pixel produced) solar_zenith_angle: sum (good quality pixel produced) t: point area: point";
      :ancillary_variables = "DQF";

Note the values in valid_range; the values themselves are appropriate for a signed data type, but they only make sense as a range if you convert signed (-6) to unsigned (65530). The values in valid_range are not incorrect though, as the standards specify that the values need to be the same type as the variable.

The current out of the box behavior is that netCDF4-python returns an entirely masked variable. The work-around is to disable masking.

The correct behavior IMO is to have valid_range and friends be handled like the data values for unsigned purposes.

I've included the sample file.