Find ts_period and clim_period once (and discard period manager afterwards) (original) (raw)

The systematic search for the available data with the period manager for each diagnostic can be time consuming and not necessary.

If you want to:

• speed up the execution of the atlas (discard the period availability check for each diagnostic and just do it once for all)
• keep the same period from one execution of the atlas to the next (especially on an ongoing simulation, from one day to another)

you can apply the period manager only once for a comparison using set_available_period_ts_clim.py:

python set_available_period_ts_clim.py comparison/ [variable]

=> It will apply the period manager and find both the ts and clim periods when necessary (requested by the user)

Note: do source setenv_C-ESM-EP.sh if you have this error at the execution of the script:

File "set_available_period_ts_clim.py", line 18, in 
from CM_atlas import *
ImportError: No module named CM_atlas

And re-run the script.

It creates a file datasets_setup_available_period_set.py in the comparison directory, containing the ‘true’ clim_period and ts_period (real dates). If this file exists, the C-ESM-EP will use it instead of datasets_setup.py to specify the datasets, and not apply the period manager.

Atmosphere_Surface			       ENSO		    NEMO_main
Atmosphere_zonmean			       HotellingTest	    NEMO_zonmean
cesmep_atlas_style_css			       job_C-ESM-EP.sh	    NH_Polar_Atmosphere_Surface
climaf.log				       job_PMP_C-ESM-EP.sh  ORCHIDEE
datasets_setup_available_period_set_223181.py  last.out		    ParallelCoordinates_Atmosphere
datasets_setup_available_period_set_238422.py  MainTimeSeries	    PISCES
datasets_setup_available_period_set.py	       Monsoons		    SH_Polar_Atmosphere_Surface
datasets_setup.py			       NEMO_depthlevels     TuningMetrics

Simply remove it if you want to come back to normal.

If there is already a datasets_setup_available_period_set.py file in the directory (as in the example), it will make a copy of it (add a number as tag) so that you can use this backup it afterwards.

The file datasets_setup_available_period_set.py is the last file produced by set_available_period_ts_clim.py and is the one that is used by the C-ESM-EP.

Advantages

By keeping the same periods from one day to the next, a big advantage is that you make use of the existing results in the cache (if you have no scientific or practical reasons to update the periods of study).

This is particularly interesting when working interactively: you do not loose time searching for available results every time you run the atlas.

You can also use this tool as a way to check if your dataset specification is correct: if you provide a variable as last argument to the script, it will actually try to get the files corresponding to the user list of datasets for this variable, and return an error if it doesn’t find it.

python set_available_period_ts_clim.py comparison/ tas

This way you can fix minor errors before running the big atlas.

Have a look at this page on the No File Found error to debug this classical error.