Variable transforms

Introduction

This section describes the various variable transforms (or conversions) available within UFO.

All variable transforms are performed using a unique filter (Observation Filters) called Variable Transforms, as shown in the example below:

time window:
  begin: 2018-04-14T21:00:00Z
  end: 2018-04-15T03:00:00Z

observations:
  observers:
  - obs space:
      name: test_relative_humidity1
      obsdatain:
        engine:
          type: H5File
          obsfile: Data/ioda/testinput_tier_1/sfc_obs_2018041500_metars_small.nc
      simulated variables: [specificHumidity, airTemperature, stationPressure]
    obs filters:
    - filter: Variable Transforms
      Transform: ["RelativeHumidity"]
      Method: UKMO
      Formulation: Sonntag
The Variable Transforms filter has the following available yaml settings:

  • Transform: name of the variable transform that need to be performed. (see Variable transforms).

  • Method: [Optional] Method used during the variable transformation. This usually refers to the different Met Center. (See individual Variable transforms for possible choices and definitions)

  • Formulation: [Optional] Name of a specific formulation used during the variable transformation. (See individual Variable transforms for possible choices and definitions)

  • UseValidDataOnly: [Optional | default = true] Should the variable transform be applied only to valid data?

  • FillMissingDerivedFromOriginal [Optional | default = false] If true, fill any missing entries of a vector in a Derived group (e.g. DerivedObsValue) with the non-missing entries of the vector in the equivalent original group (e.g. ObsValue).

  • SkipWhenNoObs [Optional | default = true] If true, the variable transform will not be performed on a core if there are no observations on that core. If creating a new variable it might be pertinent to set to false as the code may fail when saving that variable.

    Warning

    If UseValidDataOnly=true, the variable transform is not applied to observations that have a QCflag equal to either missing or bounds.

The new variable derived by the filter is then stored in the observation space within the DerivedObsValue group. Since variables in groups with the Derived prefix “overshadow” variables from corresponding groups without that prefix (see Derived Variables), these variables can normally be accessed as if they were in the ObsValue group.

Warning

Each variable transform requires a specific set of variables (as specified in the documentation). If any compulsory variable is missing, the code will raise an exception and stop.

Formulations

The formulations available are:

Table listing all the symbols used