Variable transforms

Introduction

This section describes the various variable transforms (or conversions) available within UFO.

All variable transforms are performed using a unique filter (Observation Filters) called Variable Transforms, as shown in the example below:

time window:
  begin: 2018-04-14T21:00:00Z
  end: 2018-04-15T03:00:00Z

observations:
  observers:
  - obs space:
      name: test_relative_humidity1
      obsdatain:
        engine:
          type: H5File
          obsfile: Data/ioda/testinput_tier_1/sfc_obs_2018041500_metars_small.nc
      simulated variables: [specificHumidity, airTemperature, stationPressure]
    obs filters:
    - filter: Variable Transforms
      Transform: ["RelativeHumidity"]
      Method: UKMO
The Variable Transforms filter has the following available yaml settings:

  • Transform: name of the variable transform that need to be performed. (see Variable transforms).

  • Method: [Optional | default = default] Method or “recipe” used for the variable transformation. Often this refers to a Met Center. Any method in the list of all possible variable transform methods can be specified, but in many cases this will result in a default method being used. For example, the method GoffGratchLandoltBornsteinIceWater can be specified for any variable transform, but will cause a default method to be used in all cases but the relative and specific humidity variable transforms. See individual Variable transforms for default methods, possible choices and definitions.

  • UseValidDataOnly: [Optional | default = true] If true, the variable transform is only applied to valid data.

  • FillMissingDerivedFromOriginal [Optional | default = false] If true, fill any missing entries of a vector in a Derived group (e.g. DerivedObsValue) with the non-missing entries of the vector in the equivalent original group (e.g. ObsValue).

  • SkipWhenNoObs [Optional | default = true] If true, the variable transform will not be performed on a core if there are no observations on that core. If creating a new variable it might be pertinent to set to false as the code may fail when saving that variable.

    Warning

    If UseValidDataOnly=true, the variable transform is not applied to observations that have a QCflag equal to either missing or bounds.

The new variable derived by the filter is then stored in the observation space within the DerivedObsValue group. Since variables in groups with the Derived prefix “overshadow” variables from corresponding groups without that prefix (see Derived Variables), these variables can normally be accessed as if they were in the ObsValue group.

Warning

Each variable transform requires a specific set of variables (as specified in the documentation). If any compulsory variable is missing, the code will raise an exception and stop.

Variable transforms

The variable transforms available are:

Adjusted height

Height from pressure

Humidity

Logarithm

Ocean Conversions

PotentialTemperature

Pressure from height

Profile horizontal drift

Radar beam geometry

Remap scan position

Satellite brightness temperature from radiance

Satellite radiance from scaled radiance

Satellite zenith angle correction

Surface pressure

Surface wind scaling for height coordinate

Surface wind scaling for pressure coordinate

Surface wind scaling for combined height-pressure coordinate

Wind

Symbols

Table listing all the symbols used