Creating a New Filter¶

If none of the filters described above meets your requirements, you may need to write a new one. If possible, make it generic (applicable to arbitrary observation types). The source code of UFO filters is stored in the ufo/src/ufo/filters folder. You may find it useful to refer to the JEDI Academy tutorials on writing and testing a filter.

When writing a new filter, consider using the Parameter Classes to automate extraction of filter parameters from YAML files.

Filter Tests¶

All observation filters in UFO are tested with the ObsFilters test from ufo/test/ufo/ObsFilters.h. Each entry in the observations list in a YAML file passed to this test should contain at least one of the following parameters:

passedBenchmark: Number of observations that should pass QC.
passedObservationsBenchmark: List of indices of observations that should pass QC.
failedBenchmark: Number of observations that should not pass QC.
failedObservationsBenchmark: List of indices of observations that should not pass QC.
flaggedBenchmark: Number of observations whose QC flag should be set to the value specified in the YAML option benchmarkFlag. Useful to isolate the impact of a filter executed after other filters that also modify QC flags.
failedObservationsBenchmark: List of indices of observations whose QC flag should be set to the value specified in the YAML option benchmarkFlag.
compareVariables: A list whose presence instructs the test to compare variables created by the filter with reference variables. Each element of the list should contain the following parameters:
- test: The variable to be tested.
- reference: The reference variable.
By default, the comparison will succeed only if all entries in the compared variables are exactly equal. If the compared variables hold floating-point numbers and the absTol option is set, the comparison will succeed if all entries differ by at most absTol. Example:
```
compareVariables:
  - test:
      name: ObsValue/windEastward
    reference:
      name: TestReference/eastward_wind
    absTol: 1e-5
  - test:
      name: ObsValue/windNorthward
    reference:
      name: TestReference/northward_wind
    absTol: 1e-5
```