Page info infoType Modified date prefix Last modified on type Flat
Contributors: ET. Carboni (UKRI-STFC RAL Space), G.E. Thomas (UKRI-STFC RAL Space)produced inUsedly (DWD)
Issued by: STFC RAL Space (UKRI-STFC) / Elisa CarboniDeutscher Wetterdienst / Tim Usedly
Date: 3101/0508/20232024
Ref: C3S2 C3S2_D312a_Lot1.2.2.4-v4.0_20230511_202408_PQAR_ECV_ERB_CCIEarthRadiationBudgetSLSTR_v1.2
Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1
...
Expand | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
|
List of datasets covered by this document
Expand | |||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||
|
Anchor | |||
---|---|---|---|
| |||
Anchor | |||
|
Expand | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
|
Acronyms
|
Acronyms
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
List of tables
|
List of tables
Expand | ||
---|---|---|
| ||
Expand | ||
| ||
Table 21-1: Summary of the TCDR and preliminary assessment of ICDR accuracy of the Earth Radiation Budget datasetTable 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the rangereference datasets used for validation: Table 1-2: Summary of requirements for OLR and RSF based on GCOS [D3] Table 4-1: Results of evaluation against GCOS targets for Earth Radiation Budget ECVs and CDR valuesrequirements for SLSTR OLR Table 4-2: Results of evaluation against GCOS requirements for SLSTR RSF |
List of figures
Expand | ||
---|---|---|
| ||
Figure 2-1: RSF and OLR from SLSTR (ICDR dataset) for March 2017Climatology of collocated, latitude-weighted global monthly means of Outgoing Longwave Radiation for SLSTR and reference datasets Figure 2-2: RSF and OLR from CERES dataset for March 2017 |
General definitions
The "CCI product family" Climate Data Record (CDR) consists of two parts. The ATSR2-AATSR Earth Radiation Budget CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived from the Sea and Land Surface Temperature Radiometer (SLSTR) on board of Sentinel-3A and -B. ICDR uses the same processing and infrastructure as the TCDR. Both TCDR and ICDR data have been produced by STFC RAL Space.
These Earth Radiation Budget datasets from polar orbiting satellites consist of two main variables:
Outgoing Longwave Radiation (OLR): The outgoing longwave flux, measured at the top of the atmosphere.
Reflected Solar radiation Flux (RSF): The reflected solar flux, measured at the top of the atmosphere.
...
Mathinline |
---|
b=\frac{\sum_{i=1}^N (p_i - r_i)}{N} \ \ (Eq. 1) |
...
Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Outgoing Longwave Radiation for SLSTR and reference datasets Figure 2-3: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CLARA-A3 dataset for the period 2019 – 2023 Figure 2-4: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CERES EBAF Ed4.2 dataset for the period 2019 – 2023 Figure 2-5: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and ERA5 dataset for the period 2019 – 2023 Figure 2-6: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and HIRS dataset for the period 2019 – 2023 Figure 2-7: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CERES SYN1deg dataset for the period 2019 – 2023 Figure 2-8: Global mean absolute bias for SLSTR OLR on the equal angle grid from 10/2018 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR Figure 2-9: Global mean absolute bias for SLSTR OLR on the equal area grid from 07/2022 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR Figure 2-10: Climatology of collocated, latitude-weighted global monthly means of Reflected Shortwave Flux for SLSTR and reference datasets Figure 2-11: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Reflected Shortwave Flux for SLSTR and reference datasets Figure 2-12: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CLARA-A3 dataset for the period 2019 – 2023 Figure 2-13: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CERES EBAF Ed4.2 dataset for the period 2019 – 2023 Figure 2-14: Yearly mean bias for Reflected Shortwave Flux of SLSTR and ERA5 dataset for the period 2019 – 2023 Figure 2-15: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CERES SYN1deg dataset for the period 2019 – 2023 Figure 2-16: Global mean absolute bias for SLSTR RSF compared to reference datasets. Reference datasets are collocated with SLSTR Figure 2-17: Global mean absolute bias for SLSTR RSF on the equal area grid from 07/2022 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR |
General definitions
Table 1: Summary of variables and definitions
Variables | Abbreviation | Definition |
Outgoing Longwave Radiation | OLR | Net total thermal radiation emitted by the Earth, as measured at the top of atmosphere. |
Reflected Solar Flux | RSF | Net total outgoing shortwave (UV, visible, near-IR) radiation at the top of atmosphere. This is dominated by reflected and scattered solar radiation. |
Table 2: Definition of various technical terms used in the document
Jargon | Definition |
TCDR | A Thematic Climate Data Record is a consistently processed time series of a geophysical variable. The time series should be |
...
Mathinline |
---|
bc- RMSE=\sqrt{\frac{\sum_{i=1}^N ((p-b)-r)^2}{N}} \ \ (Eq. 2) |
...
Stability: The variation of the bias over a multi-annual time period
Table 1: Definition of processing levels
Processing level | Definition |
Level-1b | The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid. |
Level-2 (L2) | Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b). |
Level-3C (L3C) | Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C. |
...
Jargon
Definition
Brokered product
The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS.
Climate Data Store (CDS)
The front-end and delivery mechanism for data made available through C3S.
Retrieval
A numerical data analysis scheme which uses some form of mathematical inversion to derive physical properties from some form of measurement. In this case, the derivation of cloud properties from satellite measured radiances.
Forward model
A deterministic model which predicts the measurements made of a system, given its physical properties. The forward model is the function which is mathematically inverted by a retrieval scheme. In this case, the forward model predicts the radiances measured by a satellite instrument as a function of atmospheric and surface state, and cloud properties.
TCDR
of sufficient length and quality. | |
ICDR | An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR. |
CDR
A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change.
Scope of the document
This document provides a description of the product validation results for the Climate Data record (CDR) of the Essential Climate Variable (ECV) Earth Radiation Budget. This CDR comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA's Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer ((A)ATSR) data and (ii) those produced under this contract for the Climate Data Store, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR).
The Thematic Climate Data Record (TCDR) is the product brokered from the European Space Agency Cloud Climate Change Initiative (ESA's Cloud_cci) ATSR2-AATSR version 3.0 (Level-3C) dataset. This is produced by the Science and Technology Facilities Council (STFC), RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) on board the second European Remote Sensing Satellite (ERS-2) which spanned the period 1995-2003 and the Advanced ATSR (AATSR) on board ENVISAT which spanned the period 2002-2012.
In addition, the Interim Climate Data Record (ICDR) is the product derived from the SLSTR instrument on board of Sentinel-3A and -B and spans the period from January 2017 to present. Validation for this SLSTR derived product for the period from January 2017 to March 2022 is described in this document.
Executive summary
The ESA Climate Change Initiative (CCI) Earth Radiation Budget Data Record (TCDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR) produced from the Sea and Land Surface Temperature Radiometer (SLSTR) is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. Please find further information in the Algorithm Theoretical Basis Document (ATBD) [D3].
The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018.
The TCDR and ICDR provide level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5° x 0.5°) and include these products: Outgoing Longwave Radiation (OLR) and Reflected Solar radiation Flux (RSF) at Top of Atmosphere (TOA). Table 2-1 (see section 2) provides a summary of the TCDR accuracies. For the ICDR, an initial validation with CERES (using the first 5 years of SLSTR data) show biases consistent with the TCDR: with a bias of 4.9 and 3.2 W/m² for RSF (SLSTR-A and B respectively) and -1.7 W/m² for OLR.
This document is divided in different sections:
- the first section presents a brief description of the validation methodology together with a series of references for further information;
- the second section presents the results of the validation and comparison of TCDR and ICDR data;
- the third section presents the compliance with user requirements and includes recommendation on the usage and know limitations
1. Product validation methodology
The validation methodology is described in section 2.4 of [D1]. In summary, the methodology uses the bias between the Cloud_cci product and the reference data to estimate the accuracy of the dataset. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of the TCDR compared to a reference data record, this is also known as the standard deviation about the mean. For the validation, the CDR dataset is compared with Clouds and Earth Radiation Energy System (CERES) Energy Balanced and Filled (EBAF) fluxes Edition 4.1 Top of atmosphere (TOA) (Loeb et al., 2018)1.
The stability for the TCDR dataset is defined as the variation of the bias over a multi-annual time period. It is obtained by calculating the linear trend of the bias between the TCDR and reference dataset (in this case CERES dataset).
The Product Validation and Intercomparison Report [D1] includes the validation and intercomparison of the TCDR Earth Radiation Budget versus the CERES satellite dataset. The same methodology is used for the ICDR.
Info | ||
---|---|---|
| ||
1 Data available here: https://ceres.larc.nasa.gov/data/ (Last accessed on 28/02/2023). |
...
The validation results for the TCDR are provided in [D1] section 3 and 5. Table 2-1 provides a summary of the resulting TCDR accuracies.
...
Product name | Accuracy for the TCDR [W/m²] | Accuracy for the ICDR – SLSTR-A [W/m²] | Accuracy for the ICDR – SLSTR-B [W/m²] | Accuracy for the ICDR – A+B [W/m²] |
Reflected Solar radiation Flux (RSF) | 5.72 | 3.80 | 4.19 | 4.63 |
Outgoing Longwave Radiation (OLR) | 1.72 | -0.97 | -1.32 | -1.03 |
2.1 TCDR validation with CERES satellite data
The TCDR and reference dataset are compared by calculating multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of monthly and multiannual Outgoing Longwave Radiation (OLR) and Reflected Solar radiation Flux (RSF) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points pairwise in the CERES and the Cloud_cci dataset.
The validation for Outgoing Longwave Radiation (OLR) and Reflected Solar radiation Flux (RSF) at TOA with the CERES dataset is described in section 3.3.1, 5.1 and 5.2 of [D1].
Validation of Cloud_cci radiation products with CERES present a bias of 5.72 W/m² and standard deviation of 1.64 W/m² for RSF, and a bias of 1.72 W/m² and standard deviation of 1.12 W/m² for OLR.
General findings:
RSF (from [D1] section 5.1)
- The CDR datasets show very similar patterns to the other Cloud_cci datasets of the global mean RSF. Highest mean RSF is found in the subtropics over land, lowest mean RSF is also found in the subtropics over the ocean.
- RSF show higher temporal variability over land areas.
- The time series plots of RSF show significant seasonal cycles in the global (60°S-60°N) mean with higher values in boreal winter and lower values in boreal summer.
OLR (from [D1] section 5.1)
- Mean global OLR are lowest over Antarctica and highest over the subtropics. Despite visible differences in the spatial resolution, all Cloud_cci datasets show very similar global patterns and comparable mean values.
- Stratocumulus regions are strongly pronounced with the highest TOA upwelling thermal radiation means. In case of the eastern Pacific, a strong gradient between the sea and land surface is noticeable. Stratocumulus regions around Africa and Australia show less differences between land and sea, which is probably due to the different topographic conditions.
- Higher temporal variability is found over land than over the ocean. Subtropical land areas show the largest temporal variabilities, e.g. Southeast Asia. The stratocumulus regions and southern hemispheric storm track region show the lowest temporal variability.
- Time series plots of OLR highlight a significant seasonal cycle in the global (60° S - 60° N) mean with maximum values in boreal summer and minimum values in boreal winter.
2.2 ICDR validation with CERES satellite data
The first 5 years of SLSTR products have been compared against CERES following the same methodology as described in [D1]. We estimate the bias, i.e. mean differences, and the monthly mean global average of C3S and the CERES data. To compute the monthly mean global average of both datasets we considered only the valid data between 60° S and 60° N latitude.
Validation with CERES shows biases consistent with the TCDR: with a bias of 3.8, 4.19 and 4.63 W/m² for RSF (SLSTR-A, B and combined A+B respectively) and -0.97, -1.32 and -1.03 W/m² for OLR.
Figure 2-1 and 2-2 show an example of the ICDR monthly products for March 2017 and the equivalent monthly product from CERES. These figures are for illustrative purposes so the user knows what to expect. Nonetheless, note that for this month, the ICDR and CERES datasets are spatially similar for both variables. However, there are some small differences observed. For example CERES RSF seems a little higher in the southern Indian ocean, and also slightly higher over Northern hemisphere land areas. For a more detailed analysis please go to the [D1].
...
...
3. Application(s) specific assessments
In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D5] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:
- TCDR against a stable, long-term and independent reference dataset
- ICDR against the same stable, long-term and independent reference dataset
The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D4] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:
- All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
- A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"
3.1 Results
The results of the KPI test are summarized in Table 3-1.
...
p2.5
p97.5
...
-1.15 W/m²
0.9 W/m²
...
-1.36 W/m²
1.15 W/m²
...
Sentinel-3B:
...
Percentiles were calculated based on the comparison of the TCDR using the Advanced Along Track Scanning Radiometer (AATSR) instrument against CERES as reference dataset for the variables Outgoing Longwave Radiation (OLR) and Reflected Shortwave Flux (RSF). Percentiles were based on the time from 2002-2012 with monthly means and applied to the ICDR from 01/2017 (10/2018) to 06/2022 for Sentinel-3A (Sentinel-3B and merged product Sentinel-3A+B) based on measurements of the Sea and Land Surface Temperature Radiometer (SLSTR).
A part of the ICDR months are outside the TCDR-based KPI limits and leading to “bad” KPI tests. For these, the ICDR is not stable in relation to the TCDR. This is due to multiple reasons starting with the fact of a five year gap (2012-2016) between TCDR and ICDR. In addition, TCDR and ICDR are based on different instruments with SLSTR on Sentinel-3 and (A)ATSR/ATSR-2 on Envisat/ERS-2, respectively. Differences occur due to a lower bias between ICDR and reference dataset and a subtraction of the monthly means (based on the TCDR) to remove the annual cycle leads to values outside of the KPI range (see method in [D4], chapter 3.2.2). Please note that significant changes between 01/2017 - 12/2020 and 01/2017 - 12/2021 are due to bugfixes.
4. Compliance with user requirements
There are no direct user requirements for the Earth Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Earth Radiation Budget [https://gcos.wmo.int/en/essential-climate-variables/earth-radiation/ecv-requirements] (Last accessed on 28/02/2023) the values for RSF and OLR are 1 W/m² uncertainty, while the TCDR dataset achieves an accuracy of 5.72 W/m² for RSF and 1.72 W/m² for OLR, therefore they currently do not meet the GCOS requirements. ICDR accuracies (estimate with the dataset up to march 2022) are consistent with TCDR accuracy for OLR and slightly lower for RSF. Please find more detailed information about the target requirements in the corresponding (Target Requirement and Gap Analysis Document) TRGAD [D2].
...
GCOS requirement | GCOS defines three requirements depending on user’s needs: - Goal (G): The strictest requirement, indicating no further improvements necessary - Breakthrough (B): Intermediate level between threshold and goal. Breakthrough indicates that it is recommended for certain climate monitoring activities - Threshold (T): Minimum requirement |
Brokered product | The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS. |
Climate Data Store (CDS) | The front-end and delivery mechanism for data made available through C3S. It is a platform that provides access to a wide range of climate data, including satellite and in-situ observations, reanalysis and other relevant datasets. |
Retrieval | A numerical data analysis scheme which uses some form of mathematical inversion to derive physical properties from some form of measurement. In this case, the derivation of cloud properties from satellite measured radiances. |
Forward model | A deterministic model which predicts the measurements made of a system, given its physical properties. The forward model is the function which is mathematically inverted by a retrieval scheme. In this case, the forward model predicts the radiances measured by a satellite instrument as a function of atmospheric and surface state, and cloud properties. |
Remapping | Interpolation of horizontal fields to a new, predefined grid. All datasets are remapped to the same grid (1°x1°, latitude from -90° to 90°, longitude from -180° to 180°) to make them comparable. The remap is done with bilateral interpolation. |
Collocation | A collocation consists in filtering nan values of different datasets in the same grid to make them uniform. This is necessary to compare e.g. the global average of two datasets. |
Cosine weighted averaging | Consideration of different grid box areas. Grid boxes on usual equal angle grid boxes have a different area depending on the latitude (with larger areas towards the equator). Towards the poles the same number of boxes covers a smaller area; therefore, a correction factor is needed to achieve equal area grid boxes. This factor is the cosine of the latitude. The method is applied for calculation of global averages. |
Table 3: Definition of processing levels
Processing level | Definition |
Level-1b | The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid. |
Level-2 (L2) | Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b). |
Level-3C (L3C) | Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C. |
Table 4: Definition of statistical measures used in the document
Statistical measures | Definition | ||
Bias (B) | Difference for each grid box (i,j) and time step between the dataset and reference dataset. Defined as:
with B the Bias, i, j grid box indices and F the dataset and reference dataset. | ||
Mean Bias (MB) | Mean Bias is defined as the overall bias between a dataset and reference dataset. Based on the calculated bias (resulting in a map) the global spatial and weighted average is calculated resulting in the mean bias:
with MB the Mean Bias, n and m the number of grid boxes for latitude (180) and longitude (360), w the latitude dependent factor for the cosine-weighted averaging and B the predefined Bias. | ||
Mean Absolute Bias (MAB) | Mean Absolute Bias is defined as subtracting the predefined Mean Bias from every grid box and time steps bias to remove the general bias. On a next step, the global spatial and weighted average is calculated:
with MAB the Mean Absolute Bias, n and m the number of grid boxes for latitude and longitude, w the latitude dependent factor for cosine-weighted averaging, B the predefined Bias and MB as the predefined Mean Bias. |
Scope of the document
This document provides a description of the product validation results for the Sea and Land Surface Temperature Radiometer (SLSTR) v4.0 based Interim Climate Data Record (ICDR) of the Essential Climate Variable (ECV) Earth Radiation Budget (ERB).
The dataset produced by RAL Space and Brockmann Consult (BC) under the Copernicus Climate Change Service (C3S) programme ranges from 01/2017 – 12/2023 and provides an Interim Climate Data Record (ICDR) to the brokered Thematic Climate Data Record (TCDR) from European Space Agency Cloud Climate Change Initiative (ESA’s Cloud_cci).
The TCDR is a brokered product based on processing of the (Advanced) Along-Track Scanning Radiometer ((A)TSR) onboard ERS-2 and Envisat by RAL Space for the ESA Cloud_cci programme and ranges from 06/1995 – 04/2012. Detailed validation methodology and results are presented in the Cloud_cci Product Validation and Intercomparison Report [D1].
The ICDR is derived with a five-year gap from SLSTR v4.0 onboard the Sentinel-3A and -3B satellites spanning from 01/2017 – 12/2023.
Executive summary
The Sea and Land Surface Temperature Radiometer onboard Sentinel-3A has provided data since January 2017. The launch of Sentinel-3B in October 2018 makes it possible to deliver not only individual data from both satellites but also a merged Sentinel-3A/3B product. The merged version (10/2018 - 12/2023) is validated against the following satellite-based datasets: CERES EBAF Ed4.2, CERES SYN1deg, CLARA-A3, HIRS, as well as ECMWF’s Reanalysis product ERA5. In addition to the merged SLSTR version, a second version on a different grid (equal area in addition to equal angle) is provided for the period from 07/2022 to 12/2023 and also validated against the same reference datasets as the equal angle version of SLSTR.
Validation to these SLSTR derived products is described in the following chapters of this document: Chapter 1 provides a summary of the product validation methodology while chapter 2 presents the validation results. A detailed validation methodology can be found in the Product Quality Assurance Document (PQAD) [D2]. Chapter 3 and 4 discuss possible application specific assessments and compliances with user requirements respectively.
Overall the SLSTR data meets the breakthrough/target GCOS requirement for the horizontal and temporal resolution. However, only OLR meets the threshold/target GCOS requirements in terms of accuracy, while RSF does not meet the threshold GCOS requirements (table 1); the values of the mean absolute bias that vary between 4-5 W/m² for OLR, and between 10-15 W/m² for RSF, depending on the reference, are much higher than the threshold requirement (1 W/m²).
1. Product validation methodology
Detailed information about the validation methodology can be found in the corresponding PQAD [D2], section 3. The validation process is separated into three parts: Data preparation (section 1.1), validation (section 1.2) and evaluation (1.3).
1.1 Data preparation
Table 1-1: provides a summary of the datasets used for the validation and their temporal availability, spatial- and temporal resolution.
Table 1-1: Summary of reference datasets used for validation:
Dataset | Time | Spatial resolution | Temporal resolution |
---|---|---|---|
SLSTR onboard Sentinel-3A | 01/2017 – 06/2022 | Monthly mean | 0.5°x0.5° |
SLSTR onboard Sentinel-3B | 10/2018 – 06/2022 | Monthly mean | 0.5°x0.5° |
Merged SLSTR product | 10/2018 – 12/2023 | Monthly mean | 0.5°x0.5° |
Merged SLSTR product on equal area grid | 07/2022 – 12/2023 | Monthly mean | 0.5°x0.5° |
CERES EBAF Ed. 4.2 | 01/2017 – 12/2023 | Monthly mean | 1°x1° |
CERES SYN1deg Ed. 4.1 | 01/2017 – 12/2023 | Monthly mean | 1°x1° |
HIRS OLR daily v1.2 | 01/2017 – 12/2023 | Daily mean | 1°x1° |
ERA5 | 01/2017 – 12/2023 | Monthly mean | 0.25°x0.25° |
CLARA-A3 | 01/2017 – 12/2023 | Monthly mean | 0.25°x0.25° |
All datasets are, if necessary, remapped to 1°x1° spatial resolution by bilinear interpolation and in case of HIRS monthly mean calculated based on daily means.
1.2 Validation
Following uncertainty metrics are calculated: Bias, Mean Bias and Mean Absolute Bias.
Bias is the difference of dataset and reference dataset for each month and grid box:
Mathdisplay |
---|
B_{i,j}=F_{Data,i,j}-F_{Ref,i,j} (1) |
With B as Bias and F as dataset/reference and i, j as indices. Prior to the bias calculation, the datasets are collocated and only grid point considered, where two (or more) datasets have valid values (not nan). Grid points with identical grid points set to nan for a different dataset are set to nan.
Mean Bias (MB) describes the overall bias with respect to a reference dataset. It is defined as the bias of two gridded data records and a subsequently calculation of the global spatial average. This results in one value per month which can be averaged over the whole time period.
Mathdisplay |
---|
MB=\frac{1}{m*n}*\sum_{i=1}^m \sum_{j=1}^n w_j(B_{i,j}) (2) |
with MB as Mean Bias, i and j (m and n, respectively) as indices, w as cosine weighting factor and B as Bias.
Mean Absolute Bias (MAB) is a bias corrected uncertainty metric and calculated by subtracting the previously calculated MB from every grid box bias. Subsequently the same steps as for the calculation of the mean bias are applied.
Mathdisplay |
---|
MAB=\frac{1}{m*n}*\sum_{i=1}^m \sum_{j=1}^n w_j*|B_{i,j}-MB| (3) |
1.3 Evaluation
The previously calculated Mean Absolute Bias is used as evaluation against the requirements defined by the Global Climate Observing System (GCOS) in The 2022 GCOS ECVs Requirements (GCOS 245) [D3]. They are summarized in table 1-2.
Table 1-2: Summary of requirements for OLR and RSF based on GCOS [D3]
Products | Requirement | Surface Incoming Shortwave Radiation | Surface Downwelling Longwave Radiation |
Horizontal Resolution | G | 10 km | 10 km |
B | 50 km | 50 km | |
T | 100 km | 100 km | |
Temporal Resolution | G | 1 h | 1 h |
B | 24 h | 24 h | |
T | 720 h | 720 h | |
Accuracy | G | 0.2 W/m² | 0.2 W/m² |
B | 0.5 W/m² | 0.5 W/m² | |
T | 1 W/m² | 1 W/m² |
Anchor | ||||
---|---|---|---|---|
|
Chapter 2.1 – 2.2 show the validation results for the two variables Outgoing Longwave Radiation and Reflected Shortwave Flux with a climatology of collocated, deseasonalized, centered and weighted global averages. After the collocation the seasonality of each dataset is removed from the climatology as well as the average of each dataset subtracted.
2.1 Outgoing Longwave Radiation
Figure 2-1: Climatology of collocated, latitude-weighted global monthly means of Outgoing Longwave Radiation for SLSTR and reference datasets Anchor figure2_1 figure2_1
The climatology of the global average shows identical annual cycles for SLSTR as well as all reference datasets with higher values towards the summer months and lower values in winter. ERA5 has the highest average (244.54 W/m²), followed by CERES EBAF (242.70 W/m²), CERES SYN (240.99 W/m²), CLARA-A3 (240.43 W/m²) and HIRS (240.14 W/m²). The SLSTR data record is closer to the three lowest references with an anomaly in summer 2020 leading to an average of 241.96 W/m².
Figure 2-2: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Outgoing Longwave Radiation for SLSTR and reference datasets Anchor figure2_2 figure2_2
The same data centered around the individual mean and without seasonality indicates that there is an overall good stability between the reference datasets (Figure 2-2). However, the SLSTR based data has unusual negative anomalies for the 2019 and 2021 and positive anomalies for 2020. Figures 2-3 to 2-7 show the anomalies for each year (2019 – 2023) and reference dataset.
Negative anomalies compared to ERA5 (Figure 2-5) are seen for every year on ocean and land areas between 60°N and 60°S. Except for mountainous regions (Rocky Mountains, Andes, Himalaya) and South East Asia where anomalies are positive. Outside of 60° the anomaly is positive on land areas and negative over ocean area. Similar pattern is noticed compared to CERES EBAF (Figure 2-4) with negative anomalies between 60°N and 60°S and positive anomalies at higher latitudes (especially for 2022 and 2023). Mountainous areas contain positive anomalies while there is also a clear structure of positive anomalies following the Inter Tropical Convergence Zone (ITCZ) around the equator.
CLARA-A3 (Figure 2-3), HIRS (Figure 2-6) and CERES SYN (Figure 2-7) have an overall positive bias (except for land area at the northern hemisphere) which is increasing with time. It is also noticeable that the positive bias is even more significant for mountains and the ITCZ, where SLSTR overestimates the Outgoing Longwave Radiation.
Figure 2-3: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CLARA-A3 dataset for the period 2019 – 2023 Anchor figure2_3 figure2_3
Figure 2-4: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CERES EBAF Ed4.2 dataset for the period 2019 – 2023 Anchor figure2_4 figure2_4
Figure 2-5: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and ERA5 dataset for the period 2019 – 2023 Anchor figure2_5 figure2_5
Figure 2-6: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and HIRS dataset for the period 2019 – 2023 Anchor figure2_6 figure2_6
Figure 2-7: Yearly mean bias for Outgoing Longwave Radiation of SLSTR and CERES SYN1deg dataset for the period 2019 – 2023 Anchor figure2_7 figure2_7
Figure 2-8: Global mean absolute bias for SLSTR OLR on the equal angle grid from 10/2018 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR Anchor figure2_8 figure2_8
Figures 2-8 and 2-9 show the global mean absolute bias for SLSTR compared to the reference datasets. The absolute biases show a reasonable stability within the short period of five years and range between 4.60 W/m² and 5.00 W/m². Absolute biases for the 1.5 years of the equal area grid version are slightly higher and range from 4.8 W/m² to 5.40 W/m². There is no bias between the two grid versions provided for the SLSTR data.
Figure 2-9: Global mean absolute bias for SLSTR OLR on the equal area grid from 07/2022 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR Anchor figure2_9 figure2_9
2.2 Reflected Shortwave Flux
Unlike the OLR data from SLSTR, the Reflected Shortwave Flux shows generally higher values compared to the four reference datasets (see Figure 2-10): SLSTR (105.24 W/m²), CERES EBAF (100.35 W/m²), ERA5 (99.11 W/m²), CERES SYN (98.62 W/m²) and CLARA-A3 (97.64 W/m²). Figure 2-11 shows the collocated, deseasonalized and centered datasets where the reference datasets are similar to each other while SLSTR having negative anomalies for 2019 and positive anomalies for 2020/2021. Positive anomalies occur for all years and references (compare Figures 2-12 to 2-15) and are clearly visible for mountainous regions as well as high latitudes. Only ocean areas between 30°N and 30°S have a partly negative anomaly.
Figure 2-10: Climatology of collocated, latitude-weighted global monthly means of Reflected Shortwave Flux for SLSTR and reference datasets Anchor figure2_10 figure2_10
Figure 2-11: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Reflected Shortwave Flux for SLSTR and reference datasets Anchor figure2_11 figure2_11
Figure 2-12: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CLARA-A3 dataset for the period 2019 – 2023 Anchor figure2_12 figure2_12
Figure 2-13: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CERES EBAF Ed4.2 dataset for the period 2019 – 2023 Anchor figure2_13 figure2_13
Figure 2-14: Yearly mean bias for Reflected Shortwave Flux of SLSTR and ERA5 dataset for the period 2019 – 2023 Anchor figure2_14 figure2_14
Figure 2-15: Yearly mean bias for Reflected Shortwave Flux of SLSTR and CERES SYN1deg dataset for the period 2019 – 2023 Anchor figure2_15 figure2_15
Figure 2-16: Global mean absolute bias for SLSTR RSF compared to reference datasets. Reference datasets are collocated with SLSTR Anchor figure2_16 figure2_16
Figures 2-16 and 2-17 presents the monthly absolute biases of each reference datasets compared to the SLSTR product on the equal angle/area grid. Biases for CLARA-A3 (10.03 W/m²), CERES EBAF (10.08 W/m²) and CERES SYN (10.40 W/m²) are close to each other while ERA5 has the highest bias with 13.90 W/m². Biases for the 1.5 year period for the equal angle grid are slightly higher (range from 10.84 W/m² to 14.40 W/m²), with no bias between the two different grid versions.
Figure 2-17: Global mean absolute bias for SLSTR RSF on the equal area grid from 07/2022 – 12/2023 compared to reference datasets. Reference datasets are collocated with SLSTR Anchor figure2_17 figure2_17
3. Application(s) specific assessments
This section is not applicable. There are no additional application specific assessments known since the dataset has just been published.
Anchor section4 section4
4.Compliance with user requirements
section4 | |
section4 |
The GCOS requirements [D3] for the ECV Earth Radiation Budget are used to evaluate the compliance for different users needs. Tables 4-1 and 4-2 show the requirements as well as the results.
GCOS defines three requirements depending on user’s needs:
Goal (G): The strictest requirement, indicating no further improvements necessary
Breakthrough (B): Intermediate level between threshold and goal. Breakthrough indicates that it is recommended for certain climate monitoring activities
Threshold (T): Minimum requirement
The SLSTR ICDR meets the breakthrough/target requirement (closely) for the horizontal/temporal resolution, respectively. However, the accuracy for OLR (depending on the reference between 4-5W/m²) and RSF (10-15 W/m²) do not meet the threshold requirement (1/W/m²).
It is worth mentioning, that the GCOS requirements, defined by the World Meteorological Organisation (WMO), are not focused on satellite-based data records but also on climate models. Satellite-based data records, especially historical observing systems, are often not able to achieve the requirements.
Table 4-1: Results of evaluation against GCOS Anchor table4_1 table4_1
...
Product name
...
GCOS targets
...
Cloud_cci dataset
...
RSF
...
Frequency
...
Monthly (resolving diurnal cycle)
...
Cloud_cci products do not meet the requirement for resolving the diurnal cycle.
...
Resolution
...
100 km
...
Cloud_cci products exceed the spatial resolution.
...
Measurement uncertainty
...
1 W/m² on global mean
...
Uncertainty: 5.72 W/m²
Standard Deviation: 1.64 W/m² on global mean
...
Stability
...
0.3 W/m²/decade
...
-0.15 W/m²/decade (Validation with CERES)
...
OLR
...
Frequency
...
Monthly (resolving diurnal cycle)
...
Cloud_cci products do not meet the requirement for resolving the diurnal cycle.
...
Resolution
...
100 km
...
Cloud_cci products exceed the spatial resolution.
...
Measurement uncertainty
...
1 W/m² on global mean
...
Uncertainty: 1.72 W/m²
Standard Deviation: 1.12 W/m² on global mean
...
Stability
...
0.2 W/m²/decade
...
-0.52 W/m²/decade (Validation with CERES)
Known limitations [From D1 table 7.1]:
- Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving Cloud Optical Thickness (COT) and Cloud Particle Effective Radius (CER) (input to the radiation calculation) in these conditions.
- Partly sparse temporal/spatial sampling, partly compensated by introduced diurnal cycles correction
- Somewhat higher uncertainties expected for TOA shortwave fluxes (RSF) for conditions with low clouds frequencies and elevated surface albedo uncertainties.
...
requirements for SLSTR OLR
Products | Requirement | Values | Outgoing Longwave Radiation | |
Horizontal Resolution | G | 10 km | Roughly 55 km at the equator | |
B | 50 km | |||
T | 100 km | |||
Temporal Resolution | G | 1 h | Monthly mean (720h) | |
B | 24 h | |||
T | 720 h | |||
Accuracy | G | 0.2 W/m² | Merged SLSTR product vs. reference datasets (10/2018 – 12/2023): CERES EBAF: 4.64 W/m² CERES SYN: 4.60 W/m² HIRS: 4.46 W/m² ERA5: 5.00 W/m² CLARA-A3: 4.65 W/m² | Equal area SLSTR product vs. reference datasets (07/2022 – 12/2023): SLSTR equal angle: 0.00 W/m² CERES EBAF: 5.14 W/m² CERES SYN: 5.02 W/m² HIRS: 4.86 W/m² ERA5: 5.40 W/m² CLARA-A3: 5.00 W/m² |
B | 0.5 W/m² | |||
T | 1 W/m² |
Table 4-2: Results of evaluation against GCOS requirements for SLSTR RSF Anchor table4_2 table4_2
Products | Requirement | Values | Reflected Shortwave Radiation | |
Horizontal Resolution | G | 10 km | Roughly 55 km at the equator | |
B | 50 km | |||
T | 100 km | |||
Temporal Resolution | G | 1 h | Monthly mean (720h) | |
B | 24 h | |||
T | 720 h | |||
Accuracy | G | 0.2 W/m² | Merged SLSTR product vs. reference datasets (10/2018 – 12/2023): CERES EBAF: 10.08 W/m² CERES SYN: 10.40 W/m² ERA5: 13.90 W/m² CLARA-A3: 10.03 W/m² | Equal area SLSTR product vs. reference datasets (07/2022 – 12/2023): SLSTR equal angle: 00.00 W/m² CERES EBAF: 10.84 W/m² CERES SYN: 11.00 W/m² ERA5: 14.40 W/m² CLARA-A3: 10.27 W/m² |
B | 0.5 W/m² | |||
T | 1 W/m² |
Info | ||
---|---|---|
| ||
This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S), operated by the European Centre for Medium-Range Weather Forecasts on behalf on the European Union (Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee of warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view. |
...