Data¶
The satellite and station data for this case study can be downloaded here:
Read netCDF files (satellite data)¶
The MSG data is provided in netcdf-Format and can be read into python via:
import xarray as xr
satellite_data = xr.open_dataset("path/to/file.nc")
The data is structured in a DataSet with multiple DataArrays, each representing one of the MSG SEVIRI bands. A cloud mask is also provided here. However, we only want to use this cloud mask for validation purposes and not for our training.
satellite_data
<xarray.Dataset>
Dimensions: (time: 1, x: 1024, y: 768)
Coordinates:
* time (time) datetime64[ns] 2005-03-15T15:00:09.743000
* x (x) float64 -1.184e+06 -1.181e+06 ... 1.883e+06 1.886e+06
* y (y) float64 5.405e+06 5.402e+06 5.399e+06 ... 3.107e+06 3.104e+06
acq_time (y) datetime64[ns] 2005-03-15T15:12:40.449000 ... 2005-03-15T15...
Data variables:
IR_016 (time, y, x) float32 ...
IR_039 (time, y, x) float32 ...
IR_087 (time, y, x) float32 ...
IR_097 (time, y, x) float32 ...
IR_108 (time, y, x) float32 ...
IR_120 (time, y, x) float32 ...
IR_134 (time, y, x) float32 ...
VIS006 (time, y, x) float32 ...
VIS008 (time, y, x) float32 ...
WV_062 (time, y, x) float32 ...
WV_073 (time, y, x) float32 ...
cmask (time, y, x) uint8 ...
Attributes:
comment: This data set was clipped to the LCRS-Domain and preprocess...
instrument: seviri
history: Created by pytroll/satpy on 2021-06-24 11:15:03.165717
Conventions: CF-1.7xarray.Dataset
- time: 1
- x: 1024
- y: 768
- time(time)datetime64[ns]2005-03-15T15:00:09.743000
array(['2005-03-15T15:00:09.743000000'], dtype='datetime64[ns]')
- x(x)float64-1.184e+06 -1.181e+06 ... 1.886e+06
array([-1183659.29504 , -1180658.891761, -1177658.488482, ..., 1879752.452392, 1882752.85567 , 1885753.258949]) - y(y)float645.405e+06 5.402e+06 ... 3.104e+06
array([5405226.708003, 5402226.304724, 5399225.901446, ..., 3109918.199889, 3106917.79661 , 3103917.393331]) - acq_time(y)datetime64[ns]...
- long_name :
- Mean scanline acquisition time
array(['2005-03-15T15:12:40.449000000', '2005-03-15T15:12:40.246707895', '2005-03-15T15:12:40.044415790', ..., '2005-03-15T15:10:05.695540016', '2005-03-15T15:10:05.493247911', '2005-03-15T15:10:05.290955807'], dtype='datetime64[ns]')
- IR_016(time, y, x)float32...
- calibration :
- reflectance
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- sunz_corrected
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_bidirectional_reflectance
- start_time :
- 2005-03-15 15:00:09.743000
- sun_earth_distance_correction_applied :
- true
- sun_earth_distance_correction_factor :
- 0.9890210339263187
- units :
- %
- wavelength :
- 1.64 µm (1.5-1.78 µm)
[786432 values with dtype=float32]
- IR_039(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 3.92 µm (3.48-4.36 µm)
[786432 values with dtype=float32]
- IR_087(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 8.7 µm (8.3-9.1 µm)
[786432 values with dtype=float32]
- IR_097(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 9.66 µm (9.38-9.94 µm)
[786432 values with dtype=float32]
- IR_108(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 10.8 µm (9.8-11.8 µm)
[786432 values with dtype=float32]
- IR_120(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 12.0 µm (11.0-13.0 µm)
[786432 values with dtype=float32]
- IR_134(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 13.4 µm (12.4-14.4 µm)
[786432 values with dtype=float32]
- VIS006(time, y, x)float32...
- calibration :
- reflectance
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- sunz_corrected
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_bidirectional_reflectance
- start_time :
- 2005-03-15 15:00:09.743000
- sun_earth_distance_correction_applied :
- true
- sun_earth_distance_correction_factor :
- 0.9890210339263187
- units :
- %
- wavelength :
- 0.635 µm (0.56-0.71 µm)
[786432 values with dtype=float32]
- VIS008(time, y, x)float32...
- calibration :
- reflectance
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- sunz_corrected
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_bidirectional_reflectance
- start_time :
- 2005-03-15 15:00:09.743000
- sun_earth_distance_correction_applied :
- true
- sun_earth_distance_correction_factor :
- 0.9890210339263187
- units :
- %
- wavelength :
- 0.81 µm (0.74-0.88 µm)
[786432 values with dtype=float32]
- WV_062(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 6.25 µm (5.35-7.15 µm)
[786432 values with dtype=float32]
- WV_073(time, y, x)float32...
- calibration :
- brightness_temperature
- end_time :
- 2005-03-15 15:12:40.449000
- georef_offset_corrected :
- false
- grid_mapping :
- seviri_0deg
- modifiers :
- []
- orbital_parameters :
- {"projection_longitude": 0.0, "projection_latitude": 0.0, "projection_altitude": 35785831.0, "satellite_nominal_longitude": -3.4000000953674316, "satellite_nominal_latitude": 0.0, "satellite_actual_longitude": -3.578777793630411, "satellite_actual_latitude": 0.03826690128997061, "satellite_actual_altitude": 35783988.94155065}
- platform_name :
- Meteosat-8
- resolution :
- 3000.403165817
- satellite_altitude :
- 35785831.0
- satellite_latitude :
- 0.0
- satellite_longitude :
- 0.0
- sensor :
- seviri
- standard_name :
- toa_brightness_temperature
- start_time :
- 2005-03-15 15:00:09.743000
- units :
- K
- wavelength :
- 7.35 µm (6.85-7.85 µm)
[786432 values with dtype=float32]
- cmask(time, y, x)uint8...
- Description :
- CM-SAF Cloud mask
- Units :
- 0 = non-processed, 1 = cloud-free, 2 = cloud-contaminated, 3 = cloud-filled, 4 = snow/ice, 5 = undefined
[786432 values with dtype=uint8]
- comment :
- This data set was clipped to the LCRS-Domain and preprocessed (sun-zenith angle corrected,normalized reflectance values) from the original MSG data set provided by EUMETSAT.For details of what has been done see the "modifiers" attribute of individual variables.The Processing script used for this data set is stored here:https://gitlab.com/lcrsmarburg/meteosat_scripts/-/blob/master/preprocessing/preprocess_MSG.py
- instrument :
- seviri
- history :
- Created by pytroll/satpy on 2021-06-24 11:15:03.165717
- Conventions :
- CF-1.7
Read GeoTiffs (DEM)¶
A digital elevation model (DEM) is provided as GeoTiff and can be opened via:
dem = xr.open_dataset("path/to/file.tif",engine="rasterio").band_data[0]
Read CSVs (station data)¶
METAR station measurements and meta data (with station location information) are provided in csv-files which can be opened via:
import pandas as pd
station_data = pd.read_csv("path/to/file.csv",parse_dates=["time"])
station_meta_data = pd.read_csv("path/to/file.csv")
station_data
| icao | time | cloudcover | cloud_altitude | |
|---|---|---|---|---|
| 0 | EBAW | 2005-01-15 00:00:00 | 3 | 2400 |
| 1 | EBBR | 2005-01-15 00:00:00 | 3 | 3600 |
| 2 | EBCI | 2005-01-15 00:00:00 | 3 | 2200 |
| 3 | EBLG | 2005-01-15 00:00:00 | 1 | -999 |
| 4 | EBOS | 2005-01-15 00:00:00 | 3 | 2600 |
| ... | ... | ... | ... | ... |
| 52007 | LZKZ | 2006-12-15 21:00:00 | 3 | 3300 |
| 52008 | LZPP | 2006-12-15 21:00:00 | 3 | 3500 |
| 52009 | LZSL | 2006-12-15 21:00:00 | 3 | 3300 |
| 52010 | LZTT | 2006-12-15 21:00:00 | 1 | -999 |
| 52011 | LZZI | 2006-12-15 21:00:00 | 3 | 2400 |
52012 rows × 4 columns
Columns are encoded as follows:
icao: ID of METAR station
time: Time of measurement (YYYYmmddHHMM)
cloudcover: Cloud cover (1: cloud free, 2: cloud contaminated, 3 cloud covered)
cloud_altitude: Cloud altitude in meters above ground (missing data value: -999)
station_meta_data
| icao | x | y | |
|---|---|---|---|
| 0 | EBAW | 2.920539e+05 | 4.611705e+06 |
| 1 | EBBR | 2.956507e+05 | 4.596070e+06 |
| 2 | EBCI | 2.967015e+05 | 4.571825e+06 |
| 3 | EBLG | 3.608589e+05 | 4.580852e+06 |
| 4 | EBOS | 1.875154e+05 | 4.613160e+06 |
| ... | ... | ... | ... |
| 268 | LZKZ | 1.430620e+06 | 4.434616e+06 |
| 269 | LZPP | 1.212834e+06 | 4.443047e+06 |
| 270 | LZSL | 1.296992e+06 | 4.439913e+06 |
| 271 | LZTT | 1.354894e+06 | 4.461921e+06 |
| 272 | LZZI | 1.246986e+06 | 4.476102e+06 |
273 rows × 3 columns
Columns are encoded as follows:
icao: ID of METAR station
x: x-Coordinate of station location in GEOS projection (MSG)
y: y-Coordinate of station location in GEOS projection (MSG)
Task¶
Download all data sets and make yourselves familiar with the data