Cloud product generationΒΆ

In a final step, we can now apply the trained models to our data in order to generate a cloud product (area and altitudes).

For this, we load our models again:

#load our previously trained models:
import joblib

classifier = joblib.load("rf_classifier.model")
regressor  = joblib.load("regressor.model")

We define the predictors that were also used in training:

# Define the predictors
predictors    = ["IR_016","IR_039","IR_087","IR_097","IR_108","IR_120","IR_134","VIS006","VIS008","WV_062","WV_073","dem"]

Note that here both models take exactly the same predictor set. Normally predictors change between different models as you should have done a feature selection during the model training procedure which should result in different feature sets depending on the target variable.

We then have to load the DEM:

import xarray as xr

# Load DEM
dem = xr.open_dataset("data/dem_cropped.tif",engine="rasterio").band_data

And now we can load an exemplary satellite scene again which we apply the models to:

import numpy as np

# Load MSG and data into DataSet
satellite_data = xr.open_dataset("data/satellite/2006/20060315_1500.nc")

# Add DEM to satellite_data-DataSet
satellite_data["dem"] = (["time","y","x"],dem.values)

# Extract all predictor variables
satellite_data_arr = np.array([satellite_data[predictor].values for predictor in predictors])

# Flatten predictors to 1d-arrays
satellite_data_arr_reshaped = satellite_data_arr.reshape(satellite_data_arr.shape[0],-1)

Model application is just a straightforward prediction:

cloud_mask = classifier.predict(np.where(np.isnan(satellite_data_arr_reshaped),0,satellite_data_arr_reshaped).T).reshape(dem.shape[1:])
cloud_alt  = regressor.predict(np.where(np.isnan(satellite_data_arr_reshaped),0,satellite_data_arr_reshaped).T).reshape(dem.shape[1:])
cloud_mask = np.where(np.isnan(satellite_data.IR_039[0]),np.nan,cloud_mask)
cloud_alt = np.where(np.isnan(satellite_data.IR_039[0]),np.nan,cloud_alt)

And to get a view of our result, we can visualize it (interactively) again:

import matplotlib.pyplot as plt

# Plotting method for creating a plot of the model predictions and the original satellite data
def plotResult(switch1, switch2):
    
    r = normArray(satellite_data.IR_039[0]*-1)
    g = normArray(satellite_data.IR_108[0]*-1)
    b = normArray(satellite_data.IR_120[0]*-1)

    fig, ax = plt.subplots(1,1,figsize=(12,8))
    ax.imshow(np.array([r,g,b]).transpose(1,2,0))
    
    if switch1: 
        ax.imshow(np.where(cloud_mask==1,1,np.nan),cmap="autumn_r",interpolation="None")
    if switch2: 
        im = ax.imshow(np.where(cloud_mask,cloud_alt,np.nan),cmap="jet",interpolation="None")
        plt.colorbar(im,fraction=0.046, pad=0.04)
    
    ax.set_title("Cloud product")
    fig.tight_layout()
    plt.show()
from ipywidgets import interact, Layout
import ipywidgets as widgets

interact(plotResult,
         switch1     = widgets.Checkbox(value=False,description='Cloud-Mask'),
         switch2     = widgets.Checkbox(value=False,description='Cloud-altitudes'));

The graph is not interactive on this website as it only works in a running Jupyter Environment.

In order to generate a cloud product, we would now apply the models to each scene in our data set and save the result.