FogFusionNet: Coastal Sea Fog Prediction by Using a Multimodal Deep Learning Approach

Abstract

In this study, we designed FogFusionNet, a multimodal sea fog prediction model, that used closed-circuit television (CCTV) images and multivariate time series observation (MTSO) data to predict three visibility classes—Normal visibility, Low visibility, and Sea fog—at 1-h intervals from the current time to 6-h in the future for a specific region. We applied weighted sampling and weighted loss to overcome the imbalance of each visibility class, and additionally evaluated the effect of replacing missing MTSO data. A total of 4 years of data regarding Incheon Port, which faces the Yellow Sea and is prone to sea fog, were collected for training and verifying FogFusionNet. Of these, 3 years of data was used for training FogFusionNet, and the remaining 1 year of data were used for verifying the performance of FogFusionNet. The prediction performance of FogFusionNet at 1-h intervals was 86.2% (0-h), 79.1% (1-h), 73.4% (2-h), 70.7% (3-h), 64.7% (4-h), 59.6% (5-h), and 49.3% (6-h), showing an average prediction performance of 69.0%. FogFusioneNet is expected to promote coastal safety and reduce economic losses due to coastal sea fog.

Keywords

• Closed-circuit television (CCTV) images
• coastal sea fog prediction
• multimodal learning
• multivariate time series observation data (MTSO)
• visibility class