Accéder directement au contenu Accéder directement à la navigation
Communication dans un congrès

Exploration of Deep Learning-based Multimodal Fusion for Semantic Road Scene Segmentation

Yifei Zhang 1 Olivier Morel 1 Marc Blanchon 1 Ralph Seulin 1 Mojdeh Rastgoo 1 Désiré Sidibé 1
1 VIBOT - Equipe VIBOT - VIsion pour la roBOTique [ImViA EA7535 - ERL CNRS 6000]
CNRS - Centre National de la Recherche Scientifique : ERL 6000, ImViA - Imagerie et Vision Artificielle [Dijon]
Abstract : Deep neural networks have been frequently used for semantic scene understanding in recent years. Effective and robust segmentation in outdoor scene is prerequisite for safe autonomous navigation of autonomous vehicles. In this paper, our aim is to find the best exploitation of different imaging modalities for road scene segmentation, as opposed to using a single RGB modality. We explore deep learning-based early and later fusion pattern for semantic segmentation, and propose a new multi-level feature fusion network. Given a pair of aligned multimodal images, the network can achieve faster convergence and incorporate more contextual information. In particular, we introduce the first-of-its-kind dataset, which contains aligned raw RGB images and polarimetric images, followed by manually labeled ground truth. The use of polarization cameras is a sensory augmentation that can significantly enhance the capabilities of image understanding, for the detection of highly reflective areas such as glasses and water. Experimental results suggest that our proposed multimodal fusion network outperforms unimodal networks and two typical fusion architectures.
Type de document :
Communication dans un congrès
Liste complète des métadonnées

Littérature citée [27 références]  Voir  Masquer  Télécharger
Contributeur : Désiré Sidibé <>
Soumis le : jeudi 7 mars 2019 - 11:43:59
Dernière modification le : mardi 3 mars 2020 - 15:25:23
Archivage à long terme le : : samedi 8 juin 2019 - 14:27:12


2019_yifei_visapp .pdf
Fichiers produits par l'(les) auteur(s)




Yifei Zhang, Olivier Morel, Marc Blanchon, Ralph Seulin, Mojdeh Rastgoo, et al.. Exploration of Deep Learning-based Multimodal Fusion for Semantic Road Scene Segmentation. VISAPP 2019 14th International Conference on Computer Vision Theory and Applications, Feb 2019, Prague, Czech Republic. ⟨10.5220/0007360403360343⟩. ⟨hal-02060222⟩



Consultations de la notice


Téléchargements de fichiers