Bezirganyan G., Sellami S., Berti-Equille Laure, Fournier S. (2023). M2-Mixer : a multimodal mixer with multi-head loss for classification from multimodal data. In :
He J. (ed.), Palpanas T. (ed.), Cuzzocrea A. (ed.), Dou D. (ed.), Slezak D. (ed.), Wang W. (ed.), Gruca A. (ed.), Chun-Wei J. (ed.), Agrawal R. (ed.). 2023 IEEE International Conference on Big Data. Piscataway : IEEE, 1052-1058. IEEE International Conference on Big Data, Sorrento (ITA), 2023/12/15-2023/12/18. ISBN 979-8-3503-2446-4.
Titre du document
M2-Mixer : a multimodal mixer with multi-head loss for classification from multimodal data
Année de publication
2023
Type de document
Partie d'ouvrage
Auteurs
Bezirganyan G., Sellami S., Berti-Equille Laure, Fournier S.
In
He J. (ed.), Palpanas T. (ed.), Cuzzocrea A. (ed.), Dou D. (ed.), Slezak D. (ed.), Wang W. (ed.), Gruca A. (ed.), Chun-Wei J. (ed.), Agrawal R. (ed.) 2023 IEEE International Conference on Big Data
Source
Piscataway : IEEE, 2023,
1052-1058 ISBN 979-8-3503-2446-4
Colloque
IEEE International Conference on Big Data, Sorrento (ITA), 2023/12/15-2023/12/18
In this paper, we propose M2-Mixer, an MLP-Mixer based architecture with multi-head loss for multimodal classification. It achieves better performances than the convolutional, recurrent, or neural architecture search based baseline models with the main advantage of conceptual and computational simplicity. The proposed multi-head loss function addresses the problem of modality predominance (i.e., when one of the modalities is favored over the others by the training algorithm). Our experiments demonstrate that our multimodal mixer architecture, combined with the multi-head loss function, outperforms the baseline models on two benchmark multimodal datasets: AVMNIST and MIMIC-III with respectively, on average, + 0.43% in accuracy and 6. 4 times reduction in training time and + 0.33% in accuracy and 13. 3 times reduction in training time, compared with previous best performing models.
Plan de classement
Informatique [122]
Localisation
Fonds IRD [F B010090983]
Identifiant IRD
fdi:010090983
