Talk:Multimodal learning

Untitled

I think stacks of Boltzmann machines are usually called Deep Belief Networks, not Deep Boltzmann Machines. The article also does not discuss stacks of autoencoders. 194.117.26.63 (talk) 16:17, 9 May 2016 (UTC)

Why is this page basically entirely about Boltzmann machines? Enervation (talk) 01:01, 22 December 2023 (UTC)

Good question. This article should be modernized. It would perhaps also be worth having a separate article on large multimodal models. Alenoach (talk) 21:06, 27 February 2024 (UTC)

Potential renaming

Should the article be renamed to something like "Multimodality (machine learning)"? It's a bit of a lengthier title, but it seems more precise, since the focus here is not just on the learning anymore (the article has been pretty significantly modified). And when i make a Google search on "Multimodal learning", a lot of the articles seem to rather be related to education and how to teach students. Alenoach (talk) 19:14, 26 May 2024 (UTC)

The title "Multimodal machine learning" could be another option. Alenoach (talk) 19:18, 26 May 2024 (UTC)

Add a Challenges section

The current article discusses specific architectures and applications but lacks the challenges in multimodal learning. I suggest adding a short section on these challenges, perhaps before "Applications". I am a co-author of one source below, so I would prefer not to make the change directly. Could any editor consider whether the following content would be useful? Thanks for taking a look.

Challenges

Multimodal learning involves challenges in representing, aligning and combining different forms of information. These have been categorised as representation, translation, alignment, fusion and co-learning, with a later taxonomy covering representation, alignment, reasoning, generation, transference and quantification. In practice, real-world applications face additional specific challenges, including missing modalities, differences in data formats and structures across modalities, aligning information across modalities, combining complementary information without unnecessary redundancy, and privacy risks such as re-identification when multiple data sources are combined. Xyl007 (talk) 15:20, 21 August 2026 (UTC)

Hi Xyl007. A "Challenges" section could indeed be an improvement and your sourcing looks fine, but it's challenging to understand, particularly the second sentence. I expect that most readers likely have basic ML knowledge but don't understand what "transference", "alignment", "translation" and "quantification" mean in this context. It could for example be clearer if instead of having a second sentence that presents two extra lists of concepts that partially overlap with the one of the first sentence, there was some explanation of one or two basic concepts in simple terms.

A section like "Challenges" is typically added near the end of the article, before or after the section "Applications". Feel free to add your section once you think it's ready, and thanks for your contribution. Alenoach (talk) 20:49, 21 August 2026 (UTC)

Hi Alenoach, thanks for your suggestions. I have simplified the content to make it more readable, as below. I will add this section, but please feel free to edit it further.

Multimodal learning is challenging because information can be represented in different formats or types across modalities. Two common challenges are alignment, determining which parts of different modalities correspond to each other, and fusion, determining how information across modalities should be combined. Real-world applications face further challenges specific to multimodal AI, including missing modalities when a data source is unavailable, selecting modalities that provide complementary rather than redundant information, and privacy risks from combining data sources, which may allow individuals to be re-identified even if they are anonymous in each source separately. xxx007 (talk) 11:20, 23 August 2026 (UTC)

Looks good, thanks for your effort. Alenoach (talk) 14:36, 23 August 2026 (UTC)

References