Talk:Catastrophic interference
Add topicLatest comment: 9 days ago by Packrat3860 in topic Software List
| This is the talk page for discussing improvements to the Catastrophic interference article. This is not a forum for general discussion of the subject of the article. |
Article policies
|
| Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL |
 | This article is rated C-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects: | |||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||
Activation overlap (Node Sharpening Technique section)
[edit]For example, if the activations at the hidden layer from one input are (0.3, 0.1, 0.9, 1.0) and the activations from the next input are (0.0, 0.9, 0.1, 0.9) the activation overlap would be (0.3 + 0.1 + 0.1 + 0.9 )/ 4 = 0.35.
Shouldn't that be (0.0 + 0.1 + 0.1 + 0.9 )/ 4, or is the 0.0 in the "next input" special cased? If so, perhaps some discussion of why would be warranted, as it's not immediately apparent to me why it would be. -- 160.129.138.186 (talk) 18:58, 27 May 2014 (UTC)
Neutral point of view violated?
[edit]The article explains neural networks as "network approach and connectionist approach to cognitive science". Neural networks are much more than that. This and the subsequent text feels like editorial bias and not an objective, Wikipedia-like overview of the subject.ThisFeelsABitOff (talk) 00:58, 24 October 2021 (UTC)
- @ThisFeelsABitOff: Neural networks are among the topics that are studied in cognitive science. In what way is the article biased? Jarble (talk) 19:17, 13 January 2022 (UTC)
Did GPT 3.5 (Assistant, ChatGPT) solved the problem?
[edit]It appears they slightly retrain it on user input. 2A00:1370:8184:9B6:8AA3:B0D:7D41:F644 (talk) 07:44, 21 January 2023 (UTC)
Software List
[edit] | This edit request by an editor with a conflict of interest was declined. A reviewer felt that this edit would not improve the article. |
Hi Wikipedia editor I would like to add a list of continual learning software projects. I am a maintainer of the CapyMOA project and have contributed to the Avalanche project in the past.
My requested edit would look something like (OCL is online continual learning):
| Software | Pypi | Initial Release | Latest Release | Licence | FOSS |
|---|---|---|---|---|---|
| Avalanche[1] | avalanche-lib | Dec 17, 2021 | Oct 29, 2024 | MIT License | Yes |
| PyCIL[2] | Dec 12, 2022 | Jul 14, 2023 | MIT License | Yes | |
| AlbinSou's OCL survey code[3] | Apr 26, 2023 | Apr 8, 2024 | MIT License | Yes | |
| RaptorMai's OCL survey code[4] | Jan 30, 2021 | Nov 6, 2021 | All rights reserved | No | |
| FACIL[5] | Sep 22, 2020 | May 27, 2023 | MIT License | Yes | |
| CapyMOA's OCL[6] | capymoa | Apr 29, 2024 | Aug 9, 2026 | BSD 3-Clause License | Yes |
TachyonicClock (talk) 00:44, 14 September 2026 (UTC)
References
- ↑ Carta, Antonio; Pellegrini, Lorenzo; Cossu, Andrea; Hemati, Hamed; Lomonaco, Vincenzo (2023). "Avalanche: A PyTorch Library for Deep Continual Learning". Journal of Machine Learning Research. 24 (363): 1--6.
- ↑ Zhou, Da-Wei; Wang, Fu-Yun; Ye, Han-Jia; Zhan, De-Chuan (2023). "PyCIL: a Python toolbox for class-incremental learning". Science China Information Sciences. 66 (9): 197101, s11432–022–3600-y. doi:10.1007/s11432-022-3600-y. ISSN 1674-733X.
- ↑ Soutif–Cormerais, Albin; Carta, Antonio; Cossu, Andrea; Hurtado, Julio; Lomonaco, Vincenzo; Van De Weijer, Joost; Hemati, Hamed (2023). "A Comprehensive Empirical Evaluation on Online Continual Learning". 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW): 3510–3520. doi:10.1109/ICCVW60793.2023.00378. ISSN 2473-9944.
- ↑ Mai, Zheda; Li, Ruiwen; Jeong, Jihwan; Quispe, David; Kim, Hyunwoo; Sanner, Scott (2022-01-16). "Online continual learning in image classification: An empirical survey". Neurocomputing. 469: 28–51. doi:10.1016/j.neucom.2021.10.021. ISSN 0925-2312.
- ↑ Masana, Marc; Liu, Xialei; Twardowski, Bartłomiej; Menta, Mikel; Bagdanov, Andrew D.; van de Weijer, Joost (2023-05-01). "Class-Incremental Learning: Survey and Performance Evaluation on Image Classification". IEEE Transactions on Pattern Analysis and Machine Intelligence. 45 (5): 5513–5533. doi:10.1109/TPAMI.2022.3213473. ISSN 0162-8828.
- ↑ Gomes, Heitor Murilo; Lee, Anton; Gunasekara, Nuwan; Sun, Yibin; Cassales, Guilherme Weigert; Liu, Justin; Heyden, Marco; Cerqueira, Vitor; Bahri, Maroua (2025-02-11). "CapyMOA: Efficient Machine Learning for Data Streams and Online Continual Learning in Python". arXiv.org. Retrieved 2026-09-14.
Response: 2026-09-21
[edit]
Not done: It's not clear where this table is even supposed to go or in what way it relates to the existing text of the article. Packrat3860 (talk) 03:39, 21 September 2026 (UTC)
- Sorry, I didn't provide enough context. For context, continual learning (online continual learning is a special case) is the field of study that addresses the catastrophic interference problem. Several research and software projects now exist to help resolve the catastrophic interference problem. This is a list of notable software/surveys. It should fit in its own section toward the bottom of the article. TachyonicClock (talk) 03:51, 21 September 2026 (UTC)
For context, continual learning (online continual learning is a special case) is the field of study that addresses the catastrophic interference problem.
- Nothing in the article currently makes this connection. I think this list would be out of place to begin with, but there's nothing right now explaining what it has to do with the rest of the article at all. Packrat3860 (talk) 05:07, 21 September 2026 (UTC)
- So I have some options. Do you have a suggestion?
- A) Make this connection clear, e.g. add something like this in the introduction:
- Continual learning (also known as lifelong learning) [1,2] studies how to overcome the catastrophic interference problem in artificial neural networks.
- Parisi, German I., Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. “Continual Lifelong Learning with Neural Networks: A Review.” Neural Networks 113 (May 2019): 54–71. https://doi.org/10.1016/j.neunet.2019.01.012.
- Wang, Liyuan, Xingxing Zhang, Hang Su, and Jun Zhu. “A Comprehensive Survey of Continual Learning: Theory, Method and Application.” IEEE Transactions on Pattern Analysis and Machine Intelligence 46, no. 8 (2024): 5362–83. https://doi.org/10.1109/TPAMI.2024.3367329.
- B) Add the software list somewhere more appropriate and keep Catastrophic Interference based only on the forgetting mechanism.
- Note that Wikipedia's article on Continual learning currently points to Incremental learning, since continual learning is a type of incremental learning. But, since the field is defined more by its opposition to catastrophic forgetting/interference than by its incremental nature I believe Catastrophic interference is the better article to discuss continual learning. Alternatively Continual learning could be promoted to its own article (like on the IBM encyclopedia).
- My recommendation would be to give continual learning its own article. My understanding is I'll only need to go through the conflict of interest process again if discussing my own or close colleagues work? TachyonicClock (talk) 23:03, 21 September 2026 (UTC)
- I really don't know all that much about this field. If you'd like to do some research and propose a section on how continual learning relates to catastrophic interference (with sources of course) I'd be happy to take a look or help you find someone that can if it's really deep in the weeds technically. But I don't have much of an opinion on where it fits in here. As long as you WP:STICKTOTHESOURCE we can find a home. Packrat3860 (talk) 01:05, 24 September 2026 (UTC)