{"id":1809,"date":"2025-07-12T14:23:49","date_gmt":"2025-07-12T14:23:49","guid":{"rendered":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=1809"},"modified":"2025-07-12T15:20:27","modified_gmt":"2025-07-12T15:20:27","slug":"rejection-of-mamba-by-prestigious-conferences","status":"publish","type":"post","link":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=1809","title":{"rendered":"Rejection of Mamba by prestigious conferences"},"content":{"rendered":"\n<figure class=\"wp-block-audio\"><audio controls src=\"http:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/How-Mamba-Positions-itself-as-a-competitor-to-the-Transformers-AI-Architecture.mp3\"><\/audio><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>PODCAST: <a href=\"https:\/\/notebooklm.google.com\/notebook\/17336321-6b43-4817-952f-388f4c30b4d8\">review of a failed Peer Review<\/a>. The Mamba paper was submitted to <strong>ICLR 2024<\/strong>, which is described as one of the most prestigious machine learning conferences in the world. In January, the paper was <strong>rejected by peer reviewers<\/strong>. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-opt-id=150640642  fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"692\" src=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:1024\/h:692\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/image-285.png\" alt=\"\" class=\"wp-image-1813\" srcset=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:1024\/h:692\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/image-285.png 1024w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:300\/h:203\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/image-285.png 300w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:768\/h:519\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/image-285.png 768w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:1341\/h:906\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/07\/image-285.png 1341w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Explore how Mamba positions itself as a competitor to the Transformers AI Architecture. The provided source introduces <strong>Mamba<\/strong>, a novel neural network architecture that significantly <strong>outperforms transformers in language modeling<\/strong>, especially at smaller model sizes, while <strong>requiring less computational power<\/strong>. It explains that Mamba&#8217;s efficiency stems from its <strong>linear O(n log(n)) compute complexity<\/strong> compared to the transformer&#8217;s quadratic O(n^2), enabling larger context sizes. The text details how Mamba is an <strong>extension of recurrent neural networks (RNNs)<\/strong>, overcoming their historical limitations (slow parallelization and training difficulties) by employing <strong>linear recurrences, a specialized parallel scan algorithm, and a unique initialization strategy<\/strong> that addresses vanishing and exploding gradients. Finally, it highlights the <strong>controversial rejection of the Mamba paper from a prestigious conference<\/strong>, attributing it to reviewers&#8217; apparent misunderstandings and misinterpretations of the architecture and its evaluation.<\/p>\n\n\n\n<p>ICLR 2024 refers to the <strong>12th International Conference on Learning Representations<\/strong>, which took place in Vienna, Austria from May 7th to May 11th, 2024. It is a leading conference in the field of machine learning, particularly deep learning. The conference featured a main conference with paper presentations, workshops, and invited talks. [<a href=\"https:\/\/dblp.org\/db\/conf\/iclr\/iclr2024#:~:text=The%20Twelfth%20International%20Conference%20on%20Learning%20Representations%2C%20ICLR%202024%2C%20Vienna%2C%20Austria%2C%20May%207%2D11%2C%202024.\">1<\/a>, <a href=\"https:\/\/www.climatechange.ai\/events\/iclr2024\">2<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/International_Conference_on_Learning_Representations#:~:text=The%20International%20Conference%20on%20Learning%20Representations%20(ICLR),in%20machine%20learning%20and%20artificial%20intelligence%20research.\">3<\/a>, <a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForWorkshops#:~:text=Update:%20The%20list%20of%20accepted%20workshops%20can,and%20moderating%20the%20program%20throughout%20the%20workshop.\">4<\/a>, <a href=\"https:\/\/iclr.cc\/virtual\/2024\/eventlistwithbios\/InvitedTalks-2024\">5<\/a>]<\/p>\n\n\n\n<p>This video provides an overview of ICLR 2024: <a href=\"https:\/\/m.youtube.com\/shorts\/uiZWo1c2Fr4\">https:\/\/m.youtube.com\/shorts\/uiZWo1c2Fr4<\/a><\/p>\n\n\n\n<p>Key Aspects of ICLR 2024:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Call for Papers:<\/strong> The conference had a call for papers with abstract submission deadlines in September 2023 and final decisions in January 2024. [<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForPapers\">7<\/a>]<\/li>\n\n\n\n<li><strong>Workshops:<\/strong> A variety of workshops were held on May 11th, covering diverse topics like Tackling Climate Change with Machine Learning, and the 5th Workshop on African Natural Language Processing (AfricaNLP 2024). [<a href=\"https:\/\/www.climatechange.ai\/events\/iclr2024\">2<\/a>, <a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForWorkshops#:~:text=Update:%20The%20list%20of%20accepted%20workshops%20can,and%20moderating%20the%20program%20throughout%20the%20workshop.\">4<\/a>, <a href=\"https:\/\/iclr.cc\/Downloads\/2024\">8<\/a>]<\/li>\n\n\n\n<li><strong>Invited Talks:<\/strong> The conference included invited talks, one of which focused on the &#8220;emerging science of benchmarks&#8221;. [<a href=\"https:\/\/iclr.cc\/virtual\/2024\/eventlistwithbios\/InvitedTalks-2024\">5<\/a>]<\/li>\n\n\n\n<li><strong>Virtual Component:<\/strong> ICLR 2024 had a virtual component accessible through the conference website, including schedule information, paper browsing, and search functionality. [<a href=\"https:\/\/iclr.cc\/virtual\/2024\/index.html\">9<\/a>]<\/li>\n\n\n\n<li><strong>Sponsors and Exhibitors:<\/strong> The conference had various sponsors and exhibitors, including Cambridge University Press and Intelligent Computing. [<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/Sponsors\">10<\/a>]<\/li>\n\n\n\n<li><strong>Tiny Papers Track:<\/strong> A special track for &#8220;Tiny Papers&#8221; was also organized, encouraging shorter, more focused research papers. [<a href=\"https:\/\/blog.iclr.cc\/category\/iclr-2024\/\">11<\/a>, <a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForTinyPapers\">12<\/a>]<\/li>\n<\/ul>\n\n\n\n<p>Overall, ICLR 2024 was a significant event for the machine learning community, showcasing cutting-edge research and fostering discussions on important topics within the field. [<a href=\"https:\/\/www.climatechange.ai\/events\/iclr2024\">2<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/International_Conference_on_Learning_Representations#:~:text=The%20International%20Conference%20on%20Learning%20Representations%20(ICLR),in%20machine%20learning%20and%20artificial%20intelligence%20research.\">3<\/a>, <a href=\"https:\/\/iclr-blogposts.github.io\/2024\/about\/\">6<\/a>, <a href=\"https:\/\/iclr.cc\/Conferences\/2024\">13<\/a>]<\/p>\n\n\n\n<p>[1]&nbsp;<a href=\"https:\/\/dblp.org\/db\/conf\/iclr\/iclr2024#:~:text=The%20Twelfth%20International%20Conference%20on%20Learning%20Representations%2C%20ICLR%202024%2C%20Vienna%2C%20Austria%2C%20May%207%2D11%2C%202024.\">https:\/\/dblp.org\/db\/conf\/iclr\/iclr2024<\/a><\/p>\n\n\n\n<p>[2]&nbsp;<a href=\"https:\/\/www.climatechange.ai\/events\/iclr2024\">https:\/\/www.climatechange.ai\/events\/iclr2024<\/a><\/p>\n\n\n\n<p>[3]&nbsp;<a href=\"https:\/\/en.wikipedia.org\/wiki\/International_Conference_on_Learning_Representations#:~:text=The%20International%20Conference%20on%20Learning%20Representations%20(ICLR),in%20machine%20learning%20and%20artificial%20intelligence%20research.\">https:\/\/en.wikipedia.org\/wiki\/International_Conference_on_Learning_Representations<\/a><\/p>\n\n\n\n<p>[4]&nbsp;<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForWorkshops#:~:text=Update:%20The%20list%20of%20accepted%20workshops%20can,and%20moderating%20the%20program%20throughout%20the%20workshop.\">https:\/\/iclr.cc\/Conferences\/2024\/CallForWorkshops<\/a><\/p>\n\n\n\n<p>[5]&nbsp;<a href=\"https:\/\/iclr.cc\/virtual\/2024\/eventlistwithbios\/InvitedTalks-2024\">https:\/\/iclr.cc\/virtual\/2024\/eventlistwithbios\/InvitedTalks-2024<\/a><\/p>\n\n\n\n<p>[6]&nbsp;<a href=\"https:\/\/iclr-blogposts.github.io\/2024\/about\/\">https:\/\/iclr-blogposts.github.io\/2024\/about\/<\/a><\/p>\n\n\n\n<p>[7]&nbsp;<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForPapers\">https:\/\/iclr.cc\/Conferences\/2024\/CallForPapers<\/a><\/p>\n\n\n\n<p>[8]&nbsp;<a href=\"https:\/\/iclr.cc\/Downloads\/2024\">https:\/\/iclr.cc\/Downloads\/2024<\/a><\/p>\n\n\n\n<p>[9]&nbsp;<a href=\"https:\/\/iclr.cc\/virtual\/2024\/index.html\">https:\/\/iclr.cc\/virtual\/2024\/index.html<\/a><\/p>\n\n\n\n<p>[10]&nbsp;<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/Sponsors\">https:\/\/iclr.cc\/Conferences\/2024\/Sponsors<\/a><\/p>\n\n\n\n<p>[11]&nbsp;<a href=\"https:\/\/blog.iclr.cc\/category\/iclr-2024\/\">https:\/\/blog.iclr.cc\/category\/iclr-2024\/<\/a><\/p>\n\n\n\n<p>[12]&nbsp;<a href=\"https:\/\/iclr.cc\/Conferences\/2024\/CallForTinyPapers\">https:\/\/iclr.cc\/Conferences\/2024\/CallForTinyPapers<\/a><\/p>\n\n\n\n<p>[13]&nbsp;<a href=\"https:\/\/iclr.cc\/Conferences\/2024\">https:\/\/iclr.cc\/Conferences\/2024<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>PODCAST: review of a failed Peer Review. The Mamba paper was submitted to ICLR 2024, which is described as one of the most prestigious machine learning conferences in the world. In January, the paper was rejected by peer reviewers. Explore how Mamba positions itself as a competitor to the Transformers AI Architecture. The provided source&hellip;&nbsp;<a href=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=1809\" rel=\"bookmark\"><span class=\"screen-reader-text\">Rejection of Mamba by prestigious conferences<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":1811,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[14,7],"tags":[],"class_list":["post-1809","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-podcast","category-the-truben-show"],"_links":{"self":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/1809","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1809"}],"version-history":[{"count":3,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/1809\/revisions"}],"predecessor-version":[{"id":1815,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/1809\/revisions\/1815"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/media\/1811"}],"wp:attachment":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1809"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1809"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1809"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}