{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kaleidoscope-an-efficient-learnable-1","title":"Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps","arxiv_id":"2012.14966","date":"2020-12-29","proceeding":"ICLR 2020 1","authors":["Tri Dao","Nimit S. Sohoni","Albert Gu","Matthew Eichhorn","Amit Blonder","Megan Leszczynski","Atri Rudra","Christopher Ré"],"abstract":"Modern neural network architectures use structured linear transformations, such as low-rank matrices, sparse matrices, permutations, and the Fourier transform, to improve inference speed and reduce memory usage compared to general linear maps. However, choosing which of the myriad structured transformations to use (and its associated parameterization) is a laborious task that requires trading off speed, space, and accuracy. We consider a different approach: we introduce a family of matrices called kaleidoscope matrices (K-matrices) that provably capture any structured matrix with near-optimal space (parameter) and time (arithmetic operation) complexity. We empirically validate that K-matrices can be automatically learned within end-to-end pipelines to replace hand-crafted procedures, in order to improve model quality. For example, replacing channel shuffles in ShuffleNet improves classification accuracy on ImageNet by up to 5%. K-matrices can also simplify hand-engineered pipelines -- we replace filter bank feature computation in speech data preprocessing with a learnable kaleidoscope layer, resulting in only 0.4% loss in accuracy on the TIMIT speech recognition task. In addition, K-matrices can capture latent structure in models: for a challenging permuted image classification task, a K-matrix based representation of permutations is able to learn the right latent structure and improves accuracy of a downstream convolutional model by over 9%. We provide a practically efficient implementation of our approach, and use K-matrices in a Transformer network to attain 36% faster end-to-end inference speed on a language translation task.","url_abs":"https://arxiv.org/abs/2012.14966v2","url_pdf":"https://arxiv.org/pdf/2012.14966v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kaleidoscope-an-efficient-learnable-1","repo_url":"https://github.com/HazyResearch/butterfly","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"kaleidoscope-an-efficient-learnable-1","repo_url":"https://github.com/HazyResearch/learning-circuits","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"channel-shuffle","method_name":"Channel Shuffle"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"groupwise-point-convolution","method_name":"Groupwise Point Convolution"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"shufflenet","method_name":"ShuffleNet"},{"method_slug":"shufflenet-block","method_name":"ShuffleNet Block"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.14966","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.14966"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HazyResearch/learning-circuits","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HazyResearch/butterfly","reach":null}],"summary":{"ran":9,"ran_fixture":5,"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"official":{"samples":23,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3b643a65ec60c95f","entry":"BbtMultUntied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b643a65ec60c95f"}},{"code_sha256_prefix":"fca125309c2b322a","entry":"BbtOrthoMultUntied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fca125309c2b322a"}},{"code_sha256_prefix":"a4e9c34b65b2f6ad","entry":"ButterflyMult","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a4e9c34b65b2f6ad"}},{"code_sha256_prefix":"c51c139065af4d76","entry":"ButterflyMultUntied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c51c139065af4d76"}},{"code_sha256_prefix":"1a5d3fd2e0f126ae","entry":"ButterflyOrthoMultTied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1a5d3fd2e0f126ae"}},{"code_sha256_prefix":"73e59bd4741ab79a","entry":"ButterflyOrthoMultUntied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73e59bd4741ab79a"}},{"code_sha256_prefix":"e3dd3b51568b948f","entry":"ComplexMul","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e3dd3b51568b948f"}},{"code_sha256_prefix":"8aa8af42641ece9f","entry":"Conjugate","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8aa8af42641ece9f"}},{"code_sha256_prefix":"981183f0e2853095","entry":"ODOMultUntied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"981183f0e2853095"}},{"code_sha256_prefix":"5e9c9a827b90d7d0","entry":"butterfly_mult_torch","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e9c9a827b90d7d0"}},{"code_sha256_prefix":"82f636e827231da2","entry":"butterfly_mult_untied_torch","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82f636e827231da2"}},{"code_sha256_prefix":"300fc5df893178c7","entry":"complex_mul_torch","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"300fc5df893178c7"}},{"code_sha256_prefix":"4eb8f139ff99f805","entry":"conjugate_torch","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4eb8f139ff99f805"}},{"code_sha256_prefix":"c2200840c1a57682","entry":"cupy2torch","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c2200840c1a57682"}},{"code_sha256_prefix":"76afe345657ec2e7","entry":"torch2numpy","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76afe345657ec2e7"}},{"code_sha256_prefix":"a34cfa6d6cd92765","entry":"twiddle_normal_to_fast_format","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a34cfa6d6cd92765"}},{"code_sha256_prefix":"56595d236ed87939","entry":"Butterfly","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56595d236ed87939"}},{"code_sha256_prefix":"412f0f9356fe1fd3","entry":"bbt_mult_untied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"412f0f9356fe1fd3"}},{"code_sha256_prefix":"7b7f6728bc8598c3","entry":"bbt_ortho_mult_tied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b7f6728bc8598c3"}},{"code_sha256_prefix":"03a76b8c72275a32","entry":"bbt_ortho_mult_untied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03a76b8c72275a32"}},{"code_sha256_prefix":"719a1fad21bbecd3","entry":"butterfly_ortho_mult_tied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"719a1fad21bbecd3"}},{"code_sha256_prefix":"3367888f332b5167","entry":"butterfly_ortho_mult_untied","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3367888f332b5167"}},{"code_sha256_prefix":"93fe676e4bff5eb2","entry":"torch2cupy","repo":"HazyResearch/butterfly","repo_kind":"official","path":"butterfly/butterfly.py","file_url":"https://github.com/HazyResearch/butterfly/blob/HEAD/butterfly/butterfly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93fe676e4bff5eb2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}