{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tighter-low-rank-approximation-via-sampling","title":"Tighter Low-rank Approximation via Sampling the Leveraged Element","arxiv_id":"1410.3886","date":"2014-10-14","proceeding":null,"authors":["Srinadh Bhojanapalli","Prateek Jain","Sujay Sanghavi"],"abstract":"In this work, we propose a new randomized algorithm for computing a low-rank\napproximation to a given matrix. Taking an approach different from existing\nliterature, our method first involves a specific biased sampling, with an\nelement being chosen based on the leverage scores of its row and column, and\nthen involves weighted alternating minimization over the factored form of the\nintended low-rank matrix, to minimize error only on these samples. Our method\ncan leverage input sparsity, yet produce approximations in {\\em spectral} (as\nopposed to the weaker Frobenius) norm; this combines the best aspects of\notherwise disparate current results, but with a dependence on the condition\nnumber $\\kappa = \\sigma_1/\\sigma_r$. In particular we require $O(nnz(M) +\n\\frac{n\\kappa^2 r^5}{\\epsilon^2})$ computations to generate a rank-$r$\napproximation to $M$ in spectral norm. In contrast, the best existing method\nrequires $O(nnz(M)+ \\frac{nr^2}{\\epsilon^4})$ time to compute an approximation\nin Frobenius norm. Besides the tightness in spectral norm, we have a better\ndependence on the error $\\epsilon$. Our method is naturally and highly\nparallelizable.\n  Our new approach enables two extensions that are interesting on their own.\nThe first is a new method to directly compute a low-rank approximation (in\nefficient factored form) to the product of two given matrices; it computes a\nsmall random set of entries of the product, and then executes weighted\nalternating minimization (as before) on these. The sampling strategy is\ndifferent because now we cannot access leverage scores of the product matrix\n(but instead have to work with input matrices). The second extension is an\nimproved algorithm with smaller communication complexity for the distributed\nPCA setting (where each server has small set of rows of the matrix, and want to\ncompute low rank approximation with small amount of communication with other\nservers).","url_abs":"http://arxiv.org/abs/1410.3886v1","url_pdf":"http://arxiv.org/pdf/1410.3886v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tighter-low-rank-approximation-via-sampling","repo_url":"https://github.com/wushanshan/MatrixProductPCA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}