{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/in-depth-analysis-of-low-rank-matrix","title":"In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting","arxiv_id":"2409.08771","date":"2024-09-13","proceeding":null,"authors":["Constantin Philippenko","Kevin Scaman","Laurent Massoulié"],"abstract":"We analyze a distributed algorithm to compute a low-rank matrix factorization on $N$ clients, each holding a local dataset $\\mathbf{S}^i \\in \\mathbb{R}^{n_i \\times d}$, mathematically, we seek to solve $min_{\\mathbf{U}^i \\in \\mathbb{R}^{n_i\\times r}, \\mathbf{V}\\in \\mathbb{R}^{d \\times r} } \\frac{1}{2} \\sum_{i=1}^N \\|\\mathbf{S}^i - \\mathbf{U}^i \\mathbf{V}^\\top\\|^2_{\\text{F}}$. Considering a power initialization of $\\mathbf{V}$, we rewrite the previous smooth non-convex problem into a smooth strongly-convex problem that we solve using a parallel Nesterov gradient descent potentially requiring a single step of communication at the initialization step. For any client $i$ in $\\{1, \\dots, N\\}$, we obtain a global $\\mathbf{V}$ in $\\mathbb{R}^{d \\times r}$ common to all clients and a local variable $\\mathbf{U}^i$ in $\\mathbb{R}^{n_i \\times r}$. We provide a linear rate of convergence of the excess loss which depends on $\\sigma_{\\max} / \\sigma_{r}$, where $\\sigma_{r}$ is the $r^{\\mathrm{th}}$ singular value of the concatenation $\\mathbf{S}$ of the matrices $(\\mathbf{S}^i)_{i=1}^N$. This result improves the rates of convergence given in the literature, which depend on $\\sigma_{\\max}^2 / \\sigma_{\\min}^2$. We provide an upper bound on the Frobenius-norm error of reconstruction under the power initialization strategy. We complete our analysis with experiments on both synthetic and real data.","url_abs":"https://arxiv.org/abs/2409.08771v1","url_pdf":"https://arxiv.org/pdf/2409.08771v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"in-depth-analysis-of-low-rank-matrix","repo_url":"https://github.com/philipco/matrix_factorization","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}