{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tight-dimensionality-reduction-for-sketching","title":"Tight Dimensionality Reduction for Sketching Low Degree Polynomial Kernels","arxiv_id":null,"date":"2019-12-01","proceeding":"NeurIPS 2019 12","authors":["Michela Meister","Tamas Sarlos","David Woodruff"],"abstract":"We revisit the classic randomized sketch of a tensor product of $q$ vectors $x_i\\in\\mathbb{R}^n$. The $i$-th coordinate $(Sx)_i$ of the sketch is equal to\n$\\prod_{j = 1}^q \\langle u^{i, j}, x^j \\rangle / \\sqrt{m}$, where $u^{i,j}$ are independent random sign vectors. Kar and Karnick (JMLR, 2012) show that\nif the sketching dimension $m = \\Omega(\\epsilon^{-2} C_{\\Omega}^2 \\log (1/\\delta))$, where $C_{\\Omega}$ is a certain property of the point set $\\Omega$ one wants to sketch, then with probability $1-\\delta$, $\\|Sx\\|_2 = (1\\pm \\epsilon)\\|x\\|_2$ for all $x\\in\\Omega$. However, in their analysis $C_{\\Omega}^2$ can be as large as $\\Theta(n^{2q})$, even for a set $\\Omega$ of $O(1)$ vectors $x$.\n\nWe give a new analysis of this sketch, providing nearly optimal bounds.\nNamely, we show an upper bound of\n$m = \\Theta \\left (\\epsilon^{-2} \\log(n/\\delta) + \\epsilon^{-1} \\log^q(n/\\delta) \\right ),$\nwhich by composing with CountSketch, can be improved to\n$\\Theta(\\epsilon^{-2}\\log(1/(\\delta \\epsilon)) + \\epsilon^{-1} \\log^q (1/(\\delta \\epsilon))$. For the important case of $q = 2$ and $\\delta = 1/\\poly(n)$, this shows that $m = \\Theta(\\epsilon^{-2} \\log(n) + \\epsilon^{-1} \\log^2(n))$,\ndemonstrating that the $\\epsilon^{-2}$ and $\\log^2(n)$ terms do not multiply each other. We also show a nearly matching lower bound of\n$m = \\Omega(\\eps^{-2} \\log(1/(\\delta)) + \\eps^{-1} \\log^q(1/(\\delta)))$.\nIn a number of applications, one has $|\\Omega| = \\poly(n)$ and in this case our bounds are optimal up to a constant factor. This is the first high probability sketch for tensor products that has optimal sketch size and can be implemented in $m \\cdot \\sum_{i=1}^q \\textrm{nnz}(x_i)$ time, where $\\textrm{nnz}(x_i)$ is the\nnumber of non-zero entries of $x_i$.\n\nLastly, we empirically compare our sketch to other sketches for tensor products, and give a novel application to compressing neural networks.","url_abs":"http://papers.nips.cc/paper/9144-tight-dimensionality-reduction-for-sketching-low-degree-polynomial-kernels","url_pdf":"http://papers.nips.cc/paper/9144-tight-dimensionality-reduction-for-sketching-low-degree-polynomial-kernels.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tight-dimensionality-reduction-for-sketching","repo_url":"https://github.com/google-research/google-research","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"dimensionality-reduction","task_name":"Dimensionality Reduction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}