{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kernel-flows-from-learning-kernels-from-data","title":"Kernel Flows: from learning kernels from data into the abyss","arxiv_id":"1808.04475","date":"2018-08-13","proceeding":null,"authors":["Houman Owhadi","Gene Ryan Yoo"],"abstract":"Learning can be seen as approximating an unknown function by interpolating\nthe training data. Kriging offers a solution to this problem based on the prior\nspecification of a kernel. We explore a numerical approximation approach to\nkernel selection/construction based on the simple premise that a kernel must be\ngood if the number of interpolation points can be halved without significant\nloss in accuracy (measured using the intrinsic RKHS norm $\\|\\cdot\\|$ associated\nwith the kernel). We first test and motivate this idea on a simple problem of\nrecovering the Green's function of an elliptic PDE (with inhomogeneous\ncoefficients) from the sparse observation of one of its solutions. Next we\nconsider the problem of learning non-parametric families of deep kernels of the\nform $K_1(F_n(x),F_n(x'))$ with $F_{n+1}=(I_d+\\epsilon G_{n+1})\\circ F_n$ and\n$G_{n+1} \\in \\operatorname{Span}\\{K_1(F_n(x_i),\\cdot)\\}$. With the proposed\napproach constructing the kernel becomes equivalent to integrating a stochastic\ndata driven dynamical system, which allows for the training of very deep\n(bottomless) networks and the exploration of their properties. These networks\nlearn by constructing flow maps in the kernel and input spaces via incremental\ndata-dependent deformations/perturbations (appearing as the cooperative\ncounterpart of adversarial examples) and, at profound depths, they (1) can\nachieve accurate classification from only one data point per class (2) appear\nto learn archetypes of each class (3) expand distances between points that are\nin different classes and contract distances between points in the same class.\nFor kernels parameterized by the weights of Convolutional Neural Networks,\nminimizing approximation errors incurred by halving random subsets of\ninterpolation points, appears to outperform training (the same CNN\narchitecture) with relative entropy and dropout.","url_abs":"http://arxiv.org/abs/1808.04475v2","url_pdf":"http://arxiv.org/pdf/1808.04475v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kernel-flows-from-learning-kernels-from-data","repo_url":"https://github.com/MatthieuDarcy/KernelFlows","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.04475","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}