{"url":"/method/linear-regression","slug":"linear-regression","name":"Linear Regression","full_name":"Linear Regression","full_name_withheld":false,"description_markdown":"**Linear Regression** is a method for modelling a relationship between a dependent variable and independent variables. These models can be fit with numerous approaches. The most common is *least squares*, where we minimize the mean square error between the predicted values $\\hat{y} = \\textbf{X}\\hat{\\beta}$ and actual values $y$: $\\left(y-\\textbf{X}\\beta\\right)^{2}$.\r\n\r\nWe can also define the problem in probabilistic terms as a generalized linear model (GLM) where the pdf is a Gaussian distribution, and then perform maximum likelihood estimation to estimate $\\hat{\\beta}$.\r\n\r\nImage Source: [Wikipedia](https://en.wikipedia.org/wiki/Linear_regression)","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Generalized Linear Models","url":"/methods/category/generalized-linear-models","pwc_aliases":[]}],"n_papers_tagged":1657,"archive_num_papers":1657,"papers_newest_first":[{"paper":null,"title":"Wild Bootstrap Inference for Linear Regressions with Many Covariates","date":"2025-06-26","arxiv_id":"2506.20972","n_code_links":0,"syntology":null},{"paper":null,"title":"The Persistent Effects of Peru's Mining MITA: Double Machine Learning Approach","date":"2025-06-23","arxiv_id":"2506.18947","n_code_links":0,"syntology":null},{"paper":null,"title":"A Simplified Analysis of SGD for Linear Regression with Weight Averaging","date":"2025-06-18","arxiv_id":"2506.15535","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing eLoran Timing Accuracy via Machine Learning with Meteorological and Terrain Data","date":"2025-06-18","arxiv_id":"2506.15235","n_code_links":0,"syntology":null},{"paper":null,"title":"Uniform Mean Estimation for Heavy-Tailed Distributions via Median-of-Means","date":"2025-06-17","arxiv_id":"2506.14673","n_code_links":0,"syntology":null},{"paper":null,"title":"Optimizing Blood Transfusions and Predicting Shortages in Resource-Constrained Areas","date":"2025-06-14","arxiv_id":"2506.12647","n_code_links":0,"syntology":null},{"paper":"/paper/statistical-machine-learning-for-astronomy-a","title":"Statistical Machine Learning for Astronomy -- A Textbook","date":"2025-06-13","arxiv_id":"2506.12230","n_code_links":1,"syntology":null},{"paper":"/paper/heavy-lasso-sparse-penalized-regression-under","title":"Heavy Lasso: sparse penalized regression under heavy-tailed noise via data-augmented soft-thresholding","date":"2025-06-09","arxiv_id":"2506.07790","n_code_links":1,"syntology":null},{"paper":null,"title":"Standard LSParameter Estimators Ensure Finite Convergence Time for Linear Regression Equations Under an Interval Excitation Assumption","date":"2025-06-09","arxiv_id":"2506.08211","n_code_links":0,"syntology":null},{"paper":null,"title":"Differentially Private Sparse Linear Regression with Heavy-tailed Responses","date":"2025-06-07","arxiv_id":"2506.06861","n_code_links":0,"syntology":null},{"paper":null,"title":"An Optimized Franz-Parisi Criterion and its Equivalence with SQ Lower Bounds","date":"2025-06-06","arxiv_id":"2506.06259","n_code_links":0,"syntology":null},{"paper":"/paper/2506-05188","title":"Counterfactual reasoning: an analysis of in-context emergence","date":"2025-06-05","arxiv_id":"2506.05188","n_code_links":1,"syntology":{"ran":5,"of":16,"unverified":11,"pointer_only":0}},{"paper":null,"title":"Meta-analysis of Life Cycle Assessments for Li-Ion Batteries Production Emissions","date":"2025-06-05","arxiv_id":"2506.05531","n_code_links":0,"syntology":null},{"paper":null,"title":"Modern approaches to building effective interpretable models of the property market using machine learning","date":"2025-06-05","arxiv_id":"2506.15723","n_code_links":0,"syntology":null},{"paper":null,"title":"Privacy Amplification Through Synthetic Data: Insights from Linear Regression","date":"2025-06-05","arxiv_id":"2506.05101","n_code_links":0,"syntology":null},{"paper":null,"title":"Relational reasoning and inductive bias in transformers trained on a transitive inference task","date":"2025-06-04","arxiv_id":"2506.04289","n_code_links":0,"syntology":null},{"paper":"/paper/2506-04379","title":"Visualizing and Controlling Cortical Responses Using Voxel-Weighted Activation Maximization","date":"2025-06-04","arxiv_id":"2506.04379","n_code_links":1,"syntology":null},{"paper":null,"title":"Replay Can Provably Increase Forgetting","date":"2025-06-04","arxiv_id":"2506.04377","n_code_links":0,"syntology":null},{"paper":null,"title":"Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds","date":"2025-06-03","arxiv_id":"2506.03100","n_code_links":0,"syntology":null},{"paper":null,"title":"An Empirical Analysis of Tiff's Impact on American Business Formation","date":"2025-06-01","arxiv_id":"2506.00999","n_code_links":0,"syntology":null},{"paper":null,"title":"Linear regression with overparameterized linear neural networks: Tight upper and lower bounds for implicit $\\ell^1$-regularization","date":"2025-06-01","arxiv_id":"2506.01143","n_code_links":0,"syntology":null},{"paper":null,"title":"Learning Aerodynamics for the Control of Flying Humanoid Robots","date":"2025-05-30","arxiv_id":"2506.00305","n_code_links":0,"syntology":null},{"paper":"/paper/the-gaussian-mixing-mechanism-renyi","title":"The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches","date":"2025-05-30","arxiv_id":"2505.24603","n_code_links":1,"syntology":null},{"paper":null,"title":"Comparing the Moore-Penrose Pseudoinverse and Gradient Descent for Solving Linear Regression Problems: A Performance Analysis","date":"2025-05-29","arxiv_id":"2505.23552","n_code_links":0,"syntology":null},{"paper":null,"title":"SlimLLM: Accurate Structured Pruning for Large Language Models","date":"2025-05-28","arxiv_id":"2505.22689","n_code_links":0,"syntology":null},{"paper":null,"title":"Empirical 3D Channel Modeling for Cellular-Connected UAVs: A Triple-Layer Machine Learning Approach","date":"2025-05-26","arxiv_id":"2505.19478","n_code_links":0,"syntology":null},{"paper":null,"title":"Better Rates for Private Linear Regression in the Proportional Regime via Aggressive Clipping","date":"2025-05-22","arxiv_id":"2505.16329","n_code_links":0,"syntology":null},{"paper":null,"title":"Decoupling Representation and Learning in Genetic Programming: the LaSER Approach","date":"2025-05-22","arxiv_id":"2505.17309","n_code_links":0,"syntology":null},{"paper":"/paper/diving-into-the-fusion-of-monocular-priors","title":"Diving into the Fusion of Monocular Priors for Generalized Stereo Matching","date":"2025-05-20","arxiv_id":"2505.14414","n_code_links":1,"syntology":null},{"paper":null,"title":"Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective","date":"2025-05-20","arxiv_id":"2505.14808","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/regression-1","name":"regression","papers":906},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":86},{"task":"/task/prediction","name":"Prediction","papers":62},{"task":"/task/time-series-1","name":"Time Series","papers":55},{"task":"/task/time-series","name":"Time Series Analysis","papers":47},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":41},{"task":"/task/in-context-learning","name":"In-Context Learning","papers":38},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":38},{"task":"/task/variable-selection","name":"Variable Selection","papers":32},{"task":null,"name":"valid","papers":32},{"task":"/task/federated-learning","name":"Federated Learning","papers":31},{"task":"/task/model-selection","name":"Model Selection","papers":31},{"task":"/task/feature-selection","name":"feature selection","papers":31},{"task":"/task/clustering","name":"Clustering","papers":29},{"task":"/task/management","name":"Management","papers":29},{"task":"/task/decision-making","name":"Decision Making","papers":26},{"task":"/task/bayesian-inference","name":"Bayesian Inference","papers":24},{"task":"/task/parameter-estimation","name":"parameter estimation","papers":23},{"task":"/task/classification","name":"General Classification","papers":22},{"task":"/task/causal-inference","name":"Causal Inference","papers":19}],"tasks_shown":20,"n_tasks":416,"usage_by_year":[{"year":"2007","papers":1},{"year":"2008","papers":3},{"year":"2009","papers":2},{"year":"2010","papers":4},{"year":"2011","papers":2},{"year":"2012","papers":3},{"year":"2013","papers":20},{"year":"2014","papers":19},{"year":"2015","papers":29},{"year":"2016","papers":37},{"year":"2017","papers":67},{"year":"2018","papers":81},{"year":"2019","papers":116},{"year":"2020","papers":237},{"year":"2021","papers":210},{"year":"2022","papers":207},{"year":"2023","papers":237},{"year":"2024","papers":257},{"year":"2025","papers":125}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/linear-regression"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}