{"url":"/task/feature-engineering","name":"Feature Engineering","slug":"feature-engineering","description_markdown":"Feature engineering is the process of taking a dataset and constructing explanatory variables — features — that can be used to train a machine learning model for a prediction problem. Often, data is spread across multiple tables and must be gathered into a single table with rows containing the observations and features in the columns.\r\n\r\nThe traditional approach to feature engineering is to build features one at a time using domain knowledge, a tedious, time-consuming, and error-prone process known as manual feature engineering. The code for manual feature engineering is problem-dependent and must be re-written for each new dataset.","categories":[{"name":"Methodology","url":"/area/methodology"},{"name":"Miscellaneous","url":"/area/miscellaneous"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":1706,"papers_with_code":472,"benchmarks":1,"benchmark_tables_in_archive":1,"benchmark_tables_shown":1,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":5,"subtasks":1,"parent_tasks":0},"benchmarks":[{"leaderboard":"/sota/feature-engineering-on-2019-test-set-1","slug":"feature-engineering-on-2019-test-set-1","dataset":"2019_test set","dataset_url":null,"rows_in_archive":1,"metrics":["14 gestures accuracy"],"first_row_in_archive_order":{"model":"CNN","paper_title":"See and Read: Detecting Depression Symptoms in Higher Education Students Using Multimodal Social Media Data","paper_url":"/paper/see-and-read-detecting-depression-symptoms-in","paper_date":"2019-12-03","arxiv_id":"1912.01131","code_links":[{"title":"paulomann/ReadOrSee","url":"https://github.com/paulomann/ReadOrSee"},{"title":"paulomann/ReadAndSee","url":"https://github.com/paulomann/ReadAndSee"},{"title":"gerardoasilva/NLP-DepressionDetection","url":"https://github.com/gerardoasilva/NLP-DepressionDetection"}],"syntology":null}}],"datasets":[{"url":"/dataset/ember","name":"EMBER","full_name":"","num_papers_in_archive":115},{"url":"/dataset/kit-motion-language","name":"KIT Motion-Language","full_name":"","num_papers_in_archive":48},{"url":"/dataset/onestopenglish","name":"OneStopEnglish","full_name":"","num_papers_in_archive":29},{"url":"/dataset/smhd","name":"SMHD","full_name":"Self-reported Mental Health Diagnoses","num_papers_in_archive":16},{"url":"/dataset/ncbi-disease-corpus","name":"NCBI Datasets","full_name":"","num_papers_in_archive":11}],"subtasks":[{"url":"/task/imputation","name":"Imputation"}],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":472,"tagged_in_all":1706,"items":[{"url":"/paper/voxelnet-end-to-end-learning-for-point-cloud","title":"VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection","date":"2017-11-17","arxiv_id":"1711.06396","repositories_listed":44,"syntology":{"n":4,"n_ran":2,"n_unverified":2,"n_pointer_only":4}},{"url":"/paper/wide-deep-learning-for-recommender-systems","title":"Wide & Deep Learning for Recommender Systems","date":"2016-06-24","arxiv_id":"1606.07792","repositories_listed":39,"syntology":{"n":5,"n_ran":0,"n_unverified":5,"n_pointer_only":5}},{"url":"/paper/end-to-end-sequence-labeling-via-bi","title":"End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF","date":"2016-03-04","arxiv_id":"1603.01354","repositories_listed":25,"syntology":{"n":24,"n_ran":4,"n_unverified":20,"n_pointer_only":3}},{"url":"/paper/deepfm-a-factorization-machine-based-neural","title":"DeepFM: A Factorization-Machine based Neural Network for CTR Prediction","date":"2017-03-13","arxiv_id":"1703.04247","repositories_listed":23,"syntology":{"n":8,"n_ran":2,"n_unverified":6,"n_pointer_only":2}},{"url":"/paper/deep-cross-network-for-ad-click-predictions","title":"Deep & Cross Network for Ad Click Predictions","date":"2017-08-17","arxiv_id":"1708.05123","repositories_listed":16,"syntology":{"n":2,"n_ran":2,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/named-entity-recognition-with-bidirectional","title":"Named Entity Recognition with Bidirectional LSTM-CNNs","date":"2015-11-26","arxiv_id":"1511.08308","repositories_listed":14,"syntology":{"n":9,"n_ran":4,"n_unverified":5,"n_pointer_only":4}},{"url":"/paper/product-based-neural-networks-for-user-1","title":"Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data","date":"2018-07-01","arxiv_id":"1807.00311","repositories_listed":10,"syntology":null},{"url":"/paper/deepfm-an-end-to-end-wide-deep-learning","title":"DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction","date":"2018-04-12","arxiv_id":"1804.04950","repositories_listed":8,"syntology":{"n":2,"n_ran":1,"n_unverified":1,"n_pointer_only":2}},{"url":"/paper/chexbert-combining-automatic-labelers-and","title":"CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT","date":"2020-04-20","arxiv_id":"2004.09167","repositories_listed":7,"syntology":{"n":17,"n_ran":4,"n_unverified":13,"n_pointer_only":0}},{"url":"/paper/knowledge-graph-convolutional-networks-for","title":"Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems","date":"2019-05-11","arxiv_id":"1905.04413","repositories_listed":6,"syntology":{"n":4,"n_ran":0,"n_unverified":4,"n_pointer_only":0}},{"url":"/paper/retrieve-merge-predict-augmenting-tables-with","title":"Retrieve, Merge, Predict: Augmenting Tables with Data Lakes","date":"2024-02-09","arxiv_id":"2402.06282","repositories_listed":5,"syntology":null},{"url":"/paper/multimodal-speech-emotion-recognition-and","title":"Multimodal Speech Emotion Recognition and Ambiguity Resolution","date":"2019-04-12","arxiv_id":"1904.06022","repositories_listed":5,"syntology":null},{"url":"/paper/190600586","title":"Discovering Neural Wirings","date":"2019-06-03","arxiv_id":"1906.00586","repositories_listed":4,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":3}},{"url":"/paper/disentangled-attribution-curves-for","title":"Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees","date":"2019-05-18","arxiv_id":"1905.07631","repositories_listed":4,"syntology":{"n":6,"n_ran":4,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/safe-ml-surrogate-assisted-feature-extraction","title":"SAFE ML: Surrogate Assisted Feature Extraction for Model Learning","date":"2019-02-28","arxiv_id":"1902.11035","repositories_listed":4,"syntology":null},{"url":"/paper/ml-net-multi-label-classification-of","title":"ML-Net: multi-label classification of biomedical texts with deep neural networks","date":"2018-11-13","arxiv_id":"1811.05475","repositories_listed":4,"syntology":null},{"url":"/paper/disfluency-detection-using-auto-correlational","title":"Disfluency Detection using Auto-Correlational Neural Networks","date":"2018-08-28","arxiv_id":"1808.09092","repositories_listed":4,"syntology":null},{"url":"/paper/smiles2vec-an-interpretable-general-purpose","title":"SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties","date":"2017-12-06","arxiv_id":"1712.02034","repositories_listed":4,"syntology":null},{"url":"/paper/neural-vector-spaces-for-unsupervised","title":"Neural Vector Spaces for Unsupervised Information Retrieval","date":"2017-08-09","arxiv_id":"1708.02702","repositories_listed":4,"syntology":null},{"url":"/paper/transfer-learning-for-sequence-tagging-with","title":"Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks","date":"2017-03-18","arxiv_id":"1703.06345","repositories_listed":4,"syntology":{"n":8,"n_ran":0,"n_unverified":8,"n_pointer_only":0}},{"url":"/paper/deepsurv-personalized-treatment-recommender","title":"DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network","date":"2016-06-02","arxiv_id":"1606.00931","repositories_listed":4,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/guided-cost-learning-deep-inverse-optimal","title":"Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization","date":"2016-03-01","arxiv_id":"1603.00448","repositories_listed":4,"syntology":null},{"url":"/paper/relbench-a-benchmark-for-deep-learning-on","title":"RelBench: A Benchmark for Deep Learning on Relational Databases","date":"2024-07-29","arxiv_id":"2407.20060","repositories_listed":3,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":3}},{"url":"/paper/dreamsampler-unifying-diffusion-sampling-and","title":"DreamSampler: Unifying Diffusion Sampling and Score Distillation for Image Manipulation","date":"2024-03-18","arxiv_id":"2403.11415","repositories_listed":3,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/volcanoml-speeding-up-end-to-end-automl-via","title":"VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition","date":"2021-07-19","arxiv_id":"2107.08861","repositories_listed":3,"syntology":{"n":6,"n_ran":0,"n_unverified":6,"n_pointer_only":0}},{"url":"/paper/enabling-collaborative-data-science","title":"Enabling Collaborative Data Science Development with the Ballet Framework","date":"2020-12-14","arxiv_id":"2012.07816","repositories_listed":3,"syntology":null},{"url":"/paper/vest-automatic-feature-engineering-for","title":"VEST: Automatic Feature Engineering for Forecasting","date":"2020-10-14","arxiv_id":"2010.07137","repositories_listed":3,"syntology":null},{"url":"/paper/cardea-an-open-automated-machine-learning","title":"Cardea: An Open Automated Machine Learning Framework for Electronic Health Records","date":"2020-10-01","arxiv_id":"2010.00509","repositories_listed":3,"syntology":null},{"url":"/paper/a-survey-of-information-cascade-analysis","title":"A Survey of Information Cascade Analysis: Models, Predictions, and Recent Advances","date":"2020-05-22","arxiv_id":"2005.11041","repositories_listed":3,"syntology":{"n":17,"n_ran":2,"n_unverified":15,"n_pointer_only":0}},{"url":"/paper/see-and-read-detecting-depression-symptoms-in","title":"See and Read: Detecting Depression Symptoms in Higher Education Students Using Multimodal Social Media Data","date":"2019-12-03","arxiv_id":"1912.01131","repositories_listed":3,"syntology":null}],"syntology_records":17,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}