{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fnspid-a-comprehensive-financial-news-dataset","title":"FNSPID: A Comprehensive Financial News Dataset in Time Series","arxiv_id":"2402.06698","date":"2024-02-09","proceeding":null,"authors":["Zihan Dong","Xinyu Fan","Zhiyuan Peng"],"abstract":"Financial market predictions utilize historical data to anticipate future stock prices and market trends. Traditionally, these predictions have focused on the statistical analysis of quantitative factors, such as stock prices, trading volumes, inflation rates, and changes in industrial production. Recent advancements in large language models motivate the integrated financial analysis of both sentiment data, particularly market news, and numerical factors. Nonetheless, this methodology frequently encounters constraints due to the paucity of extensive datasets that amalgamate both quantitative and qualitative sentiment analyses. To address this challenge, we introduce a large-scale financial dataset, namely, Financial News and Stock Price Integration Dataset (FNSPID). It comprises 29.7 million stock prices and 15.7 million time-aligned financial news records for 4,775 S&P500 companies, covering the period from 1999 to 2023, sourced from 4 stock market news websites. We demonstrate that FNSPID excels existing stock market datasets in scale and diversity while uniquely incorporating sentiment information. Through financial analysis experiments on FNSPID, we propose: (1) the dataset's size and quality significantly boost market prediction accuracy; (2) adding sentiment scores modestly enhances performance on the transformer-based model; (3) a reproducible procedure that can update the dataset. Completed work, code, documentation, and examples are available at github.com/Zdong104/FNSPID. FNSPID offers unprecedented opportunities for the financial research community to advance predictive modeling and analysis.","url_abs":"https://arxiv.org/abs/2402.06698v1","url_pdf":"https://arxiv.org/pdf/2402.06698v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fnspid-a-comprehensive-financial-news-dataset","repo_url":"https://github.com/zdong104/fnspid","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"fnspid-a-comprehensive-financial-news-dataset","repo_url":"https://github.com/benstaf/finrl_deepseek","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"fnspid-a-comprehensive-financial-news-dataset","repo_url":"https://github.com/ruijian-zha/finrl-dapo-sr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"financial-analysis","task_name":"Financial Analysis"},{"task_slug":"time-series-1","task_name":"Time Series"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.06698","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.06698"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zdong104/fnspid","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruijian-zha/finrl-dapo-sr","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/benstaf/finrl_deepseek","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1},"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"15ff1074c89a741d","entry":"convert_to_utc","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/preprocess.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"15ff1074c89a741d"}},{"code_sha256_prefix":"d060e1be97e812b7","entry":"convert_to_utc","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/price_news_integrate.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/price_news_integrate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d060e1be97e812b7"}},{"code_sha256_prefix":"583ea508a61447c2","entry":"fill_missing_dates_with_exponential_decay","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/price_news_integrate.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/price_news_integrate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"583ea508a61447c2"}},{"code_sha256_prefix":"c3b91ec8c0212463","entry":"fill_missing_dates_with_log_decay","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/price_news_integrate.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/price_news_integrate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c3b91ec8c0212463"}},{"code_sha256_prefix":"8d392d696298efce","entry":"increase_weight_for_key_words","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/summarize.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/summarize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8d392d696298efce"}},{"code_sha256_prefix":"5e089b4eca2710c5","entry":"convert_volume","repo":"benstaf/finrl_deepseek","repo_kind":"listed","path":"train_trade_data_deepseek_risk.py","file_url":"https://github.com/benstaf/finrl_deepseek/blob/HEAD/train_trade_data_deepseek_risk.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e089b4eca2710c5"}},{"code_sha256_prefix":"e75178b215e7ae08","entry":"from_csv_get_sentiment","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/score_by_gpt.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/score_by_gpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e75178b215e7ae08"}},{"code_sha256_prefix":"0992c0f6762e08d5","entry":"get_sentiment","repo":"zdong104/fnspid","repo_kind":"official","path":"data_processor/score_by_gpt.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/data_processor/score_by_gpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0992c0f6762e08d5"}},{"code_sha256_prefix":"9fcf945f9e9ae1a4","entry":"get_sentiment","repo":"benstaf/finrl_deepseek","repo_kind":"listed","path":"sentiment_deepseek_deepinfra.py","file_url":"https://github.com/benstaf/finrl_deepseek/blob/HEAD/sentiment_deepseek_deepinfra.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fcf945f9e9ae1a4"}},{"code_sha256_prefix":"12d0ac40dfc494f9","entry":"re_syn","repo":"zdong104/fnspid","repo_kind":"official","path":"dataset_test/integrate_result.py","file_url":"https://github.com/zdong104/fnspid/blob/HEAD/dataset_test/integrate_result.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"12d0ac40dfc494f9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}