{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gsasrec-reducing-overconfidence-in-sequential","title":"gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling","arxiv_id":"2308.07192","date":"2023-08-14","proceeding":null,"authors":["Aleksandr Petrov","Craig Macdonald"],"abstract":"A large catalogue size is one of the central challenges in training recommendation models: a large number of items makes them memory and computationally inefficient to compute scores for all items during training, forcing these models to deploy negative sampling. However, negative sampling increases the proportion of positive interactions in the training data, and therefore models trained with negative sampling tend to overestimate the probabilities of positive interactions a phenomenon we call overconfidence. While the absolute values of the predicted scores or probabilities are not important for the ranking of retrieved recommendations, overconfident models may fail to estimate nuanced differences in the top-ranked items, resulting in degraded performance. In this paper, we show that overconfidence explains why the popular SASRec model underperforms when compared to BERT4Rec. This is contrary to the BERT4Rec authors explanation that the difference in performance is due to the bi-directional attention mechanism. To mitigate overconfidence, we propose a novel Generalised Binary Cross-Entropy Loss function (gBCE) and theoretically prove that it can mitigate overconfidence. We further propose the gSASRec model, an improvement over SASRec that deploys an increased number of negatives and the gBCE loss. We show through detailed experiments on three datasets that gSASRec does not exhibit the overconfidence problem. As a result, gSASRec can outperform BERT4Rec (e.g. +9.47% NDCG on the MovieLens-1M dataset), while requiring less training time (e.g. -73% training time on MovieLens-1M). Moreover, in contrast to BERT4Rec, gSASRec is suitable for large datasets that contain more than 1 million items.","url_abs":"https://arxiv.org/abs/2308.07192v1","url_pdf":"https://arxiv.org/pdf/2308.07192v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gsasrec-reducing-overconfidence-in-sequential","repo_url":"https://github.com/asash/gsasrec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"gsasrec-reducing-overconfidence-in-sequential","repo_url":"https://github.com/asash/gsasrec-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"sequential-recommendation","task_name":"Sequential Recommendation"}],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.07192","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.07192"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asash/gsasrec","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asash/gsasrec-pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran_draft_wrong":2,"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"81e3e95accb45fcd","entry":"get_max_test_users","repo":"asash/gsasrec","repo_kind":"official","path":"evaluation/run_experiment.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"81e3e95accb45fcd"}},{"code_sha256_prefix":"455c95e4d2eb8300","entry":"get_metrics","repo":"asash/gsasrec","repo_kind":"official","path":"evaluation/analyze_experiment_in_progress.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/analyze_experiment_in_progress.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"455c95e4d2eb8300"}},{"code_sha256_prefix":"90f30c1ba3ef9bf4","entry":"is_experiment_start","repo":"asash/gsasrec","repo_kind":"official","path":"evaluation/analyze_experiment_in_progress.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/analyze_experiment_in_progress.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90f30c1ba3ef9bf4"}},{"code_sha256_prefix":"d2263417dafdd010","entry":"parse_experiment","repo":"asash/gsasrec","repo_kind":"official","path":"evaluation/analyze_experiment_in_progress.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/analyze_experiment_in_progress.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d2263417dafdd010"}},{"code_sha256_prefix":"61aa1a96b85940bd","entry":"collate_val_test","repo":"asash/gsasrec-pytorch","repo_kind":"listed","path":"dataset_utils.py","file_url":"https://github.com/asash/gsasrec-pytorch/blob/HEAD/dataset_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61aa1a96b85940bd"}},{"code_sha256_prefix":"7eab2ae2aa929824","entry":"collate_with_random_negatives","repo":"asash/gsasrec-pytorch","repo_kind":"listed","path":"dataset_utils.py","file_url":"https://github.com/asash/gsasrec-pytorch/blob/HEAD/dataset_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7eab2ae2aa929824"}},{"code_sha256_prefix":"c77b2756000b5584","entry":"get_global_tensorboard_dir","repo":"asash/gsasrec","repo_kind":"official","path":"evaluation/run_experiment.py","file_url":"https://github.com/asash/gsasrec/blob/HEAD/evaluation/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c77b2756000b5584"}},{"code_sha256_prefix":"df8bda213ff3826a","entry":"get_num_items","repo":"asash/gsasrec-pytorch","repo_kind":"listed","path":"dataset_utils.py","file_url":"https://github.com/asash/gsasrec-pytorch/blob/HEAD/dataset_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"df8bda213ff3826a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}