{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/implicit-regularization-via-soft-ascent","title":"Soft ascent-descent as a stable and flexible alternative to flooding","arxiv_id":"2310.10006","date":"2023-10-16","proceeding":null,"authors":["Matthew J. Holland","Kosuke Nakatani"],"abstract":"As a heuristic for improving test accuracy in classification, the \"flooding\" method proposed by Ishida et al. (2020) sets a threshold for the average surrogate loss at training time; above the threshold, gradient descent is run as usual, but below the threshold, a switch to gradient ascent is made. While setting the threshold is non-trivial and is usually done with validation data, this simple technique has proved remarkably effective in terms of accuracy. On the other hand, what if we are also interested in other metrics such as model complexity or average surrogate loss at test time? As an attempt to achieve better overall performance with less fine-tuning, we propose a softened, pointwise mechanism called SoftAD (soft ascent-descent) that downweights points on the borderline, limits the effects of outliers, and retains the ascent-descent effect of flooding, with no additional computational overhead. We contrast formal stationarity guarantees with those for flooding, and empirically demonstrate how SoftAD can realize classification accuracy competitive with flooding (and the more expensive alternative SAM) while enjoying a much smaller loss generalization gap and model norm.","url_abs":"https://arxiv.org/abs/2310.10006v2","url_pdf":"https://arxiv.org/pdf/2310.10006v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"implicit-regularization-via-soft-ascent","repo_url":"https://github.com/feedbackward/bdd-flood","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[{"method_slug":"sam","method_name":"SAM"},{"method_slug":"sharpness-aware-minimization","method_name":"Sharpness-Aware Minimization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.10006","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.10006"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/davda54/sam","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/feedbackward/bdd-flood","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3e6e279e26dc7bb9","entry":"Loss_SoftAD","repo":"feedbackward/bdd-flood","repo_kind":"official","path":"bdd-flood/setup/losses.py","file_url":"https://github.com/feedbackward/bdd-flood/blob/HEAD/bdd-flood/setup/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e6e279e26dc7bb9"}},{"code_sha256_prefix":"b16898d04085f381","entry":"get_named_loss","repo":"feedbackward/bdd-flood","repo_kind":"official","path":"bdd-flood/setup/losses.py","file_url":"https://github.com/feedbackward/bdd-flood/blob/HEAD/bdd-flood/setup/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b16898d04085f381"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}