{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-12639","title":"CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation","arxiv_id":"2602.12639","date":"2026-02-13","proceeding":null,"authors":["Yiran Rex Ma","Yuxiao Ye","Huiyuan Xie"],"abstract":"Legal text generated by large language models (LLMs) can usually achieve reasonable factual accuracy, but it frequently fails to adhere to the specialised stylistic norms and linguistic conventions of legal writing. In order to improve stylistic quality, a crucial first step is to establish a reliable evaluation method. However, having legal experts manually develop such a metric is impractical, as the implicit stylistic requirements in legal writing practice are difficult to formalise into explicit rubrics. Meanwhile, existing automatic evaluation methods also fall short: reference-based metrics conflate semantic accuracy with stylistic fidelity, and LLM-as-a-judge evaluations suffer from opacity and inconsistency. To address these challenges, we introduce CLASE (Chinese LegAlese Stylistic Evaluation), a hybrid evaluation method that focuses on the stylistic performance of legal text. The method incorporates a hybrid scoring mechanism that combines 1) linguistic feature-based scores and 2) experience-guided LLM-as-a-judge scores. Both the feature coefficients and the LLM scoring experiences are learned from contrastive pairs of authentic legal documents and their LLM-restored counterparts. This hybrid design captures both surface-level features and implicit stylistic norms in a transparent, reference-free manner. Experiments on 200 Chinese legal documents show that CLASE achieves substantially higher alignment with human judgments than traditional metrics and pure LLM-as-a-judge methods. Beyond improved alignment, CLASE provides interpretable score breakdowns and suggestions for improvements, offering a scalable and practical solution for professional stylistic evaluation in legal text generation (Code and data for CLASE is available at: https://github.com/rexera/CLASE).","url_abs":"https://arxiv.org/abs/2602.12639","url_pdf":"https://arxiv.org/pdf/2602.12639","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.12639"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/rexera/CLASE","reach":null}],"summary":{"ran_honours":5,"ran_draft_wrong":12,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":20,"ran":17,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4242172a7e5c5abe","entry":"calculate_character_percentage","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4242172a7e5c5abe"}},{"code_sha256_prefix":"e5c01f01bf81d45d","entry":"calculate_second_most_common_percentage","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5c01f01bf81d45d"}},{"code_sha256_prefix":"b811ef24a1c8e9ca","entry":"calculate_stroke_ratios","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b811ef24a1c8e9ca"}},{"code_sha256_prefix":"9c81670a59d88906","entry":"count_words","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c81670a59d88906"}},{"code_sha256_prefix":"a3b69be84b392c29","entry":"feature_19_to_24","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3b69be84b392c29"}},{"code_sha256_prefix":"9a3b058b63e215e3","entry":"feature_1_to_3","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a3b058b63e215e3"}},{"code_sha256_prefix":"406646e492867d9a","entry":"feature_25_to_53","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"406646e492867d9a"}},{"code_sha256_prefix":"6a44075205b52eb2","entry":"feature_4_to_7","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a44075205b52eb2"}},{"code_sha256_prefix":"66145e6bdf610ace","entry":"feature_60_to_65","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66145e6bdf610ace"}},{"code_sha256_prefix":"e0852090dc7875ca","entry":"feature_66_to_78","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0852090dc7875ca"}},{"code_sha256_prefix":"a85f527c01058502","entry":"feature_8_to_18","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a85f527c01058502"}},{"code_sha256_prefix":"15a2af46ac5adcac","entry":"find_idioms_in_text","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15a2af46ac5adcac"}},{"code_sha256_prefix":"3a82c645ff23711b","entry":"get_word_pos","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a82c645ff23711b"}},{"code_sha256_prefix":"dc866e2d5707488d","entry":"load_common_characters","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc866e2d5707488d"}},{"code_sha256_prefix":"c8bb249e65f69d67","entry":"load_stroke_data","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8bb249e65f69d67"}},{"code_sha256_prefix":"20cb2908e81784fb","entry":"split_chinese_sentences","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20cb2908e81784fb"}},{"code_sha256_prefix":"410516a5a8ae2900","entry":"split_into_sentences","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"410516a5a8ae2900"}},{"code_sha256_prefix":"322b83ed256e96df","entry":"extract_all_features","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"322b83ed256e96df"}},{"code_sha256_prefix":"9cb8f0cd9e686419","entry":"feature_54_to_59","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9cb8f0cd9e686419"}},{"code_sha256_prefix":"37b7962ab95bea4f","entry":"load_idioms","repo":"rexera/CLASE","repo_kind":"found_in_text","path":"linguistic_features/feature_extractor.py","file_url":"https://github.com/rexera/CLASE/blob/HEAD/linguistic_features/feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"37b7962ab95bea4f"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}