Papers › Does ChatGPT Comprehend the Place Value in Numbers When Solving Math Word Problems?
Does ChatGPT Comprehend the Place Value in Numbers When Solving Math Word Problems?
Jisu An, Junseok Lee, Gahgene Gweon
With the recent advancements in GPT, there’s been a growing trend to integrate GPT in solving math word problems using strategies such as “Chain-of-Thought”(CoT) and “Program-of-Thought”(PoT). Based on the observation that CoT tends to yield lower accuracy than PoT when large numbers are involved, we conducted two experiments to examine whether chatGPT understands place values in numbers. In the first experiment, to examine whether GPT can correctly order numbers based on an understanding of place values, we order source and permutation multiplets that contain 3-6 numbers in base 1000. In the second experiment, We examine whether GPT based models that utilize English expressions (CoT_Eng and PoT_Eng), rather than numerical expressions (CoT_Num and PoT_Num) can yield better performance in solving math word problems. The results of the first experiment showed that the ordering accuracy for the permutation multiplets (6 elements = 60.5%) was lower than that of source multiplets (6 elements = 96.8%). The results of the second experiment showed that accuracy increased when information about the place value was provided explicitly in the format of English expression (79.9% in CoT, 82% in PoT) compared to numerical expression(76.8% in CoT, 80% in PoT). The observations from both experiments suggest that the concept of place value isn’t adequately integrated when numbers are represented as tokens in gpt3.5-turbo. Thus, research on training models to understand the concept of place value in numbers would be a possible direction to pursue as future research.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Math Word Problem Solving | SVAMP | PoT_Eng (self-consistency @ 5) | Execution Accuracy | 83.70 | #7 of 26 | Archive leaderboard | report |
| Math Word Problem Solving | SVAMP | CoT_Eng (self-consistency @ 5) | Execution Accuracy | 82.50 | #8 of 26 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections