{"url":"/dataset/game-of-24","name":"Game of 24","full_name":null,"description_markdown":"Game of 24 is a mathematical reasoning challenge, where the goal is to use 4 numbers and basic arithmetic operations (+-*/) to obtain 24. For example, given input “4 9 10 13”, a solution output could be “(10 - 4) * (13 - 9) = 24”. We scrape data from 4nums.com, which has 1,362 games that are sorted from easy to hard by human solving time, and use a subset of relatively hard games indexed 901-1,000 for testing. For each task, we consider the output as success if it is a valid equation that equals 24 and uses the input numbers each exactly once. We report the success rate across 100 games as the metric.","description_withheld":null,"homepage":"https://github.com/princeton-nlp/tree-of-thought-llm","introduced_date":"2023-05-17","introduced_date_note":null,"introduced_by":null,"license":null,"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Arithmetic Reasoning","url":"/task/arithmetic-reasoning","datasets_with_task":"/datasets/task/arithmetic-reasoning"}],"languages":[],"variants":["Game of 24"],"data_loaders":[],"num_papers_in_archive":61,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/arithmetic-reasoning-on-game-of-24","task":"Arithmetic Reasoning","dataset_variant":"Game of 24","rows":1,"metrics":["Success"],"first_row_in_archive_order":{"model":"Tree of Thoughts (b=5)","paper":"/paper/tree-of-thoughts-deliberate-problem-solving-1","metrics":{"Success":"0.74"},"code_links":[{"title":"ysymyth/tree-of-thought-llm","url":"https://github.com/ysymyth/tree-of-thought-llm"},{"title":"princeton-nlp/tree-of-thought-llm","url":"https://github.com/princeton-nlp/tree-of-thought-llm"},{"title":"codelion/optillm","url":"https://github.com/codelion/optillm"},{"title":"appl-team/appl","url":"https://github.com/appl-team/appl"},{"title":"huiwy/reflection-on-trees","url":"https://github.com/huiwy/reflection-on-trees"},{"title":"katamb/tree-of-thought-llm-ca","url":"https://github.com/katamb/tree-of-thought-llm-ca"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/tree-of-thoughts-deliberate-problem-solving-1","title":"Tree of Thoughts: Deliberate Problem Solving with Large Language Models","date":"2023-05-17","rows_on_this_dataset":1,"code_links":6,"syntology":{"read_at":"2026-09-25T09:33:49+00:00","samples_harvested":24,"samples_ran":10,"samples_unverified":14,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-25T09:33:49+00:00","papers_with_samples":1,"samples_harvested":24,"samples_ran":10,"samples_unverified":14,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}