Methods › Natural Language Processing › Transformers › PaLM

Pathways Language Model

PaLM

151 papers tagged archive 2025-07-28

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

PaLM (Pathways Language Model) uses a standard Transformer model architecture (Vaswani et al., 2017) in a decoder-only setup (i.e., each timestep can only attend to itself and past timesteps), with several modifications. PaLM is trained as a 540 billion parameter, densely activated, autoregressive Transformer on 780 billion tokens. PaLM leverages Pathways (Barham et al., 2022), which enables highly efficient training of very large neural networks across thousands of accelerator chips.

Image credit: PaLM: Scaling Language Modeling with Pathways

Source: PaLM: Scaling Language Modeling with Pathways

Papers archive 2025-07-28

30 shown of 151, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 183 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Language Modelling24
Question Answering17
Language Modeling16
In-Context Learning10
Large Language Model9
Math9
Text Generation8
GSM8K7
Retrieval7
Common Sense Reasoning6
Few-Shot Learning6
Mathematical Reasoning6
Multi-task Language Understanding6
Decoder5
Diversity5
Natural Language Inference5
Natural Language Understanding5
model5
Arithmetic Reasoning4
Coreference Resolution4

Usage over time archive 2025-07-28

Papers per year tagged with PaLM: 2022 to 2025, peak 65 65 0 2022: 21 papers 2022 2023: 65 papers 2023 2024: 55 papers 2024 2025: 10 papers 2025
Papers per year the archive tags with this method, by the paper's archive date (151 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Transformers

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections