{"url":"/method/coberl","slug":"coberl","name":"CoBERL","full_name":"Contrastive BERT","full_name_withheld":false,"description_markdown":"**Contrastive BERT** is a reinforcement learning agent that combines a new contrastive loss and a hybrid [LSTM](https://paperswithcode.com/method/lstm)-[transformer](https://paperswithcode.com/method/transformer) architecture to tackle the challenge of improving data efficiency for RL. It uses bidirectional masked prediction in combination with a generalization of recent contrastive methods to learn better representations for transformers in RL, without the need of hand engineered data augmentations.\r\n\r\nFor the architecture, a residual network is used to encode observations into embeddings $Y\\_{t}$. $Y_{t}$  is fed through a causally masked [GTrXL transformer](https://www.paperswithcode.com/method/gtrxl), which computes the predicted masked inputs $X\\_{t}$ and passes those together with $Y\\_{t}$ to a learnt gate. The output of the gate is passed through a single [LSTM](https://www.paperswithcode.com/method/lstm) layer to produce the values that we use for computing the RL loss. A contrastive loss is computed using predicted masked inputs $X_{t}$ and $Y_{t}$ as targets. For this, we do not use the causal mask of the Transformer.","description_state":"present","introduced_year":null,"introduced_by":{"title":"CoBERL: Contrastive BERT for Reinforcement Learning","paper":"/paper/coberl-contrastive-bert-for-reinforcement","first_author":"Andrea Banino","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/coberl-contrastive-bert-for-reinforcement"},"source":{"url":"https://arxiv.org/abs/2107.05431v2","title":"CoBERL: Contrastive BERT for Reinforcement Learning","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"RL Transformers","url":"/methods/category/rl-transformers","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/coberl-contrastive-bert-for-reinforcement","title":"CoBERL: Contrastive BERT for Reinforcement Learning","date":"2021-07-12","arxiv_id":"2107.05431","n_code_links":2,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":0}}],"papers_shown":1,"tasks":[{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":1},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":1},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":1}],"tasks_shown":3,"n_tasks":3,"usage_by_year":[{"year":"2021","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/coberl"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}