Papers › Improving Cross-Lingual Transfer through Subtree-Aware Word Reordering

Improving Cross-Lingual Transfer through Subtree-Aware Word Reordering

20 Oct 2023arXiv:2310.13583archive 2025-07-28

Ofir Arviv, Dmitry Nikolaev, Taelin Karidi, Omri Abend

Despite the impressive growth of the abilities of multilingual language models, such as XLM-R and mT5, it has been shown that they still face difficulties when tackling typologically-distant languages, particularly in the low-resource setting. One obstacle for effective cross-lingual transfer is variability in word-order patterns. It can be potentially mitigated via source- or target-side word reordering, and numerous approaches to reordering have been proposed. However, they rely on language-specific rules, work on the level of POS tags, or only target the main clause, leaving subordinate clauses intact. To address these limitations, we present a new powerful reordering method, defined in terms of Universal Dependencies, that is able to learn fine-grained word-order patterns conditioned on the syntactic context from a small amount of annotated data and can be applied at all levels of the syntactic tree. We conduct experiments on a diverse set of tasks and show that our method consistently outperforms strong baselines over different language pairs and model architectures. This performance advantage holds true in both zero-shot and few-shot scenarios.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

ofirarviv/ud-based-word-reordering officialmentioned in papermentioned on GitHubSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Cross-Lingual TransferPOSXLM-R

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

AdafactorAttentionAttention DropoutBPEDense ConnectionsDropoutGated Linear UnitInverse Square Root ScheduleLayer NormalizationLinear LayerMulti-Head AttentionResidual ConnectionSETSentencePieceSoftmaxT5XLM-RmT5

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections