Papers › Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation

Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation

24 Aug 2023arXiv:2308.13116archive 2025-07-28

Kevin Krahn, Derrick Tate, Andrew C. Lamicela

Contextual language models have been trained on Classical languages, including Ancient Greek and Latin, for tasks such as lemmatization, morphological tagging, part of speech tagging, authorship attribution, and detection of scribal errors. However, high-quality sentence embedding models for these historical languages are significantly more difficult to achieve due to the lack of training data. In this work, we use a multilingual knowledge distillation approach to train BERT models to produce sentence embeddings for Ancient Greek text. The state-of-the-art sentence embedding approaches for high-resource languages use massive datasets, but our distillation approach allows our Ancient Greek models to inherit the properties of these models while using a relatively small amount of translated sentence data. We build a parallel sentence dataset using a sentence-embedding alignment method to align Ancient Greek documents with English translations, and use this dataset to train our models. We evaluate our models on translation search, semantic similarity, and semantic retrieval tasks and investigate translation bias. We make our training and evaluation datasets freely available at https://github.com/kevinkrahn/ancient-greek-datasets .

Paper PDF Code

Code

kevinkrahn/ancient-greek-datasets officialmentioned in papermentioned on GitHub report

TickleForce/ancient-greek-datasets officialmentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Authorship Attribution Knowledge Distillation Lemmatization Morphological Tagging Part-Of-Speech Tagging Retrieval Semantic Retrieval Semantic Similarity Semantic Textual Similarity Sentence Sentence Embedding Sentence Embeddings Sentence-Embedding Translation

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

ALIGN Adam Attention Attention Dropout BERT Dense Connections Dropout Knowledge Distillation Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections