Papers › VICTOR: a Dataset for Brazilian Legal Documents Classification

VICTOR: a Dataset for Brazilian Legal Documents Classification

1 May 2020LREC 2020 5archive 2025-07-28

Pedro Henrique Luz de Araujo, Te{\'o}filo Em{\'\i}dio de Campos, Fabricio Ataides Braz, Nilton Correia da Silva

This paper describes VICTOR, a novel dataset built from Brazil{'}s Supreme Court digitalized legal documents, composed of more than 45 thousand appeals, which includes roughly 692 thousand documents{---}about 4.6 million pages. The dataset contains labeled text data and supports two types of tasks: document type classification; and theme assignment, a multilabel problem. We present baseline results using bag-of-words models, convolutional neural networks, recurrent neural networks and boosting algorithms. We also experiment using linear-chain Conditional Random Fields to leverage the sequential nature of the lawsuits, which we find to lead to improvements on document type classification. Finally we compare a theme classification approach where we use domain knowledge to filter out the less informative document pages to the default one where we use all pages. Contrary to the Court experts{'} expectations, we find that using all available data is the better method. We make the dataset available in three versions of different sizes and contents to encourage explorations of better models and techniques.

PaperPDFConference PDFCode

Code

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

ClassificationGeneral Classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Multi-Label Text Classification BVICTOR XGBoost Average F1 0.8843 #1 of 3 Archive leaderboard report
Multi-Label Text Classification BVICTOR XGBoost Weighted F1 0.8957 #1 of 3 Archive leaderboard report
Multi-Label Text Classification BVICTOR SVM Average F1 0.7761 #2 of 3 Archive leaderboard report
Multi-Label Text Classification BVICTOR SVM Weighted F1 0.8235 #2 of 3 Archive leaderboard report
Multi-Label Text Classification BVICTOR NB Average F1 0.6335 #3 of 3 Archive leaderboard report
Multi-Label Text Classification BVICTOR NB Weighted F1 0.6955 #3 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) XGBoost Average F1 0.8882 #1 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) XGBoost Weighted F1 0.9072 #1 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) SVM Average F1 0.6642 #2 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) SVM Weighted F1 0.8137 #2 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) NB Average F1 0.3797 #3 of 3 Archive leaderboard report
Multi-Label Text Classification MVICTOR (theme) NB Weighted F1 0.6062 #3 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) XGBoost Average F1 0.8887 #1 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) XGBoost Weighted F1 0.8634 #1 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) SVM Average F1 0.8246 #2 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) SVM Weighted F1 0.8231 #2 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) NB Average F1 0.5121 #3 of 3 Archive leaderboard report
Multi-Label Text Classification SVICTOR (theme) NB Weighted F1 0.4875 #3 of 3 Archive leaderboard report
Text Classification MVICTOR (type) CNN + CRF Average F1 0.7505 #1 of 5 Archive leaderboard report
Text Classification MVICTOR (type) CNN + CRF Weighted F1 0.9537 #1 of 5 Archive leaderboard report
Text Classification MVICTOR (type) BiLSTM Average F1 0.7092 #2 of 5 Archive leaderboard report
Text Classification MVICTOR (type) BiLSTM Weighted F1 0.9433 #2 of 5 Archive leaderboard report
Text Classification MVICTOR (type) CNN Average F1 0.7061 #3 of 5 Archive leaderboard report
Text Classification MVICTOR (type) CNN Weighted F1 0.9464 #3 of 5 Archive leaderboard report
Text Classification MVICTOR (type) SVM Average F1 0.6792 #4 of 5 Archive leaderboard report
Text Classification MVICTOR (type) SVM Weighted F1 0.9288 #4 of 5 Archive leaderboard report
Text Classification MVICTOR (type) NB Average F1 0.4772 #5 of 5 Archive leaderboard report
Text Classification MVICTOR (type) NB Weighted F1 0.8477 #5 of 5 Archive leaderboard report
Text Classification SVICTOR (type) CNN + CRF Average F1 0.7740 #1 of 5 Archive leaderboard report
Text Classification SVICTOR (type) CNN + CRF Weighted F1 0.9533 #1 of 5 Archive leaderboard report
Text Classification SVICTOR (type) SVM Average F1 0.7632 #2 of 5 Archive leaderboard report
Text Classification SVICTOR (type) SVM Weighted F1 0.9425 #2 of 5 Archive leaderboard report
Text Classification SVICTOR (type) CNN Average F1 0.7584 #3 of 5 Archive leaderboard report
Text Classification SVICTOR (type) CNN Weighted F1 0.9472 #3 of 5 Archive leaderboard report
Text Classification SVICTOR (type) BiLSTM Average F1 0.7281 #4 of 5 Archive leaderboard report
Text Classification SVICTOR (type) BiLSTM Weighted F1 0.9465 #4 of 5 Archive leaderboard report
Text Classification SVICTOR (type) NB Average F1 0.5979 #5 of 5 Archive leaderboard report
Text Classification SVICTOR (type) NB Weighted F1 0.8893 #5 of 5 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections