Papers › Predicting Subjective Features of Questions of QA Websites using BERT

Predicting Subjective Features of Questions of QA Websites using BERT

24 Feb 2020ICWR 2020 2arXiv:2002.10107archive 2025-07-28

Issa Annamoradnejad, Mohammadamin Fazli, Jafar Habibi

Community Question-Answering websites, such as StackOverflow and Quora, expect users to follow specific guidelines in order to maintain content quality. These systems mainly rely on community reports for assessing contents, which has serious problems such as the slow handling of violations, the loss of normal and experienced users' time, the low quality of some reports, and discouraging feedback to new users. Therefore, with the overall goal of providing solutions for automating moderation actions in Q&A websites, we aim to provide a model to predict 20 quality or subjective aspects of questions in QA websites. To this end, we used data gathered by the CrowdSource team at Google Research in 2019 and a fine-tuned pre-trained BERT model on our problem. Based on the evaluation by Mean-Squared-Error (MSE), the model achieved a value of 0.046 after 2 epochs of training, which did not improve substantially in the next ones. Results confirm that by simple fine-tuning, we can achieve accurate models in little time and on less amount of data.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

Moradnejad/Predicting-Subjective-Features-on-QA-Websites officialmentioned in papermentioned on GitHubSyntology: not harvested report
google-research/bert tfApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and... (arXiv:2302.05574; that paper's own run record: 17 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 1 honoured, 0 violated, 12 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
graykode/nlp-tutorial pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Efficient Estimation of Word Representations in Vector Space (arXiv:1301.3781; that paper's own run record: 43 ran (of which 7 constructed an object rather than computing a result; 38 with no instrument failure: 4 honoured, 0 violated, 34 with no contract checked; 5 where Syntology's instrument failed) · 19 unverified), Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (arXiv:1406.1078; that paper's own run record: 13 ran (of which 6 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 3 where Syntology's instrument failed) · 9 unverified), Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
huggingface/transformers pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified), HuggingFace's Transformers: State-of-the-art Natural Language Processing (arXiv:1910.03771; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified), DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
tensorflow/models tfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (arXiv:1502.03167; that paper's own run record: 19 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified), Deep Residual Learning for Image Recognition (arXiv:1512.03385; that paper's own run record: 254 ran (of which 108 constructed an object rather than computing a result; 163 with no instrument failure: 3 honoured, 1 violated, 159 with no contract checked; 91 where Syntology's instrument failed) · 123 unverified), Wide Residual Networks (arXiv:1605.07146; that paper's own run record: 66 ran (of which 31 constructed an object rather than computing a result; 47 with no instrument failure: 0 honoured, 0 violated, 47 with no contract checked; 19 where Syntology's instrument failed) · 30 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Community Question AnsweringQuestion Answering

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Common Sense Reasoning CrowdSource QA BERT MSE 0.046 #1 of 1 Archive leaderboard report
Community Question Answering CrowdSource QA BERT MSE 0.046 #1 of 1 Archive leaderboard report
Question Quality Assessment CrowdSource QA BERT MSE 0.046 #1 of 1 Archive leaderboard report
Reading Comprehension CrowdSource QA BERT MSE 0.046 #1 of 1 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

AdamAttentionAttention DropoutBERTDense ConnectionsDropoutLayer NormalizationLinear LayerLinear Warmup With Linear DecayMulti-Head AttentionResidual ConnectionSoftmaxWeight DecayWordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections