{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ladiff-ulmfit-a-layer-differentiated-training","title":"LaDiff ULMFiT: A Layer Differentiated training approach for ULMFiT","arxiv_id":"2101.04965","date":"2021-01-13","proceeding":null,"authors":["Mohammed Azhan","Mohammad Ahmad"],"abstract":"In our paper, we present Deep Learning models with a layer differentiated training method which were used for the SHARED TASK@ CONSTRAINT 2021 sub-tasks COVID19 Fake News Detection in English and Hostile Post Detection in Hindi. We propose a Layer Differentiated training procedure for training a pre-trained ULMFiT arXiv:1801.06146 model. We used special tokens to annotate specific parts of the tweets to improve language understanding and gain insights on the model making the tweets more interpretable. The other two submissions included a modified RoBERTa model and a simple Random Forest Classifier. The proposed approach scored a precision and f1 score of 0.96728972 and 0.967324832 respectively for sub-task \"COVID19 Fake News Detection in English\". Also, Coarse-Grained Hostility f1 Score and Weighted FineGrained f1 score of 0.908648 and 0.533907 respectively for sub-task Hostile Post Detection in Hindi. The proposed approach ranked 61st out of 164 in the sub-task \"COVID19 Fake News Detection in English and 18th out of 45 in the sub-task Hostile Post Detection in Hindi\".","url_abs":"https://arxiv.org/abs/2101.04965v1","url_pdf":"https://arxiv.org/pdf/2101.04965v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ladiff-ulmfit-a-layer-differentiated-training","repo_url":"https://github.com/sheikhazhanmohammed/AAAI-Constraint-Shared-Tasks-2021","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"fake-news-detection","task_name":"Fake News Detection"}],"methods":[{"method_slug":"awd-lstm","method_name":"AWD-LSTM"},{"method_slug":"activation-regularization","method_name":"Activation Regularization"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropconnect","method_name":"DropConnect"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"embedding-dropout","method_name":"Embedding Dropout"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"roberta","method_name":"RoBERTa"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"slanted-triangular-learning-rates","method_name":"Slanted Triangular Learning Rates"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"temporal-activation-regularization","method_name":"Temporal Activation Regularization"},{"method_slug":"ulmfit","method_name":"ULMFiT"},{"method_slug":"variational-dropout","method_name":"Variational Dropout"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"weight-tying","method_name":"Weight Tying"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}