{"url":"/dataset/vpfrc-llm-vulnerability-classifier","name":"vpfrc_llm_vulnerability_classifier","full_name":"VPFRC LLM Vulnerability Classifier Data","description_markdown":"# LLM-Based Vulnerability Classification in Police Narratives\r\n\r\nThis repository contains datasets used in our research on applying large language models (LLMs) to identify indicators of vulnerability in police incident narratives. These resources support the replication of findings in our paper: \"Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives.\"\r\n\r\n## Project Overview\r\n\r\nLaw enforcement frequently encounters vulnerable individuals, but identifying vulnerability factors in police records remains challenging. Our research explores how LLMs can assist in identifying four key vulnerability indicators in police Field Interrogation and Observation (FIO) narratives:\r\n\r\n- Mental health issues\r\n- Drug abuse\r\n- Alcoholism\r\n- Homelessness\r\n\r\nThis project advances police research methodology by:\r\n1. Evaluating LLM performance in vulnerability classification against human labelers\r\n2. Comparing different LLM architectures and prompt engineering approaches\r\n3. Investigating potential demographic biases through counterfactual analysis\r\n4. Developing a reusable framework for qualitative text analysis\r\n\r\n## Datasets\r\n\r\nThis repository includes four key datasets:\r\n\r\n- **boston_narratives_test_classified_4000.csv**: 4,000 narratives classified with our LLM pipeline, including all labels and model explanations\r\n- **counterfactual_narratives_all_coded.csv**: Systematically generated counterfactual narratives with varied demographic characteristics\r\n- **examples_for_counterfactuals.csv**: 100 base narratives used for counterfactual generation\r\n- **labelled_fio_data_for_analysis.csv**: 500 pre-processed examples with human and GPT-4o labels\r\n\r\n## Code Repository\r\n\r\nThe complete codebase for replicating our research is available in our GitHub repository: [llm-deductive-coding](https://github.com/yourusername/llm-deductive-coding) (particularly in the `boston_fio_paper` directory).\r\n\r\nThe repository includes:\r\n- Data preprocessing scripts\r\n- Classification pipeline implementation\r\n- Counterfactual generation code\r\n- Analysis notebooks\r\n- Visualization tools\r\n\r\n## Citation\r\n\r\nIf you use these resources in your research, please cite our paper:\r\n\r\n```bibtex\r\n@article{author2023llm,\r\n  title={Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives},\r\n  author={Relins, S. and Birks, D and Lloyd, C},\r\n  journal={Arxiv Preprint},\r\n  year={2023},\r\n  note={Currently under review for the Journal of Quantitative Criminology}\r\n}\r\n```\r\n\r\n## License\r\n\r\nThese datasets are released under the MIT License. The original Boston FIO data is released under the Open Data Commons Public Domain Dedication and License (PDDL).\r\n\r\n## Contact\r\n\r\nFor questions about this research or datasets, please contact the authors or open an issue in our GitHub repository.","description_withheld":null,"homepage":"https://osf.io/har9m/","introduced_date":"2024-12-16","introduced_date_note":null,"introduced_by":{"paper":"/paper/using-instruction-tuned-large-language-models","title":"Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives","first_author":"Sam Relins","url":null},"license":{"name":"MIT","url":null},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["vpfrc_llm_vulnerability_classifier"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}