BP^C (A Benchmark Dataset for Causal Business Process Reasoning)

Large Language Models (LLMs) are increasingly used for boosting organizational efficiency and automating tasks. While not originally designed for complex cognitive processes, recent efforts have further extended to employ LLMs in activities such as reasoning, planning, and decision-making. In business processes, such abilities could be invaluable for leveraging on the massive corpora LLMs have been trained on for gaining a deep understanding of such processes. In adherence to this goal, we attach here the BPC dataset, a newly developed set of process-aware Q&A that can be used to assess the ability of LLMs to reason about causal and process perspectives of business operations. We refer to this view as Causally-augmented Business Processes (BP^C). The benchmark comprises a set of domain-specific BPC related situations, a set of questions about these situations, and a set of ground truth answers to these questions. Reasoning on BP^C is of crucial importance for process interventions and process improvement. The benchmark could be used in one of two possible modalities: testing the performance of any target LLM and training an LLM to advance its capability to reason about BP^C.