Policy Compliance Detection via Expression Tree Inference
By Neema Kotonya, Andreas Vlachos, Majid Yazdani, Lambert Mathias, Marzieh Saeidi. 2022
Policy Compliance Detection (PCD) is a task we encounter when reasoning over texts, e.g. legal frameworks. Previous work to address PCD relies heavily on modeling the task as a special case of Recognizing Textual Entailment. Entailment is applicable to the problem of PCD, however viewing the policy as a single proposition, as opposed to multiple interlinked propositions, yields poor performance and lacks explainability. To address this challenge, more recent proposals for PCD have argued for decomposing policies into expression trees consisting of questions connected with logic operators. Question answering is used to obtain answers to these questions with respect to a scenario. Finally, the expression tree is evaluated in order to arrive at an overall solution. However, this work assumes expression trees are provided by experts, thus limiting its applicability to new policies. In this work, we learn how to infer expression trees automatically from policy texts. We ensure the validity of the inferred trees by introducing constrained decoding using a finite state automaton to ensure the generation of valid trees. We determine through automatic evaluation that 63% of the expression trees generated by our constrained generation model are logically equivalent to gold trees. Human evaluation shows that 88% of trees generated by our model are correct.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Po-Sen Huang
- Geoffrey Irving
- Jianfeng Gao
- Mohit Bansal
- Iason Gabriel
- Abigail See
- John F. J. Mellor
- Susannah Young
Citations in the corpus
- Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “…of collecting data via detailed rules is that conflicts and weighting between rules can be changed after the fact: Saeidi et al. (2021) express policies as expression trees with rules as the leaves, with the expression either written by experts or inferred from prose (Kotonya et al., 2022).”