Supplementary Material for Paper "Distilling Event Sequence Knowledge From Large Language Models"

Supplementary Material for Paper:
Distilling Event Sequence Knowledge From Large Language Models
Somin Wadhwa, Oktie Hassanzadeh, Debarun Bhattacharjya, Ken Barker, and Jian Ni

Appendix:
- Appendix.pdf: contains our prompts and a description of our human evaluation details.

Data:
- base_kg_v7.jsonl: Our Wikidata-based Event Causal Knowledge Graph.

Outputs:
- sample_new_patterns_discovered.txt: examples of observed new patters through application of sequential pattern mining algorithms, described in section 3.
- precision_eval_sample.txt: examples of output evaluated with a precision-evaluator model. 
- bsumm_output.txt: sample outputs of identified influencing events through the application of binary summary markov model, described in section 5.2.

Code:
- src/generator.py: ingests ICL prompts and generates requisite event sequences.
- src/benchmarking.py: ingests a _trained_ Flan-style seq2seq model to evaluate precision, and recall from the base KG.
- src/utils.py: utilities for generator and benchmarking.

To cite:

@inproceedings{WadhwaHBBN24,
  author    = {Somin Wadhwa and
               Oktie Hassanzadeh and
               Debarun Bhattacharjya and
               Ken Barker and
               Jian Ni},
  title     = {Distilling Event Sequence Knowledge From Large Language Models},
  booktitle = {The Semantic Web - 23rd International Conference, {ISWC} 2024},
  series    = {Lecture Notes in Computer Science},
  publisher = {Springer},
  year      = {2024},
}