# Benchmark overview

## Systems

These benchmarks are done using the "OpenFE year 1 systems", which are
effectively half way point structures in the Protein-Ligand benchmark 0.3.0 refactor.

The structures and input networks are stored in `openfe-benchmarks` commit bebb2ec5ac6c902868865e99a9e75c145b87d088

## Environment

### Simulation submission & gathering

The exact environment used for creating and submitting the benchmark systems can be found
in the file `alchemiscale_environment.yml`.

Specifically we use:
  - OpenFE v0.13.0
  - gufe v0.9.4
  - openeye-toolkits 2023.1.1
  - openff-nagl-modules 0.1
  - openff-nagl-base 0.3.1
  - openff-toolkit 0.14.4+10.gdcd78c97
  - openmm 8.0.0
  - alchemiscale 0.2.1

### Plotting and further analysis

Plotting is done in a separate environment using cinnabar 0.4.1.

## File organisation

### Top-level directory description

- `antechamber`: benchmarks using partial charges using antechamber specifically using the input conformer.
- `antechamber_bad`: benchmarks using partial charges using antechamber w/ a randomly generated conformer.
- `elf_antechamber`: benchmarks using antechamber w/ RDKit elf selection (max N = 10, default RMS criteria), averaging partial charges derived from selected elf conformers
- `oechem`: benchmarks using oechem's AM1BCCELF10
- `nagl`: benchmarks using the NAGL 0.1.0-rc1 model
- `plotting`: plots for each benchmark & comparisons to oechem results

### Individual directory description

Each benchmark contains directories labelled `mcl1`, `hif2a`, `tyk2`, and `thrombin` which each contain:

- `submit.ipynb` which contains a notebook with the submission code.
- `scoped-key.dat` which has the serialized alchemiscale scope key.
- `utils`: some python scripts for checking on the status of a network submission & restarting errors
- `outputs`: serialized protocol results & initial plots
