Code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code"

Source code and experiment data from the NeurIPS 2025 paper "Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code"

Installing Dependencies

You'll need uv  installed on your system. Then you can run: uv sync

 API Keys

We support several frameworks (i.e., APIs to communicate with the LLMs). For each API, you need to setup its related environment variable with a valid API key:

- for the Google Gemini API you must define GOOGLE_API_KEY
- for the DeepSeek API you must define DEEPSEEK_API_KEY
- for the OpenAI API you must define OPENAI_API_KEY

Heuristic Generation

The basic command is:

uv run llm-heuristics.py --domain DOMAIN

where `DOMAIN` is the name of the domain for which the heuristic is generated. The domains used in the paper are blocksworld, childsnack, floortile, miconic, rovers, sokoban, spanner, and transport.

There are several other options you can pass to the command line:

- --domain: choose the domain that you want to use.
- --framework: choose what framework you want to use. Available choices are: `gemini` (default), `deepseek`, and `openai`.
- --model: choose the LLM model to be used. The available models depend on the selected framework.
- --prompt-format: choose prompt format. *Note: this option was used during development and is no longer necessary; it will be removed in the future.* (Default: "neurips")
- --heuristic-name: name of the heuristic and of its class.
- --heuristic-file: file where the learned heuristic is stored. It must end with `.py`.
- --temperature: temperature (default: 1.0)
- --top-p: top-k value (default: 0.5)
- --ablation: choose a component to do the ablation. Omit this option if you want the complete prompt.

Pyperplan

To run Pyperplan with your generated heuristic, execute

uv run src/pyperplan/pyperplan.py -s gbfs_early_goal_test -H HEURISTIC-FILE /path/to/domain.pddl /path/to/instance.pddl

where HEURISTIC-FILE is the (relative or absolute) path to the heuristic generated in the previous step.

Alternatively, you can use hff to run the FF heuristic or blind to run the blind heuristic (effectively, no heuristic).

 Example

Let's say you want to learn a heuristic for the `blocksworld` domain with the name `AmazingHeuristic`. Then you should run

uv run llm-heuristics.py --domain blocksworld --heuristic-name AmazingHeuristic --heuristic-file amazing-heuristic.py

Now we can call Pyperplan using the new heuristic with the following command to solve instance
an instance of the `blocksworld` testing set:

uv run src/pyperplan/pyperplan.py -H amazing-heuristic.py -s gbfs_early_goal_test benchmarks/ipc2023-learning/testing/blocksworld/easy-p03.pddl

End-to-End Plan Generation

The basic command is:

uv run end-to-end.py --domain DOMAIN --instance INSTANCE

where `DOMAIN` is the name of the domain of the instance, and `INSTANCE` is the particular instance for which we are computing a plan.

Other available options are:
- --framework: choose LLM framework. Available choices are: `gemini` (default), `deepseek`, and `openai`.
- --model: choose the LLM model. The available models depend on the selected framework.
- --plan-file: file where the computed heuristic is stored.
- --temperature: temperature (default: 0.1)
- --top-p: top-k value (default: 0.5)

Experimental Data

We also include the relevant experimental data for the main contributions of the paper. These are the directories containing experimental data used in the paper:

- archived-heuristics: Contains all heuristics generated by the models used, including the ones for the ablation study.
- logs: Contains all logs from the heuristic generation phase. It contains the prompt, the answer, and the code extracted from the answer.
- benchmarks: Contains all the domains from the Learning Track of the IPC 2023 benchmark used in the paper. The benchmark set is split into training and testing sets.
- experiments: Contains the scripts used for the planning experiments (i.e., Pyperplan experiments). The subdirectory `data` contains the HTML reports summarizing all experiments.