Bond dissociation energy benchmark

[gelzinyte]

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.

Summary

Performance in predicting C-H bond dissociation energies (BDEs) for 60 CYP3A4 drug-like substrates (CHO elements only), comprising 1117 sp3 C-H bonds across all molecules. The BDEs are computed as E_radical + E_isolated_H - E_molecule. The first metric computes the BDE error directly. The second metric compares in models' ability to predict relative BDEs. This is evaluated by comparing predicted and reference BDE ranks (lowest-highest order assignments) for each molecule and averaged across all molecules in the dataset. Direct BDE and BDE rank prediction is computed on DFT-optimised geometries and on MLFF-optimised geometries.

Linked issue

Resolves #425

Progress

• Calculations
• Analysis
• Application
• Documentation

Testing

• MACE-OFF23
• MACE-MatPES-r2scan
• MACE-MPA-0
• MACE-MP

New decorators/callbacks

A callback showing mlff-optimised structure. Currently as part of the app definition.

[gelzinyte]

BDEs.zip

Structures needed for the test.

[joehart2001]

Suggested change

	level of theory: B3LYP-D3BJ/def2-SV(P)
	level_of_theory: B3LYP-D3BJ/def2-SV(P)

[joehart2001]

Suggested change

	level of theory: B3LYP-D3BJ/def2-SV(P)
	level_of_theory: B3LYP-D3BJ/def2-SV(P)

[joehart2001]

Suggested change

	level of theory: B3LYP-D3BJ/def2-SV(P)
	level_of_theory: B3LYP-D3BJ/def2-SV(P)

[joehart2001]

Suggested change

	level of theory: B3LYP-D3BJ/def2-SV(P)
	level_of_theory: B3LYP-D3BJ/def2-SV(P)

[joehart2001]

Hey @gelzinyte, sorry for the delay, but the PR is looking nice! Would you also be able to send the MLFF opt file so i could uplaod it as well. A few comments:

calc:

• i think its probably best to combine the two calc files into one
• adding tqdm to the calc could be nice to track benchmark progress
• worth adding @pytest.mark.slow above @pytest.mark.parametrize("mlip", MODELS.items()) in the calc

app:

• i think a lot of the custom structure visualisation functions could be replaced with utlities we've written for use in other benchmarks. so for the DFT structures, where all the structures are the same, something like this could be a good idea:

        structs_dir = DATA_PATH / MODELS[0] / "dft_opt"
        xyz_files = sorted(structs_dir.glob("*.xyz"), key=lambda p: int(p.stem))
        structs_dft = [
            f"assets/molecular/BDEs/{MODELS[0]}/dft_opt/{f.stem}.xyz"
            for f in xyz_files
        ]

• for the opt we'd probably want to view the optimised structures for each model. This one is more fiddly so probably best to come back to it once we've got all the data uplaoded so i can properly test it

general:

• rebase

Thanks!

[ElliottKasoar]

Hi @gelzinyte, just wanted to check in and see if there's anything we can do to help?

In addition to the above comments, please can you take a look at our new filtering guidelines: https://ddmms.github.io/ml-peg/developer_guide/filter.html

The principles are relatively simple, but you do have to be a little careful, so again, if anything is unclear, please do ask!

(You'll need to rebase to test this - you may find you need to fix a few imports as part of this too if you rerun things to test, but nothing major!)