Been discussed a bit internally with the Chrome team, but here's a public issue to track and discuss this.

Currently LoAF is a separate view of Performance Timeline entries, meaning there is no direct link to an INP entry. As one of the primary reasons for this new API was as a tool for improving responsiveness (as measured by INP) should it be easier to get the LoAF entries for an INP entry?

Currently, this requires quite a lot of processing (example WhyNP script).