I would suggest

    fd -t f . cache -X rm --

Which reads as find "any file", "matching .", "in directory cache", then "execute rm -- followed by an argument list of all found files".

This ensures even if you have filenames starting with - they won't be interpreted as options for rm. For even more sanity of mind you may want to turn on -a for absolute paths, although I don't see an example right now where using relative paths would go wrong.