In the US, mineral rights are land use rights. It allows you to extract minerals, which has priority over some other land use rights. You are responsible for restoring lower priority use rights only to the extent you have deprived them. You are not responsible for any deprivation of use rights you did not create, which existing mines would be.

I have owned water, mineral, grazing, and other rights in the western US in environments where these were all considered valuable. YMMV.