People talk about tradeoffs with GC, the worst one is that I've seen an occasional game that has a terrible GC pause, for instance Dome Keeper based on Godot which also runs in .NET. I used play a lot of PhyreEngine (also .NET) games on the Playstation Vita and never noticed GC pauses but I think those games did a gc on every frame instead of letting the garbage pile up.
Not all GCs are created equally either. Unity, for example, is based on an ancient version of Mono and so it uses the Boehm GC which is significantly slower than the one used by .NET. Godot probably has two GCs because it primarily runs GDScript (their custom language) and only supports using .NET in a separate engine build. They'll all have their own performance characteristics that the developer will need to adjust for.
If you're up for it you should give it another try. Your example of subclassing GenericType<X> and GenericType<ConcreteClass> may be supported with covariance and contravariance in generics [1]. It's probably not very well known among C# developers (vs. basic generics) but it can make some use cases a lot easier.
[1] https://learn.microsoft.com/en-us/dotnet/standard/generics/c...