Why do people use GPUs instead of CPUs?

For some workloads, an FPGA is orders of magnitude faster than a CPU.