ASCII DEL (^?) is natively a ‘forward delete’ operation; it erases the character under the cursor while advancing to the next character. That's why its value is 0x7F: on paper tape it's all holes, so it can punch over any other character.

The problem arose from two later, non-Unix (and arguably deliberately anti-Unix¹) influences.

One, as the Debian notes linked in a sibling comment mentions, was GNU Emacs, whose bindings came from custom non-ASCII, bucky-bit² devices, and gleefully stepped on ASCII control characters like Backspace (^H) and Device Control 3 (^S).

The other was DEC's new VT2x0 series of terminals, with the VMS-oriented LK201 keyboard³, that hid Backspace and Escape on unlabelled function keys (F12 and F11 respectively) and made the key in the typist's backspace position send DEL. (The VT100⁴ had had both Backspace, in the typist's location, and DEL, where US PC keyboards have \|.) This was the big one, as BSD 4.2 for the VAX spread and dragged the LK201 along with it.

¹ http://richard.esplins.org/static/downloads/unix-haters-hand...

² http://en.wikipedia.org/wiki/Bucky_bit

³ http://deskthority.net/wiki/DEC_LK201

⁴ http://terminals.classiccmp.org/wiki/index.php/DEC_VT100

The best policy to achieve a consistent behavior from all programs for backspace, delete, Ctrl-H, etc. is the one used by the Debian project:

https://www.debian.org/doc/debian-policy/ch-opersys.html#s9....

This was one of the things that convinced me to start using Debian more than 15 years ago. They, like no other Unix at the time, had looked really deeply into the problem and come up with a good configuration to best fit all programs and systems. I, as a system administrator who was familiar with this problem, was very impressed by their solution, and I deemed it better than the one I had (which I have since forgotten).