I think you were mixing it up with the PS3, which initially had full PS2 hardware backwards compatibility before switching for a short time to a hybrid hardware/emulation setup to cut cost prior to removing the final chip entirely and eliminating the feature. PS1 support remains to this day however, thanks to being emulation based.
49% as I recall of the original Xbox library is on the 360's backwards compatibility list, so it's odd you never got any up and running. Is your 360 equipped with a hard drive, rather than something like a USB memory stick? You need the HDD to enable Xbox backwards compatibility.
Furthermore, it needs to be an official hard drive. For some reason, those that sell unofficial hard drives seemingly always forget to include the partition where Xbox emulation profiles reside, breaking the feature (Although it's possible to resolve this issue yourself).
But like I said, between the games that won't run and the games with issues, only about 10% of the library truly qualifies as compatible in my eyes.