Personally, I thought the D50 is a nice little DSLR -- I had one briefly last year when my old D70 was in the shop for the dreaded BGLOD problem. The D50 is certainly a great value and excellent entry level DSLR. Yeah, the body feels a bit plasticky, but not nearly as much so as the Canon equivalent. It's small-ish (compared to most DSLRs anyway), but not too small for most folks -- the Canon 350D/DRebel XT is smaller w/ a slimmer grip, which some folks do not like.
I did miss not having the front dial, but that might not be much problem if you're not already used to having a front dial. There are a couple other useful buttons missing, but again, might not be a big deal to you. The viewfinder will definitely seem small compared to film SLRs, but that's to be expected for a low-end DSLR (for now anyway) -- the D70/D70s viewfinder is no bigger (nor brighter), so you'll need to go up to the D200 if you want bigger (and a tad brighter). The D50 viewfinder also doesn't have optional on-screen grid lines, if that matters to you. Also, manual ISO settings are in 1-stop steps instead of 1/3-stop steps (like all other Nikon DSLRs), but that's probably no big deal -- but if you want ISO 100, then you'll need to get a D200 or D2X. The D50 (and D70/D70s) also doesn't provide metering support for old manual AI lenses, if that matters to you. And I wasn't too crazy about having to use SD cards for it (vs CF cards for all other Nikons).
On the bright side,

the D50's onboard flash offers better coverage than D70/D70s (and also D200) though it doesn't provide any wireless flash control at all -- the D70/D70s offers basic wireless flash control of iTTL speedlights while D200 offers more comprehensive CLS control (but still not quite equal to an SB800 or dedicated CLS module). Of course, if you put an SB800 (or dedicated CLS module) on top of the D50, then you can have all the wireless control you want anyway.

Also, as Brent says, straight-from-camera D50 images will tend to look more pleasing (to most folks) than most other Nikons although I'd still recommend shooting RAW w/ intention to do some postprocessing, if you want best results (or more control over the final results) -- Auto WB w/ indoor lighting would still be a problem though.
So really, for most folks, there isn't much appreciable difference between D50 and D70/D70s -- and certainly, most casual shooters will actually prefer the D50 over the D70/D70s. The body differences are much more pronounced when you jump to the D200 -- and it also offers higher resolution, but at a modest cost to noise level (at least at the "per pixel" level). However, even then, not everyone will find it worthwhile to pay the big premium for the jump.
_Man_