Probably both. Your Chromecast is plugged into the TV which only plays stereo. So all the Chromecast can output will be stereo. So Plex will convert anything more than stereo back to stereo. You can see if this is true by looking at the Plex logs while playing a video. It will tell you...
The only way to pass 5.1 is thru optical (or digital coax). The RCA jacks will be stereo only.
There may be TVs that pass digital audio but I have found any (not that I've looked - just a general consensus on boards like these).
Maybe you could get a bluray player with built in apps? Many...
The TV is probably outputting everything as stereo. Most TVs will output 5.1 only for the built in apps and tuner.
Do you have any other options? Maybe a PC that can be connected to the TV and the AVR? Or maybe it is time to upgrade.
FWIW - I had an HK 325 and also really liked it.