Seems you should be able to disable just the BT Airplay, not all BT capability, if you want:
https://www.hometheaterforum.com/community/threads/airplay-confusion.374003/post-5055256
_Man_
From the Apple link @Clinton McClure provided, probably wanna change this AirPlay setting on your ATV4K to the 2nd option, if that's not what you already have... unless you really want the peer-to-peer BT capability:
_Man_
But won't that mean disabling the included Siri remote too?
I don't usually use mine, but I'm finding there are some specific instances/uses my Harmony 665 remote don't/can't seem to match, so I keep mine at hand just for those instances...
_Man_