The cable for the sub is just a coaxial cable with no ends, so I need to stick an RCA end on, which I've never done. I tried it, but I get some serious hum when I power on the sub, ...

But to say that RG-6 is not a good video quality cable for analog video is hogwash. Doesn't matter what the core of the conductor is, even at 1Mhz, the bulk of the signal is not in there. More options ...