That's what no-electricity-needed propane heaters are for.
I have a no-electricity natural gas heater in the garage. In a pinch, I can turn that on and open the door to the house and keep it above freezing. I also have back-up electric heat in the house in case the furnace breaks.