"Blue stuff" sounds like Evans, which supposedly doesn't freeze.
On my car (sig pic), even though it always lived in a heated shop, our power occasionally goes out in winter.
I've always just drained its water & Water Wetter (block drains & radiator) and never had an issue.
Water expands when it freezes, but if there's air in the system, I don't think it'll crack anything.
Another alternative is a block heater, or external heat.