I'm guilty of rarely waxing my car, but I've never seen that. The modern waxes can really make your car look great for a while. They help hide a lot of micro swirls etc. I think it really depends on which carnauba or hybrid wax you use. Carnauba wax is essentially gone in a 6 weeks to a few months, hybrids last upwards of 6 to 8 months and paint sealants can last over a year. There are even specialty waxes that add color to enhance black or dark colored cars.

If you are one of those folks who still wash your car with water, dish washing soap will strip wax in one one or two washings.