If you are out driving the car on one of those 3 or 4 days a year, and you get into an accident, would it really matter if that would have been the 120th day you drove it? You will still expect the insurance company to pay the agreed value.
Another aspect you are missing is that annual coverage covers the car all the time. Should a house fire consume the car, even if you haven't driven it at all this year, you would still expect the insurance company to cover your loss.

The premium you are paying is what the insurance company determines is the cost of replacing that car at the agreed price, weather or not your driving it at the time of loss. If you happen to be driving it at the time of loss, the company also covers, up to policy limits, the liability you may be on the line for as well. If you look at your policy, it will tell you how much the vehicle coverage cost and how much the liability costs, there is probably a policy cost, and may even be towing coverage as well added in. I suspect the liability cost is pretty low anyway, and that amount is the only part that could be impacted by how much you drive. The only limiting factor I've ever seen on an insurance policy is how much you drive, not how little you drive.

I have liability insurance only on my coupe. It costs me over $100 a year for liability only even with several discounts (good driving record, low traffic area, & multiple vehicles just to name a few). That would be zero coverage on the car itself. If it gets wrecked and its not my fault, I will get whatever the others persons insurance will pay.

If you think that classic car insurance is a rip off, put just liability coverage on it and drive it. That might save you a $100 a year, but it could cost you a lot more with a total loss. Gene