Tell them you're running a fever and barfing...
My gut reaction is they're basically looking for higher assessments, in the form of improvements.
Which is essentially a 2nd tax.
I suppose that, "push come to shove" they can get in there, but I believe it to be an invasion of privacy, unless there's cause to believe illegal activity.
And even that is subject to interpretation.
This ^^^^^^^^ Our county is always seeking additional revenue. I know for a fact if you don't let them in,they ASSUME you made improvements. Happened to two friends of mine in the same county here. I let the guy in and he looked around. Asked if I did any improvements. I did and he wrote them down. I also showed him my basement that had damaged paneling from the last flood. I would not let him into the spare bedroom because that is a storage area and full of junk. He had no problem with that. Make sure the guy shows his official ID and call the cops if there is anything suspicious...