Around here it's a $100 a day fine for any unapproved variances, last time I looked. You might find that your new to you building is now in violation and get fined till you fix the violation, said fix being demolition, which needs a permit around here as well. Best at least consult an attorney familiar with the relevant statutes. Now if I was out in the county rather than the city limits this would not apply.