It's called window setting tape and yup it's a pain to get out. AFAIK the originals were not bonded in any way, just the compression of the rubber held the glass in. Mine had also had some later 'help' by a previous owner with some urethane or silicone

My windows were out of the car, and I got them off by caution and running a thin blade down the sides and under the bottom of the tape. Easier said than done, you have to take it bit by bit, I used some thinners as well during the process, but I did it to both sides with no damage (I planned to reuse my windows. If you are not reusing yours I guess you don't need to be as cautious.

If they won't budge or are not accessible in the car, taking them out might be your best bet......