Probably the rubber brake line hose to banjo on the caliper. If that is older and you can't tell if is has broken down, the hose will expand and distort making the caliper "hang" while the brake fluid flow has to decompress through a distorted hose. Change both, bleed the system and brakes are as good as new.
Other option would be to remove the bolt on the caliper holding the rattle plate and rotate the caliper off the rotor. Take some sand paper and really clean the pad sliding groove. Brake clean off the crap and apply some Anti Seize. That will make your brakes better anyway.