Of all the many things involved with restoring a car, it is hard to pin down any one thing that is the worst that everyone will agree on.
What annoys one guy may be an attractive challenge to another.
I hate waiting on someone else to do their job so I can do mine. One example, Machine Shop services. Some places don't seem all that busy when you drop your stuff off but somehow you still get l-o-n-g delays for the completion of your stuff.
I'm a decent bodyman and a mediocre welder but I can make my way through metal replacement and body work.
Finding parts for many of our cars is a matter of luck and timing. Many one year models make it hard to find fenders or grilles that are not reproduced.
1966-1967-1968-1969-1970 A body cars all use a different fender for each year.
1967-1968-1969-1970-1971-1972-1973 Chargers also use a different fender and grille for each year as well. Finding that needle in the haystack can be extremely difficult.