Going to take a stab at it and say that is a early GM/ Corvette style M/C size 1 1/8" bore. That is most likely the start of your problems. They would assume that with that size bore you would have some power assist behind it. Not a great combo for a manual set-up.
If you are willing to spend some cash, go hydroboost and all your problems will magically disappear. Less adventerous folks would say get a 4 hole to 2 hole adapter and put a 15/16 M/C bore on there.
