Technically, a vintage Mopar such as your Challenger has a center link, with the term drag link usually reserved for steering linkages lacking an idler arm. In those setups, the drag link connects directly from the pitman arm to a steering knuckle, with a really long tie rod then going over to the other side.

Now that the picking of nits has been put aside, to your question:

I suppose there are four orientations in which one could try to install a center link. Two of them would be immediately ruled out (as was pointed out previously) by the tapered nature of the attaching components. Of the remaining two possible orientations, the "drop" in the center being slightly down and to the rear (to clear oil pans and the like) is the correct one.

Although there is no interference, the inner tie rod end on the left side passes pretty close to the t-bar on my 1970 'vert (if it's any consolation to you).