You have to grind some of the block away. The manifold is making contact with the block. I've done this conversion several times (318 to early A-body).

I have a 84 318 in the garage that I just took out of a 66 Dart with 273 manifolds if you need a pic