You aren't getting it tight enough. There is no way a properly tightened pick up will move. Unless the crank or a rod are hitting it and knocking it loose.