Make sure the sensor, lens, and work to be copied (neg or transparency in this case) are all in alignment. Use a copy stand, if you can (tripod if you don’t have a copy stand), and either a laser leveling tool or a spirit level. I use an Omega spirit level designed for aligning enlargers, which can be calibrated to one plane, so that you can adjust the other planes to align with the first one.
How much overlap do you have between frames? When I want to do a hi-res scan, I may shoot 6 or 9 frames, and there should be at least 20% overlap between frames to get good alignment. I use Photomerge in PS5.
Some people make jigs to move the copy work in perfect alignment with the camera or vice versa, but I’ve found this to be unnecessary. Photomerge has no problem aligning panels that have been rotated, as long as they are all in the same plane.
Sometimes it may turn out that the whole stitched image is crooked, but then just straighten it out using the crop and rotate tool. That can cause a slight loss in resolution, because you’re cropping out the misaligned portion of the frame, but this is minor, and if you really need it, that’s the time to consider making a jig that lets you keep the work perfectly straight with respect to the sensor or just shoot more frames per image to get a higher resolution.