One place not to overlook for dust is the light trap through which the dark slide moves -- that felt in there is usually full of it. Every 3rd or 4rth film loading I take the slides out, blow through the trap, rap it hard, slam the slide home through it etc...
Also, when loading them up, I clean the holders and slides and stack them, slides closed, in a pile and only open the slides the bare minimum necessary to slip the film in -- less time and motion for dust to get in there. Who knows whether that really does anything, but in theory it should.