Here's my routine for new "used and often grungy" holders.
First, remove excess dust, dirt, etc. with a damp cloth.
Often, holders have stickers, markings, etc. on them that I don't want. Goo-Gone and fingernail polish remover (both with and without acetone) are your friends here. Old toothbrushes, cotton swabs and even a pocketknife are often needed to get the crud off. Sometimes, if the previous user has written lots of stuff on the white data recording areas of the holder with permanent marker, I simply cut white stick-on labels to size and cover the offending areas.
Pull dark slides and check the film backing plates. If they are flaking, you will need to sand them down with some very fine sandpaper and then repaint them with flat black paint. You could use them as is, but you risk new flakes getting on your film as well as unwanted reflection from the shiny aluminum areas. I use flat black Rustoleum spray paint. Mask the plastic/wood part of the holders with normal masking tape before painting. You can paint the black side of the darkslide handle if it is metal and flaking as well, but I often just use a big Sharpie to cover chips. Plastic slide handles are usually ok.
Getting at the light trap to get dust out is the biggest problem. I find that working the darkslide in and out often dislodges most of the dust. However, I have had some holders that were dirty enough that they warranted taking to the sink and running water down through the light traps while working the slide in and out. On very old, wooden holders this isn't a good idea, but on the more modern plastic Fidelity, Riteway, etc. holders, the water causes no damage. Let them dry completely. This often takes a couple of days, even in dry conditions.
When you are ready to load the new holders, drag out the vacuum cleaner. I run a ring of masking tape around the rectangular brush upholstery attachment, leaving just a quarter-inch of brush or less exposed, to increase the suction. Vacuum the light trap from above and below, vacuum the holder and both dark slides and then, insert the darkslide and pull it in and out of the light trap a few times while holding the vacuum attachment below the trap inside the holder to get those last pieces of dust in the trap. Give the inside of the holder one more suctioning and close the darkslide. Stack your holders under a plastic cover to keep dust from landing on them. I try to vacuum in another room from where I load to keep the vacuum from stirring up dust. If I can't, I run the hot water and let things sit for a while to settle the dust.
This last bit is repeated every time I load holders. Once you get a clean, keep it as clean as possible by reducing exposure to dust. I keep mine in Ziploc bags, which I change at least every other time I load.
Best and good luck,
Doremus Scudder
www.DoremusScudder.com