Could be the squeegee, the rubber compound does detioriate at some point, especially if not dried after use, - I had one that began smearing a black mark, more visible on the reverse, and a hand roller type that did begin 'pitting' on the roller surface, both discarded. Now I drip dry fibre paper until it is damp, before drying it flat. If you use a squeegee, I would recommend the broad bladed darkroom type - I used to use one in a work darkroom - and replace them after a year or two. For RC paper I find a vertical drying rack to be enough and usually finish them off in an RC warm air dryer, which improves the finish of gloss and lustre surfaces.
Apart from that, the specks could be from contamination comng through a mains water supply - not unknown for short periods - or a release of limescale into your system. Try filtering the water, if you haven't already: a water filter is desirable for processing and the relatively inexpensive Pro-Co or Amfilter (?) type that holds a 10 inch cartridge can be run by hose attachment off most taps/faucets (mine is held to the wall by a couple of drainpipe brackets, a hose from tap to inlet then a 90 bend off the outlet with a hose dropping to the sink).
Sorry to go on,
Regards, Mark Walker.