As stated, a red under-lens filter holders and filters are common enlarger accessories, and most enlargers I've dealt with have a provision built in for the filter. I believe Ilford's under-lens set VC filter includes a red filter as well. Depending on your enlarger, the filter may be available as a replacement part, and they show up on ebay for various enlargers. Most are quite similar, and you can likely find one that works that was actually made for a different unit.
As for fogging, certainly you'll want to test whatever filter you obtain, but if stray enlarger light doesn't fog your exposures, it likely won't fog the paper under the filter unless you have the enlarger on for a very long time, which you'd want to avoid anyway.
It's worth another test, but if you're getting fogging from stray light, you have a different problem to solve.