I would not touch them unless you know where to get replacements. They are frequently very thin and fragile and almost impossible to replace.
For example, the yellow filter will reflect the blue wavelengths away, if the blue waves get trapped in dirt on the filter or reflect into the black paint in the head and get trapped there, what would be the difference? The yellow still comes through.
Personally I'd concentrate on alignment of negative and lens and centering of lens, make sure the lamp socket is clean, maybe replace that as they have limited life expectance. Make sure cords are in good shape. Find a good lens, maybe tear the lens apart to clean well, look for light leaks in bellows and head and plug them, make sure bottom diffuser is clean, any tiny imperfection there could show up. Maybe get new negative glass if there are any scratches. Every used negative carrier I have bought had scratches and chips in the glass.
I just replaced a socket in one of my heads yesterday. Resist the temptation to poke around where the filters are unless the enlarger is broken and the filters don't move.