My reasoning goes this way:
1) I want to be able to bounce safelight around the darkroom to make the place as bright, comfortable and pleasing to work in as possible.
2) I use amber safelights a lot, but at times also use red when necessary for materials that might fog in amber light.
3) Semi-gloss white reflects all colors fairly equally without glare, and will thus take on the color of whatever safelight is being used, reflecting the maximum amount of light around. Gray would be neutral, but then reflect only a fraction of the safelight (18% gray reflects only 18% of the light falling on it. Even "skin tone" is only 36% reflectance. White gets up to the 90% range.)
4) Yellow would probably do this as well for the yellow and red safelights, but then everything would take on a yellow cast when the white light was on. This is a problem because...
5) I want to be able to evaluate print tone in as neutral an environment as possible, so white is needed so that I'm not projecting a color-cast from the walls onto my prints. (This is valid for green, blue and other-colored walls as well.) Illumination needs to be taken into account for viewing prints too, but this is separate from the wall-color issue. If I were only developing film, this would not be an issue, but since I print there too, white seems the only way to go.
6) Every one of the Beseler 45 and Omega D enlargers I have worked with spills some unwanted light from the negative carrier/head, no matter what I do in the way of adjusting light source and carrier holder. This has a tendency to bounce off white walls and could possibly fog prints during long print exposures. Yellow (i.e., safe-colored) walls would fix this, but would result in the color-cast problem mentioned above. Therefore...
7) I have surrounded my enlarger's light leaks with flat-black as much as possible. Sometimes this is just a flat-black mat board mounted on the wall, sometimes a baffle or frame hung from the enlarger itself, whatever works in the setup I have. The object is to catch stray light with a minimum of black surface area while leaving the rest of the darkroom white and reflecting safelight. I could maybe get the same result with yellow and red and have a more cheerful (or at least more colorful) darkroom. This is a definite possibility.
Final verdict: Semi-gloss (or flat) white walls for as much of the darkroom as possible. Flat-black (or yellow/red) around enlarger heads to limit unwanted light from the head/negative carrier. I also find it helpful to have safelights around the enlargers that can be turned off easily (handy switch or chain) so that unwanted light from the safelights doesn't swamp detail in the print when focusing.
Boy, that got long real fast... Hope it helps,
Best,
Doremus Scudder
www.DoremusScudder.com