It's already been covered, but generally you won't see an issue with step up filters if you keep the number of steps to a minimum (ideally 1 ring). For example, if you have a 26mm filter thread on your lens, but only own a 77mm polarizer, your best bet is to get a 26 -> 77mm step up ring rather than stacking a 26 -> 30, then a 30 -> 49, then a 49 -> 58, then a 58 -> 62, etc. etc. The reason being essentially that the closer your filter is to the lens, the more out of focus it will be. The further it gets from the lens, the more noticeable things like dust, scratches, etc. will be in your photos. When using a single step up ring I can't discern any loss in quality.
Step down rings are to be avoided whenever possible. It's just asking for vignetting.