You should be fine doing that. Once the developer step and stop bath is completed, even though the paper isn't technically insensitive, there'll be no more development; only printing-out (a slow process by comparison) will produce more silver, and that won't generate dye and will be bleached away in the bleach or blix.
Large temperature swings are generally bad, but even in C-41 and E-6, the baths after developer have significantly wider temperatue tolerance (several degrees vs. a fraction of a degree C). If temperature is a concern for you, you might want to be aware that RA-4 can be done at room temperature, too, obviating the need for water baths and close control. Takes longer, and will require rebalancing your filtration, but IMO it's worth that.