You hit the nail on the head with your last observation, Stan--not every color shifts with EIR all the time - it depends on what the material the object in question is made of. The more organic components in the object usually means more IR response, and thus a better color shift. For example, metal surfaces very often shift very little, or not at all...silver & chrome especially. In your case, the stone castle and the road are not shifting very much because neither stone nor asphalt(?) contain enough material to generate the IR response. Example of mine: a number of years ago I did some oceanside shooting in Santa Monica, and shot a wave crashing on a barnacle-encrusted rock. The rock did not shift at all (though the color was a bit deeper than the original), but the barnacles turned bright red. Reason: barnacles are organic, rocks are not.
As far as the sky goes, that is dependent on a number of factors - time of day, the angle of the shot & the sun, how much blue is in the sky that original day (i.e. how hazy is it?), and others. In general, skies that have a good amount of blue in them on that particular day become deeper blue. If you use a polarizer, you can go all the way to black sometimes. Without one, late afternoon shots can generate really deep blues when the camera is pointed toward the east when the sun is getting ready to set in the west.