Let's say we did move the beer bottle. We as the photographer are no longer a neutral observer of the scene (and never really are, especially if people are in it). We have modified the scene. It is a fair question as to whether physically moving the beer bottle before the shot is less of an infraction (or even an infraction) than removing it digitally or in the darkroom. There is a cultural expectation that a photograph represents the moment in time as the photographer recorded it, and thus as it was when he came. I suspect the answer comes in terms of the magnitude of the modification. For instance, physically removing a beer bottle or a piece of dog poop form the scene to be photographed is physically very feasible, and unlikely to create an image which could not have occurred. Removing it digitally later would be an admission that you were not paying attention when you took the shot (or that things happened too fast, or you could not access the beer bottle). But it is modifying the recorded scene. I suspect this is less of an issue than adding a set of clouds that would never be seen in the locale whether ever or that time of year, or that time of day, etc. An even worse situation would be a set of clouds wholly inconsistent with the lighting on the ground. On the other hand such a move (inconsistent clouds) COULD be used for artistic purposes (to create a sense of mystery or unease). The question is, would it be better at that point to call it something other than photography? Maybe Graphic Arts? I don't know, and it is not going to matter much because people are going to manipulate images proportionally to the magnitude of the effect and the technological ease of doing so.
I'd say you are imposing an artificial restriction on art - which is what we're talking about here. Intellectually it's the same argument that the classic painting Academies used against the Impressionists - it's not accurate enough, the subject matter is wrong, etc. etc. We're talking about art photography here, not documentary or evidentiary photography where there is an expectation of high verisimilitude. You have to start with what the intent of the photograph is before you can determine what is and is not acceptable in an image. I think it's a very restrictive and some would even say outdated idea to presume people are expecting 1:1 verisimilitude when looking at a photograph now.
I'll agree with you that after a point, major manipulation of an image ceases to be photography and is in fact Graphic Arts (I've even proposed this as the name for the current movement in photography). But it is hard to separate it out from photography when so many people practicing it self-identify as photographers. When it comes down to it, all photographs are illusions - they proffer a simultaneous high degree of verisimilitude and a complete absence of it - they're rich in detail, but they're tonally, dimensionally, and perspectively completely inaccurate. They offer the illusion of texture and detail, but don't actually provide it. And when zoomed in enough, be it pixels or grains, in the end they're just dots of varying tones arranged to provide the illusion of the thing they claim to represent.