So what you'd need is a chemical, or actually chemicals, that not only react to light in the way that silver nitrate does, but also have bond-lengths such that each different chemical only reacts to a certain wavelength of light.
Presumably, this would mean that once a photon of a certain wavelength is absorbed by such chemical, it breaks the bond and the chemical transforms into something else, and no further photons can be absorbed by this bond.
So you'd need, say, 3 different chemicals that contain bond lengths of 680nm for red, 520nm for green, 450nm for blue, that would break or otherwise transform when hit with a photon of that wavelength, then the resulting chemical left would be developable into something of that colour (or its inverse for negatives). For even more accurate colours, you could add in more chemicals at something like 700nm, 600nm, 480nm, or 400nm.
All of these would presumably be coated in a single coating if they could then be directly-developable into colour, or they'd still be in layers if they had to develop something with dye-coupling (as I understand colour films work now).
Definitely possible, how long have you got to research different chemicals? Could take rather a long time to find them...