He simply didn't visualize composition in color, although his relatively few color shots were tasteful. He was also hung up on Zone System manipulations, and really didn't understand how color contrast in controlled. AA was surrounded by friends and neighbors who were competent dye transfer printers; but I still find it inexplicable that he himself seemingly did not ever seriously experiment with contrast control masking.
And I suspect an amount of jealousy was in play when Eliot Porter's color work was overtaking his own style and becoming the predominant form of visual communication in the conservation movement. That was one of the main points of contention between Adams and David Brower for control of the Sierra Club, and behind its split. None of these people were devoid of egos; but one primary underlying factor was the significantly greater expense of high-quality color reproduction back then, which was draining Sierra Club funds, yet at the same time, garnering tremendous public interest just like Brower argued. I still have a copy of Porter's iconic Glenn Canyon book, with its varnished pages. That level of color bookmaking would simply be unaffordable today.
As far as Polaroid goes, they were essentially a cash-poor start-up when they asked AA to do sample work with their products. So they paid him with stock instead. That stock was one of the main sources of his personal wealth later on.
As per Kodachrome, it was available in sheet film sizes then, so a bit of grain was irrelevant. It was a beautiful product, and even Edward Weston made a few iconic images on Kodachrome, even though he felt uncomfortable with color himself.