Film speed basically depends on how many Silver atoms need to be in a latent image center such that the grain is rendered developable. Sensitization techniques gave us emulsions which need latent image centers consisting of just three Silver atoms. The fewer atoms form a latent image center, the higher reduction potential the developer needs to have, and not every development agent has the same ability to discriminate between image and fog. And then there are developers which will actively bleach away small latent image centers but develop larger ones. Remember that a small latent image center is not only harder to develop, but also easier to bleach, and it depends on reduction potential of developer whether a latent image center is developed or bleached.
So yes, it is absolutely possible to formulate a developer which creates a fully developed negative but loses one or more stops of speed. The reason why it's not possible to gain a few stops: because we are a bunch of spoiled brats, taking advantage of >120 years of research into photographic emulsions and developers. Almost every single commercial developer sold today will give more or less full speed.