Although I don't have an SLR mount-to-M mount adapter, I do have M42-to-Pen F and Nikon F-to-Pen F adapters. But I rarely use them. The main problem is that lenses designed for SLRs are disproportionately large, particularly in barrel diameter, compared to the small Pen F body. So at least for my taste, the combination of lens on adapter plus camera body makes for an awkward shape and awkward handling. Working with native Pen F lenses is just way more comfortable. I suspect it would be the same with SLR lenses on a Leica M.
PS: Sure, there are folks out there who are comfortable with their Noctiluxes and new-generation wide-angle mega-Summiluxes who might not mind.