Mirrorless camera lenses can be more compact because optical corrections can be done with in-camera software. Equally they can be bigger and use software to allow for better image quality at each end of a wide zoom range, 24-200 for example, where image performance of traditional SLR lenses often degraded at the limits.
I think the answer regarding SLR lenses being larger lies between form and function (the average balance and weight on the camera body), the aperture stop down mechanism, and price. Common sense says all three elements need to be worked in harmony. When Nikon made the F the last thing they wanted was a camera that was fiddly to use, overly complex, or too expensive. On the other hand when the Leica rangefinders were invented there wasn't much extra technology around to load onto cameras so there was no reason to make them bigger. The general smaller size was set in stone by never needing the stop down mechanism and there wasn't a mirror to avoid. The hidden advantage over the years though was that it allowed rangefinder cameras to be used in different ways to SLR's, and they could be combined in warzones where many photographers had one or more of each around their neck. Form and function came back to bite Leica with the M5 where the majority of users said 'too big, not interested'.
I think the answer regarding SLR lenses being larger lies between form and function (the average balance and weight on the camera body), the aperture stop down mechanism, and price. Common sense says all three elements need to be worked in harmony. When Nikon made the F the last thing they wanted was a camera that was fiddly to use, overly complex, or too expensive. On the other hand when the Leica rangefinders were invented there wasn't much extra technology around to load onto cameras so there was no reason to make them bigger. The general smaller size was set in stone by never needing the stop down mechanism and there wasn't a mirror to avoid. The hidden advantage over the years though was that it allowed rangefinder cameras to be used in different ways to SLR's, and they could be combined in warzones where many photographers had one or more of each around their neck. Form and function came back to bite Leica with the M5 where the majority of users said 'too big, not interested'.