I currently add 9 g of gouache to 300 ml of glop
The problem with this is that there's just no telling how much pigment that is, exactly. In my limited experience with gouache, there's not a lot of actual pigment in it, but it'll depend greatly on the brand, product line and especially the color (due to the different tinting strengths of pigments).
But as a general rule, a 3% w/v 'pigment' load with the pigment being some kind of paint really isn't very low. In fact, if you'd do this with e.g. Winsor & Newton watercolors (to name a random, but well-known example), it would constitute a pretty high pigment load with high contrast and limited relief. The fact that you report a lot of relief suggests the pigment load of the paint you're using is very, very low indeed. to give you an example; I'm working with dry pigments that I disperse myself, and a typical pigment load for a Kremer Pigmente Pbk7 black tissue will be around 0.15% w/v (that's not a typo: far less than one percent!) at a gelatin load of 8% w/v. This is comparable to roughly a 1% v/v India ink 'pigment' load in my experience.
Note also that pigment load in itself means very little without information on gelatin load. Keep in mind your final transfer will be just pigment trapped in gelatin - al the water is gone from the finished print. So things like contrast and relief depend on
pigment to gelatin ratio, not on pigment load of the watery glop!
Anyway, if you want to decrease relief, just increase pigment. Start by adding 25% or 50% and see where that gets you; adjust from there. There's no 'golden rule' for how much pigment one 'should' use. As said, it depends hugely on what kind of pigment in what form is used, and what your printing style is.
If you're doing multilayer prints, thinner layers tend to work best - not so much because of the paper curl, but mostly because of the problems you'll run into with air gaps/bubbles between consecutive layers.
Are you transferring each layer directly to the paper (single transfers on top of each other), or are you transferring the layers on top of each other onto a temporary support (e.g. a polyester sheet of some sort)? In case of the former, you'll get into trouble with registration issues, among others, at least going by my own experience. Paper isn't dimensionally stable and you'll experience increasing problems getting consecutive layers to align with previous ones. But as said, it may be challenging or even impossible to do a successful multi-layer dual transfer process with very thick layers.
So for all intents and purposes, I'd suggest going for thinner, more high contrast layers, and perhaps work around the inevitable highlight problems using multiple layers - which may be the reason for your multilayer approach in the first place? Are you doing a color process or a monochrome (or perhaps toned monochrome/polychrome) process?