Clifford,
I have had good luck with Diafine and Tri-X 400 in a Holga (a 120N in my case). But it was no accident. For me, this combination was explicitly the "plan" I worked out when I first bought the camera; the idea being that it would provide the widest exposure latitude possible given the limited adjustability of the camera. It sounds like you were thinking of something similar. Overall I think it worked well and was consistent with the whole idea of keeping things as simple and casual as possible. On the other hand, I have to admit that getting to that point included going to the resort of measuring the actual shutter speed of the camera, and modifying the aperture of the camera to get two real settings that were exactly what I wanted (not exactly "casual") . All this to insure that exposures would be "in range" for the lighting conditions in which I wanted to use the camera.
I guess what I am saying is that Diafine should be able to work well for you. It may just be that you are on the edge of the usable exposure range of the system (i.e., the particular combination of film speed, shutter speed, and aperture you have).
I am not entirely sure what you mean by washed out though. Is this the prints or the negatives? Assuming you are referring to an effect indicative of overexposure, then I think the ND filter idea could be a good one. It also occurred to me that you could use contrast filters instead of (or in addition to) the ND filter to help knock down the exposure while simultaneously picking up the contrast-enhancing effects get you get with those.
So yeah, in general whatever it takes to get you into the right range for better exposure! As for reducing the development time, it probably won't help. My understanding is that Diafine is designed to develop to completion, and anything less than the 3 minute minimum will probably just make life difficult and unrepeatable (if it even has the effect you were hoping for at all - I am not sure).
Other ideas: Double check that you have mixed the A and B components correctly in the first place; and think about whether it is possible that you have contaminated them (e.g., gotten some B into A).
Jeff