Assuming the window glass is flat (not decorative like some are) then it should be fine. I'm currently using the lid off a fish tank Go to a glass shop and get a bit of 4-5mm thick glass with beveled edges (for safety). I recomend getting it slightly longer than your baseboard to make it easy to lift off the negative(s) and you'll keep your finger prints on the edges away from where you place it on the neg.. Should only be a few dollars.
A word of warning. All glass is not the same. I went into the local Aaron Bros. frame and art supply store and inquired about glass. They had several nice varieties, including some very heavy glass that I thought would hold negative in contact (It's all about contact) with the paper. I happened to ask at the last minute if there was any anti-UV in the glass. They said, "Yes, all our glass has UV blocking" This is not good for alternative printing. UV makes it work. Choose your printing glass with care.
Thanks for mentioning that. I would never have thought of asking if the glass blocked UV. While I am not currently entertaining a notion of experimenting with alternative processes, I may wish to do so in the future. I would not want to buy twice what I only need to buy once.
Most glass will block the mid and far band of UV only and have minimal effect in comparison on the near band UV. The near band UV is what is primarily of effect in exposing photosensitive materials. Typical "window" glass has very little effect on UV transmission as it applies to photography. If one encounters the actual UV blocking glass then you will know it by the price that you are asked to pay.