It might be a custom frame, but mostly it's an exercise in tricky mat cutting. Frames are available in many standard sizes, and one finds squares occasionally. If you go with pairs of sections you can easily enough put together a square or odd rectangular frame. A number of frame suppliers allow ordering just about anything within reason at a pretty much standard price.
The mat and mounting would be the more difficult question. Cutting a mat like that would be a notch or two in tedium beyond cutting a standard single window, but could be done with some patience (and a mat cutter!) Some mat cutters are fairly limited on the width of the borders they can cut, which could make getting at the inner cuts difficult. I'm sure many frame shops have the ability to do that, but 'frame shops' and 'inexpensive' do not correlate!
I can't say I've ever seen that combination as a "product" and I imagine not many photographers would want to do what that guy is doing; e.g, selling the original image.