In my project I need to mask a sprite hierarchy in order to clip it to a region of a certain shape so that whatever is outside of that shape won't be visible.
To be more precise, I'm trying to make an eye of a character with moving parts that the player could choose the color of the iris (I'm modulating a grayscale image of the iris with a color) and then the iris would be able to move inside the eyehole and remain clipped to its shape instead of sticking out.
How can I use an image of the eyehole shape to clip the sprite hierarchy of the eyeball?
I tried the approach with light map described here, and it seems to work fine:
but only as long as there's only one eye on the screen :P As soon as I introduce another one and make them overlap, their light maps start to interfere in a weird way as shown here:
as if the masks were XORing with each other or something (the invisible region of one light mask makes the other mask's invisible region visible again, and only where they overlap, they mask each other correctly again). Which obviously is not good, because I need to have more than one eye on the screen (there may be more than one character visible at any time).
Is it some kind of a bug? Or am I doing something wrong?
How to make it work right?
Or maybe there is some better way to do it than with light masks? E.g. some texture with alpha channel or something? (Except that for an entire hierarchy of sprites instead of just a single one.)
Edit: I tried to fix it by adding some transparent pixels around the masking shape and it helped a little bit: at least now the eyes don't interfere that often because their masks overlap most of the time.
But the problem is far from solved: When I added the eye as a part of another hierarchy of sprites (a head), the masks now interfere with the head too! :o
Even if the head (green) is clearly in front of the eye on the left, and normally occludes it, the area of the right eye that was previously hidden by the mask (the white rectangle for the white background of the eye) starts to be visible now through the area of the left eye's mask even if it's behind the head and shouldn't be even visible!
It seems that the light masks don't care about the Z order of the scene at all and if they overlap with some pixels, they make them visible no matter what :P
Here's the project file if you want to play with it and see the derps in action: