First I want to state the use case: Consider that three semi-related modules depend on stencils, the first module sets its stencil, draws the second modules, which sets it's own stencil, then draws the third module, which sets its own stencil, then the third draws something, the second draws something else and the first draws other stuff
One option would be to say, well the first module sets the stencil to 1 the second to 2 and the third to 3 so the StencilTest can check if the value is greater than the value they set.
Now if I change the order of the modules to first -> third -> second then the when the second draws then all the area of the third one is actually greater than the second, and after the second finishes drawing, the third one can't draw in the area where the second was used.
Of course you could pass a number through the API and avoid this kind of problems, but if you are using this modules multiple times then that can be cumbersome.
I propose other method, and it has its benefits and disadvantages: So, if we could check if a specific bit is set to 1 or 0 in the module we could have up to 8 modules using a different bit. How would this look like:
love.graphics.stencil(myStencilFunction, "setbit", 4) --Sets the bit 4 to 1 love.graphics.stencil(myStencilFunction, "unsetbit", 3) --Sets the bit 3 to 0 love.graphics.stencil(myStencilFunction, "toggle", 2) --Toggles bit 2 -- Only allow rendering on pixels with a 1 at bit 4 love.graphics.setStencilTest("bit", 4) --Only allow rendering on pixels with a 0 at bit 3 love.graphics.setStencilTest("notbit", 3)
Possible disadvantages are: Implementation may be complex
it doesn't fix all cases, for example if you wanted to use the modules like so first -> second -> second then the second use of the second module would override the stencil anyway.
Also there is no easy way to combine stencil tests, so if you used modules like first -> second then the second could overflow the first one since it only checks for its own bit
This is just another tool and I wanted to open the discussion about it