Issues

Issue #559 invalid

Problems created by not exposing gl_FragColor

Anonymous created an issue

Because pixeleffect will write:

gl_FragColor = effect(gl_Color, tex0, gl_TexCoord[0].xy, gl_FragCoord.xy);

Theres no way to create a pass-through shader that wont set RGBA. This limits the user, by not allowing things such as:

void main ()
{
    gl_FragColor.a = 0.0;
}

To punch a hole in a already existing background.

One example where this is useful is: http://devmaster.net/posts/3079/shader-effects-refraction

This forces the addition of extra passes with unnecessary switching of Canvas/FBO objects in the previous example

Comments (4)

  1. Alex Szpakowski

    I'm not really sure what the issue is. The value of gl_FragColor is undefined (meant to be write-only) until you write to it for the first time in that particular shader, and if you don't write to it at all (and don't use the discard keyword), the output fragment color is also undefined.

    Your example can be achieved by doing this:

    vec4 effect(vec4 vcolor, Image texture, vec2 texcoord, vec2 pixcoord)
    {
        return vec4(0.0);
    }
    

    However, you're setting gl_FragColor to something, so it does set RGBA (depending on the blend mode). You may want the discard keyword:

    vec4 effect(vec4 vcolor, Image texture, vec2 texcoord, vec2 pixcoord)
    {
        discard;
        return vec4(0.0);
    }
    

    You mentioned on IRC that you can't send sampler2d uniforms, but this is not true. Take this example:

    extern Image myothertex;
    vec4 effect(vec4 vcolor, Image texture, vec2 texcoord, vec2 pixcoord)
    {
          vec4 texcolor = mix(Texel(texture, texcoord), Texel(myothertex, texcoord), texcoord.x);
          return texcolor;
    }
    

    And Lua-side:

    myshader:send("myothertex", image)
    

    EDIT: What's actually going on in the "pixel tag" shader in your link, is that the WebGL Javascript code disables all color writes except the alpha channel before using the shader (via glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE)), and re-enables them after it's finished.

  2. xenthral

    Hey there, thanks for the reply but I believe it doesnt address the issue at hand, unless I'm mistaken (very possible its the case I'm rusty on my GLSL and a LOVE neophyte!)

    The value of gl_FragColor is undefined (meant to be write-only)

    I am in no way attempting to read it, I just want to write to alpha value 0.0 and output just it.

    Your example can be achieved by doing this: (...)

    No, that example would write the equivalent of gl_FragColor(0, 0, 0, 0), producing the incorrect result (at least on my hardware). Discard also produced incorrect result.

    You mentioned on IRC that you can't send sampler2d uniforms, but this is not true.

    Yes I mentioned it after I removed all of the code in love.graphics._effectCodeToGLSL, meaning the compiled shader did not have the #define Image sampler2D, I have now fixed this problem, at any rate its irrelevant to my original query.

    EDIT: What's actually going on in the "pixel tag" shader in your link, is that the WebGL Javascript code disables all color writes except the alpha channel before using the shader (via glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE)), and re-enables them after it's finished.

    Yes! The goal in the end is just to set the alpha bit to 0.0, but I looked and couldnt find access to glColorMask from the LOVE side of things, is there?

    Anyways at the moment I have a good enough workaround Fire. Thank you for the reply.

  3. Alex Szpakowski

    No, that example would write the equivalent of gl_FragColor(0, 0, 0, 0), producing the incorrect result (at least on my hardware). Discard also produced incorrect result.

    The code in the pixel tag shader will produce a color of (0.5, 1, 0, 0) or (1, 1, 1, 0) or (0, 0, 1, 0) or any other combination of r,g,b values, because the r,g,b components of gl_FragColor are not written to so they are undefined. The only guarantee is that the alpha component will be 0, because it's set as such in the code.

    The final result color depends on the blending operations. An alpha of 0 (as set by gl_FragColor) could be added to the current pixel's alpha value, or multiplied with the current pixel's rgb values, or it could replace the current pixel's existing alpha value, or a variety of other possibilities. It all depends on the blend mode.

    Depending on the desired outcome, you may simply be able to change the blend mode without needing to use color masking at all.

    Personally I think being able to set the color mask could potentially be a useful API addition, but it's also true that the current combination of blend modes and PixelEffect shaders covers most possibilities already, and some of the ones which aren't possible right now with LÖVE just need a new blend mode.

  4. Log in to comment