[DX11] validate shader signatures (DEBUG mode only)

#333 Merged at fc3b24c
Repository
ogre
Branch
default
Author
  1. Christopher Christensen
Reviewers
Description

Currently with DX11 if the vertex-pixel shader input/output signatures don't match exactly, no warnings or errors are given, just garbage output. This patch checks the signatures and generates an exception with a message naming the offending shaders if they aren't compatible.
Because this is done in _render it is only done in OGRE_DEBUG_MODE to avoid impacting performance.

Comments (6)

  1. Amit Rojtblat

    Hello,

    I might be making a huge mistake, and if so please correct me if im doing sth wrong. but this commit produces error on dx11 domain to geometry shader. Stuff that used to work before produces this error. I dont know if this is directly related to ds->gs, but isnt it possible that the ds output has a different signature then the gs input? or for that matter vs->gs?

    basic code:

    struct HCO {
        float Edges[2]  :SV_TessFactor;
    };
    
    struct D2G {
        float4 pos  :SV_Position;
    };
    
    //tesselation variables
    [domain("isoline")]
    D2G main_ds(    HCO In,
                    OutputPatch<H2D, 2> outPatch,
                    float2 uv       :SV_DomainLocation,
                    uint PatchID    :SV_PrimitiveID,
                    uniform float4x4 wvpMat             )   {
    
        D2G Out;
    
    
        float3 position =   uv.x * outPatch[0].pos.xyz * In.Edges[0] *.25 +
                            uv.y * outPatch[1].pos.xyz * In.Edges[1] *.25;/* +
                            uvw.z * outPatch[2].pos;*/
    
        Out.pos = mul(wvpMat, float4(position,1));
    
        return Out;
    }
    
    [maxvertexcount(256)]
    void main_gs(   point D2G In[1],
                    uint pID    :SV_PrimitiveID,
                    inout TriangleStream<G2P> Out,
                    uniform float4x4 wvpInvMat,
                    uniform float4x4 wvpMat,
                    uniform float4x4 vMat,
                    uniform float4 viewSize,
                    uniform float radius,
                    uint instID :SV_GSInstanceID    ) { ...blablabla}
    

    All the best!

    Amit.

  2. Christopher Christensen author

    What does the error message say? My understanding is that the outputs from one shader stage need to match the inputs to the next stage. If the signatures do not match, there will be no warning or error from DX11, the next shader will just silently get garbage inputs.
    In this case, it looks like your domain shader is outputting just "SV_Position", while the geometry shader is expecting "SV_Position" plus "SV_PrimitiveID" and "SV_GSInstanceID". I haven't tested this code on anything other than vertex and fragment shaders, so its possible that special cases are needed to handle other kinds of shaders.

    1. Amit Rojtblat

      Thanks for the fast replay :)
      the error i get is from : "Shader " + progA->getName() + " produces not enough output parameters for shader " + progB->getName(), + "D3D11RenderSystem::validateShaderSignatures" ); from my ds to my gs.
      As far as I know, both SV_GSInstanceID and SV_PrimitiveID are only used as inputs. they cant be modified. more so SV_GSInstanceID can only be used by the gs.

      1. Christopher Christensen author

        So it looks like the current shader signature validation isn't robust enough to deal with some of these special SV semantics like SV_PrimitiveID that are passed directly to a stage in the middle of the pipeline. I'll see about fixing the validation to handle these cases. I'll make a pull request when I have something.

          1. Christopher Christensen author

            I discovered that a far better way to go is simply to turn on the debug layer, and it will handle the shader signature validation. I'm going to back out this change.