We will hunt down and depict the bug!

My nVidia GPU fails to divide 33,37,41,47,55,66,74,75,82 or 94 by itself

hexagonal fields gradient red and green, showing the bug as yellow lines Just when i finished debugging a generative shader, that renders a texture for proper addressing of a hexagonal neighbourhood (see previous post), i ran into that glitch for edges of 21, 24, 28, 31 and probably others.

I started media player classic, opened it's nice little pixel shader editor and reduced the code to the fllowing lines:

(Sorry my bLog has no highlighting for HLSL yet.)

sampler s0 : register(s0);
float4 p0 :  register(c0);

float4 main(float2 tex : TEXCOORD0) : COLOR {

    // Map x to numbers 0 to 100:
    float n = floor(tex.x * 100);
    
    // This expression should always result 1.0, but due to rounding errors
    // for n in {33,37,41,47,55,66,74,75,82,94} it's resulting below 1.0:
    float err = trunc(1.01 * n ) / n;
    
    // Let's visualize it
    float4 c = 1.0;
    // Original error in red:
    c.r = frac( err );    
    // adding this epsilon prevents the rounding error in green:
    c.g = frac( err + 0.000001);
    // just a little less will not prevent the error in blue:
    c.b = frac( err + 0.00000099);
    
    // So, red is the error and green prevents it while blue shows up again, right?
    // I get ten magenta stripes badly hidden in 90 black ones:
    return c;
}

Obviously, the line in charge is the division by itself. To prevent the compiler from optimizing, i left one multiplication that should in any case lead to the original integer after the truncation:

float err = trunc(1.01 * n ) / n;

And that is what my media player shows up (open some dummy video file to reproduce it):

Media Player Classic showing magenta bars. Menu [View] is open, showing entry for pixel shader editor.

Melchior blausand Monday 20 October 2014 at 11:42 pm | | nicht unlösbar
Used tags: , ,

No comments

Remember personal info?
Notify
Hide email
Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.