2D outline shader issue

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By nonomiyo

Hi,

I’m using the shader from the 2D demo “sprite_shaders” (Godot 3.1 alpha 2)
but as I enlarge the outline, it’s not smooth and it overlaps on the drawing :

> SCREENSHOT HERE <

Is there a way to make it look better ? :confused:
Thanks !

here is the code :

shader_type canvas_item;
uniform float outline_width = 2.0;
uniform vec4 outline_color: hint_color;

void fragment(){
	vec4 col = texture(TEXTURE, UV);
	vec2 ps = TEXTURE_PIXEL_SIZE;
	float a;
	float maxa = col.a;
	float mina = col.a;

	a = texture(TEXTURE, UV + vec2(0, -outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(0, outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(-outline_width,0)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(outline_width, 0)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	COLOR = mix(col, outline_color, maxa-mina);
	}
:bust_in_silhouette: Reply From: Xrayez

The demo uses 4-way kernel method for drawing outlines, in order to address smoothness problem it’s better to use 8-way kernel. To eliminate overlap, you only need to fill pixels that are transparent:

shader_type canvas_item;
uniform float outline_width = 2.0;
uniform vec4 outline_color: hint_color;

void fragment(){
	vec4 col = texture(TEXTURE, UV);
	vec2 ps = TEXTURE_PIXEL_SIZE;
	float a;
	float maxa = col.a;
	float mina = col.a;
	
	// Use 8-way kernel for smoothness
	//-----------------//
	// 1    X   X   X  //
	//        \ | /    //
	// 2    X - O - X  //
	//        / | \    //
	// 3    X   X   X  //
	//-----------------//

	// First row
	a = texture(TEXTURE, UV + vec2(-outline_width, -outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);

	a = texture(TEXTURE, UV + vec2(0, -outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(+outline_width, -outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);

	// Second row
	a = texture(TEXTURE, UV + vec2(-outline_width, 0)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(+outline_width, 0)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);

	// Third row
	a = texture(TEXTURE, UV + vec2(-outline_width, +outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);

	a = texture(TEXTURE, UV + vec2(0, +outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);
	
	a = texture(TEXTURE, UV + vec2(+outline_width, +outline_width)*ps).a;
	maxa = max(a, maxa); 
	mina = min(a, mina);

	// Fill transparent pixels only, don't overlap texture
	if(col.a == 0.0) {
		COLOR = mix(col, outline_color, maxa-mina);
	}
	else {
		COLOR = mix(col, outline_color, 0);
	}
	
	}

I’m new to shaders as well so I think this can be optimized.

I’m not a hundred percent sure this would be an optimization, but it definitely shortens down the code.

shader_type canvas_item;
uniform float outline_width = 2.0;
uniform vec4 outline_color: hint_color;

void fragment(){
    vec4 col = texture(TEXTURE, UV);
    vec2 ps = TEXTURE_PIXEL_SIZE * outline_width; // multiply only once instead of eight times.
    float a;
    float maxa = col.a;
    float mina = col.a;

    // Use 8-way kernel for smoothness
    //------------------//
    //    X   X   X     //
    //      \ | /       //
    //    X - O - X     //
    //      / | \       //
    //    X   X   X     //
    //------------------//

    for(float y = -1.0; y <= 1.0; y++) {
        for(float x = -1.0; x <= 1.0; x++) {
            if(vec2(x,y) == vec2(0.0)) {
                continue; // ignore the center of kernel
            }
            
            a = texture(TEXTURE, UV + vec2(x,y)*ps).a;
            maxa = max(a, maxa); 
            mina = min(a, mina);
        }
    }

    // Fill transparent pixels only, don't overlap texture
    if(col.a == 0.0) {
        COLOR = mix(col, outline_color, maxa-mina);
    } else {
        // Note on old code: if the last mix value is always 0, why even use it?
        COLOR = col;
    }
}

SIsilicon | 2018-11-23 11:39

Yeah this works, and much more concise, thanks!

Xrayez | 2018-11-23 12:14

Wow, that’s a great improvement, thanks so much SIsilicon and Xrayez ! :smiley:

> HERE IS HOW IT LOOKS NOW <

nonomiyo | 2018-11-23 13:12

This gives the outline a slightly rectangular look, where diagonal lines produce a thicker outline than horizontal/vertical lines.

I produced better results for my game by multiplying the diagonal vectors by 0.7071 (one over the square root of two)

Poobslag | 2020-04-01 14:11

:bust_in_silhouette: Reply From: nonomiyo

I managed to get a super smooth outline shader, by tweaking SIsilicon and Xrayez’s contribution :

shader_type canvas_item;
uniform float outline_width = 2.0;
uniform vec4 outline_color: hint_color;

void fragment(){
    vec4 col = texture(TEXTURE, UV);
    vec2 ps = TEXTURE_PIXEL_SIZE * outline_width;
    float a;
    float maxa = col.a;
    float mina = col.a;


    for(float x = -1.0; x <= 1.0; x+=0.05) {
		float y = 1.0 - (x*x);
        if(vec2(x,y) == vec2(0.0)) {
            continue; // ignore the center of kernel
        }

        a = texture(TEXTURE, UV + vec2(x,y)*ps).a;
        maxa = max(a, maxa); 
        mina = min(a, mina);
    }
	
	for(float x = -1.0; x <= 1.0; x+=0.05) {
		float y = -1.0 + (x*x);
        if(vec2(x,y) == vec2(0.0)) {
            continue; // ignore the center of kernel
        }

        a = texture(TEXTURE, UV + vec2(x,y)*ps).a;
        maxa = max(a, maxa); 
        mina = min(a, mina);
    }


    // Fill transparent pixels only, don't overlap texture
    if(col.a < 0.5) {
        COLOR = mix(col, outline_color, maxa-mina);
    } else {
        COLOR = col;
    }
}

Not sure how efficient this is but it’s better looking !

Thanks for that, it was very helpful. I have a question. What needs to be done to give the border lines and make the inside completely transparent. So the borders are visible, but I want the sprite to be completely alpha 0.

ayhanasker | 2020-09-14 03:06

So the borders are visible, but I want the sprite to be completely alpha 0.

I’m not super advanced at shaders, but wouldn’t changing:

COLOR = col;

to

col.a = 0.0;
COLOR = col;

do it just fine? Alternatively you could just do:

COLOR = vec4(0.0, 0.0, 0.0, 0.0);

And both should work. Dunno which is faster though…

Haaldor | 2021-01-12 23:24