How to create a shader for casting shadows on an isometric tilemap ?

Question

Hi, I'm trying to create a shader that will cast the shadows of isometric objects placed on a tilemap :

To achieve this, this article here https://mattgreer.dev/blog/dynamic-lighting-and-shadows/ suggests using a heightmap for each sprite.
So is there a way I give each tile in my tilemap a heightmap texture and use it in a shader ?

As a workaround, since my textures are rather small, I was thinking I could use transparency on the tile normalmap texture as heightmap data. But I still have no idea on how to access this normalmap from a shader to then draw a shadow outside of that specific tile.

I'm rather new to shader programming so any help is appreciated !

Answer 1

Giving a tile a normal map is easy - just look for the property under "selected tile".

I couldn't find a way to access the normal map from inside a shader though. I'm not sure if you could write a shader like this with Godot right now since you can't draw outside the UV coordinates of a tile. Maybe you can use a Light2D instead?

(I tried using a Light2D but I couldn't get it to work so I probably did something wrong)

Edit:
I finally got something working - what you need to keep in mind is that Godot's y-coordinates are flipped, so you will need to vertically flip your normal maps as well. For example, here is my cube.

And here is what it looks like with the lighting and occlusion shape added.

Comments

Thanks a lot for your answer !
I already know how to use Godot's light2D but like we can see in your example, the shadow of your cube just continues indefinitly until it leaves the light2D texture so it's not exactly what I'm trying to do. I added a response to my own post where I describe how I almost implemented what I was talking about if you're interested (though I still have two major issues).

Answer 2

Hi, in the meantime I actually came up with a solution that almost works properly :

First I have a simple shader for my sprite where I give it a heighmap. Depending on the color of each pixel in the heightmap, I use the HDR value of that pixel to pass the height information onto another full-screen shader.

I'm sorry if my code is not super clean, I pretty much got my implementation to work by trial and error.

shader_type canvas_item;
uniform sampler2D heightmap;

void fragment(){
    vec4 heightmap_tex = texture(heightmap, UV));
    if(heightmap_tex.a > 0.9){
        // I'am adding 1.0 * h to to that pixel's color, where h
        // is the height of that pixel on the heightmap
        COLOR = texture(TEXTURE, UV) + vec4(1.0, 1.0, 1.0, 0.0) * (heightmap_tex.r * 255.0 + 1.0);
    } else {
        COLOR = texture(TEXTURE, UV);
    }
}

Then my screen shader knows that if a pixels'color is greater than 1.0, it means it has some height.

shader_type canvas_item;
uniform float xyangle;
uniform float zangle;

void fragment(){
    bool isinshade = false;
    float dist;
    float height;
    float lookupheight;
    float traceheight;
    vec4 col;

    //first I calculate the height of the current pixel
    col = texture(SCREEN_TEXTURE, SCREEN_UV);
    height = col.r;
    if(height < 1.0){height = 0.0;}
    height = round(height);

    //then I look at some pixels in a given direction
    for(int i = 0; i < 100; i ++){
        //distance of the pixel I'm looking at
        dist = SCREEN_PIXEL_SIZE.x * float(i)*2000.0;
        //UV of that pixel on screen
        float lookupx =SCREEN_UV.x + (0.001) * (-4.0) * dist ;
        float lookupy =SCREEN_UV.y + (0.001) * 0.0 * dist ;
        //Height of that pixel
        lookupheight = texture(SCREEN_TEXTURE, vec2(lookupx,  lookupy)).r;

        if(lookupheight < 1.0){lookupheight = 0.0;}
        lookupheight = round(lookupheight);

        //If that pixel's height is greater than the height of my current pixel
        if(lookupheight > height){
            //I calculate the height that that pixel should be if it was to draw a shadow
            //on my current pixel
            traceheight = dist * tan(zangle) + height;
            if(traceheight < 1.0){traceheight = 0.0;}
            traceheight = round(traceheight);

            //If the height of that pixel and the height it needs to be are close enough
            //then my current pixel needs to be shaded
            if(abs(traceheight-lookupheight) < 4.0){
                isinshade = true;
            }
        }
    }

    //Now I'm juste resetting the pixel's color to it's real value
    if(col.r > 1.0 || col.g > 1.0 || col.b > 1.0){
            col.r = mod(col.r, 1.0);
            col.g = mod(col.g, 1.0);
            col.b = mod(col.b, 1.0);
            COLOR = col;
            isinshade = false;
        }

    if(isinshade){
        COLOR = col - vec4(0.2, 0.15, 0.15, 0.0);
    } else {
        COLOR = col;
    }
}

Ok so as I said, using those two shaders I get some decent results but I still have two major problems:

First you may have notice that I'm not using xyangle in my screen shader. It's because whenever I use :
float lookupx =SCREENUV.x + (0.001) * cos(xyangle) * dist ;
float lookupy =SCREENUV.y + (0.001) * sin(xyangle) * dist ;
it causes ugly artifacts:

I'm pretty sure it has something to do with an imprecision caused by the floating values of the sin() and cos() but I'm not exactly sure.
So does anyone know how I could fix those ?

Second since I'm using SCREEN_TEXTURE to read the height value of each pixel, I can't access the height of a pixel off screen. So whenever a sprite is partly outside the screen, its shadow gets partly cutted out. So is there a way I could access pixels slightly off screen in my second shader ?

Thanks a lot for reading me !

Comments

Unfortunately I don't think there's a way to access pixels off screen with a shader

---

Hmm, would it be possible to just take the original sprite's sprite, rotate it and scale it, and then change the color?

---

Yes that could work for simple sprites like the one I showed, but for bigger, more complex ones it really doesn't look great. I guess could just use hand drawn shadows on an other tilemap for example but then I would not have as much control on it.

Right now I'm trying to see if I can use a viewport to render a bigger area and then only display a portion on screen. This way, maybe I could access pixels outside the screen by using that viewport texture. But since all that is pretty far from what I initially asked, I'm probably going to open a new question for it.

Thanks for your help !