0 votes

I'm trying to implement a Photoshop style magic wand that will create image masks of islands of colors given a cursor position. My current implementation runs out of memory on the stack. The scene is very bare-bones, just a Sprite, ColorRect, and a Label. I'm using an image pulled out of the viewport, (via get_texture().get_data()) and I made sure to lock and pass the Image data around instead of creating it again for each recursion.
Stack maxes at 1024. The crash (right now) occurs inside my Util singleton, but I'm guessing it shouldn't be near the limit at all if I were to write this properly -- I just don't know how to do that yet.

I figured I should put my code in a Pastebin since it's fairly long:
https://pastebin.com/awAVGPA7
Any help or guidance is appreciated.

(Util.gd: https://pastebin.com/MAXMYFhV )

Edit: the caller for this procedure is as follows:
given input trigger:
var mask = magic_wand(get_viewport().get_mouse_position(), OCEAN)
save_mask(mask)
OCEAN aliases to a const var at the top of the file but it gets overwritten anyway since it was a mistake to pass the desired color_to_match instead of setting it in magic_wand() via getting the color of the clicked pixel.

Edit2: Would it be more effective to do this via shaders (and, I'm guessing, offload the task to the GPU)? Would that be possible?

in Engine by (674 points)
edited by

I don't know gdscript well enough to review your code. From a general software development perspective, I would suggest you create a way to call your function with test cases so you can narrow down where the error is. Is the approach valid for a 10x10 matrix? 100x100? Color, or just black and white?

I suspect your issue will boil down to not having a tail call optimization with your current approach. There's a description of what that is at this link: https://wiki.haskell.org/Tail_recursion

Thanks for the advice and the resource.

Please log in or register to answer this question.

Welcome to Godot Engine Q&A, where you can ask questions and receive answers from other members of the community.

Please make sure to read How to use this Q&A? before posting your first questions.
Social login is currently unavailable. If you've previously logged in with a Facebook or GitHub account, use the I forgot my password link in the login box to set a password for your account. If you still can't access your account, send an email to webmaster@godotengine.org with your username.