About multi-threading and usage of mutex

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By Gary_CHAN

I have read some tutorials and try to use multi-threading to create some TextureRect for my game. However, I am not sure if I understand it correctly. I write the following code to achieve it.

The code seems to be working but the progress bar is not moving. All the TextureRect will show up at the same time.

I try a while ago with similar code where the “old_image” is set in create_new_image() and the progress bar did move and the TextureRect come up 4 after 4. (So I guess the multi-thread is working?) However, I cannot reproduce it again.

I believe I am messing up the usage of mutex but I don’t know how I get wrong.

Main.gd:

var threads = []
var test_scene = preload("res://test.tscn") # it is a TextureRect
var drawn = []   setget on_drawn_changed
var draw_called_time = 0
var total_num = 100 # total number of test needed

func create_threads():
    for index in range(4):
        threads.append(Thread.new())

func _ready():
    create_threads()
    old_image = load("res://testing.png")
    for index in range(4):
        draw_next_one()

func draw_next_one():
    if {no need to draw more}:
        return
    test_instance = test_scene.instance()
    for thread in threads:
	if !thread.is_active():
		test_instance.thread = thread
		break
    test_instance.old_image = old_image
    self.add_child(test_instance)
    draw_called_time += 1

func on_drawn_changed(value):
    update_progress()
    if draw_called_time <= total_num:
        draw_next_one()
    drawn.append(value)

func update_progress():
    var ratio = float(drawn.size()) / total_num
    var progress = int((ratio * 100) / 10) * 10
    get_node("ProgressBar").value = progress
    if drawn_puzzle.size() == total_num:
        # i will start the game

test.gd

var old_image
var end_x = 100
var end_y = 100
var mutex = Mutex.new()
var thread
func _ready():
    thread.start(self, "create_new_image", [start_point, size])

func create_new_image(argument):
    mutex.lock()
    var new_image = old_image.get_rect(Rect2(argument[0], argument[1]))
    mutex.unlock()
    new_image.lock()
    mutex.lock()
    for x_coordinate in range(end_x):
	for y_coordinate in range(end_y):
		new_image.set_pixel(x_coordinate, y_coordinate, Color(0,0,0,0))
    mutex.unlock()
    new_image.unlock()
    call_deferred("create_new_texture")
    return new_image

func create_new_texture():
    var new_image = thread.wait_to_finish()
    var new_texture = ImageTexture.new()
    new_texture.create_from_image(new_image) 
    self.texture = new_texture
    get_node("/root/Main").drawn = self    # to trigger setget

First thing you should know about threading, just in case:
Do NOT access the scene tree. The scene tree is not thread-safe, any attempt to access it from a thread will cause undefined behavior.

I’m not sure what’s causing your problem, but I had a read at your code.
In your first script Main.gd:

func _ready():
    create_threads()
    old_image = load("res://testing.png")

Where is old_image defined? I don’t see any var in that function and it’s not a member var either.

for index in range(4):
    threads.append(Thread.new())

Wow, 4 threads to load a puzzle game, sounds like a lot of processing power.

func draw_next_one():
    if {no need to draw more}:

That’s clearly pseudocode so I’ll ignore that, but it hides potential issues we could help you with.

if !thread.is_active():
    test_instance.thread = thread

I’m not sure about this. Your thread could, or could not have started. Maybe it can start between the if and the assignment.

test_instance.old_image = old_image

This is where mutexes are to be used, it’s missing here. If a variable can be accessed from two threads, you must lock a mutex before touching it, and unlocking the mutex once you’re done with it. Otherwise the variable could mutate while you are in the middle of using it.

In test.gd:

What is test actually? Is it just the thread logic? If it is, you probably don’t need a node for that, I see you add it to the tree but it doesn’t seem to be of any use once it finishes.

func _ready():
    thread.start(self, "create_new_image", [start_point, size])

Oh, so you are actually exploiting the fact it’s a node to use _ready to actually start the thread? Sounds a bit convoluted to understand but fine.

for x_coordinate in range(end_x):
for y_coordinate in range(end_y):
    new_image.set_pixel(x_coordinate, y_coordinate, Color(0,0,0,0))

You can use fill(color), if that’s the slow part then maybe you don’t need threads.

get_node("/root/Main").drawn = self    # to trigger setget

Why use a property? A function would have been clearer. Or better, use a signal, so you don’t need to access the root node by path (which is not a good practice on the long run).

Overall, I have difficulty to understand what your code is doing and in which order things happen. Maybe you are seeing the progress bar fill all at once because all your threads complete at the same time? If you want to load them in sequence then a single thread loading them one by one could just work.

Zylann | 2019-08-13 12:57

I tried to simplify the code so I added some pseudo-code but instead made too many mistakes and jeopardised the case. Sorry
I also try to make a simple test version but seems cannot reproduce the situation. So, I think better to put the full code here.
MovingPuzzle(multi-thread)1.rar - Google Drive

The ChoosePic.tscn is the welcoming page and by change number of puzzles, it determines the number puzzles to be drawn.Then click the pictures below to start the game.
The fruit picture is 256256 for fast testing and the cat one is 20482048 for heavier situation.
The game will change to the Main.tscn and start loading the puzzles.

Most of my code is in Main.gd and PuzzlePiece.gd
In Main.tscn(with Main.gd), run_num is the number of threads. If Show_loading is true, then the loading screen will be shown(if not lagged)
Depending on run_num, the Main.tscn will call draw_next_puzzle() that many times.

The PuzzlePiece.tscn(with PuzzlePiece.gd) is a TextureRect with Area2D and CollisionShape2D
The code will check if the puzzle is created before. if so, it will load from user://. Otherwise, it will start the threads to call create_new_image() and draw the puzzle.
When create_new_image() finishes, it passes the image to create_new_texture() and when it finishes, I suppose I can see the puzzle on screen.
Then PuzzlePiece sends a signal to the Main.tscn and call puzzle_drawn() on Main.tscn. puzzle_drawn() will determine that if not all puzzle is drawn, call draw_next_puzzle again

The problem is if I use one threads, the puzzle will show up one by one and if loading screen is set to show, the progress bar will be moving. (I didn’t try using one thread before seeing your comment because 4 threads used to be working and puzzles come up 4 by 4. but now one thread is working. It just significantly slower than 4 threads)

However, if I use two or more threads, the screen will freeze and all puzzles will show up at the same time (but it also loads much faster).

Also, since I am still figuring out how to delete saved file, every time the programme run the same picture, you have to manually delete all the png file in user://Save before running again. Sorry for the inconvenience.

Gary_CHAN | 2019-08-14 17:54

There is sometimes showing error _poll_events: Condition ’ err != OK ’ is true. Continuing
I googled it but didn’t find a proper solution. But it seems to be no affecting the game and it never shows up when I exported the game.

Thank you for helping me and watching my long question

Gary_CHAN | 2019-08-14 18:03

:bust_in_silhouette: Reply From: Gary_CHAN

I think I got a “solution” by adding yield(get_tree().create_timer(0.00001), “timeout”) before self.emit_signal(“finish_drawing”, self) in create_new_texture()
The code would be something like this:

func :
    var new_image = thread.wait_to_finish()
    var new_texture = ImageTexture.new()
    new_texture.create_from_image(new_image) 
    self.texture = new_texture
    yield(get_tree().create_timer(0.00001), "timeout")
    self.emit_signal("finish_drawing")

I don’t know why but it solved the problem. Maybe it need a bit time to let the viewport(?) draw the texture out