I still didn’t fix it, but testing with godot 3.1 instead of 3.0.6 made a perceivable difference.
Many reloads are needed in order to feel the drop + the drop is about 40 fps instead of 30, so it might be a bug in my code, but the engine version does matter too.
PS: godot 3.1 works like a charm on my old laptop were as 3.0.6 is too damn heavy !
(CPU reaching 80 degrees)
Also, made this other project change in 3.2 and gained almost 15 FPS:
Display -> Window -> Stretch -> Mode -> 2D (instead of viewport)