I solved this issue by using blender's NLA (nonlinear animation editor).
Reference from where I understood it better: https://www.youtube.com/watch?v=mfdufhaiKtI
Basically what you need is to create a new window with NLA (Nonlinear animation) panel, on your action editor, click one of your animations so that you can see it in action, and then back in the NLA panel, click on the button to the right of the animation's name (the button is the "push down action") and rename the action you pushed down to your animation name. Now when you export the gltf or glb, under animation make sure the "group by NLA track" is selected and godot should import all your animations correctly, or at least this is what worked to me, especially when the first frame of the animation did not match the resting pose of the model.
I noticed that the other answer said something about NLA but I didn't understand it well, so in case anyone gets confused as I did, remember to try this out!