Yep friction is 0, sorry didn't mention that. It is a stuttering, the movement will continue from 0. The other changed properties are:
Continuous CD = RayCast
Can Sleep = false
Though it doesn't feel quite right, I'm working around in a way something like pseudo code below:
var prev_lv = Vector2()
func _integrate_forces(state):
var curr_lv = state.get_linear_velocity()
if(curr_lv.length() < 1.0 and prev_lv.x >= X_THRESHOLD):
curr_lv.x = prev_lv.x
# ... custom gravity, friction, etc.
state.set_linear_velocity(curr_lv)
prev_lv = curr_lv