Consider it the following way: a physics object belongs to some layers, and collides with other objects specified by its collision mask.
A collides with
A.layers has any bits in common with
B.mask, or if
B.layers has bits in common with
A.mask (In the code it is done with a simple bitwise-and operation).
But, raycasts aren't actually physics objects in the same sense. They are detectors, they don't collide with stuff (except
RayCastShapes, but that's another story). This means that they have only a mask, and no layers, so a body
A would be detected by a ray
R only if
A.layers has bits in common with
R.mask (we cannot reverse that statement, since the raycast doesn't have layers).
So, the problem is that raycasts (by node or by code) match layer masks, not collision masks. This means that if you do
set_layer_mask(2) on your object, it would be correctly matched by the raycast.
Finally, if you do think (and need it for a game of yours) that matching objects from raycasts has to be done by both collision layers and masks, feel free to open an issue on the godot repo on github.