If I want to make a game that I want to be playable over LAN without wanting either users to have to set up a dedicated server, would Godot be able to do it?
Yes. You just need to have a "listen server" mode, which basically creates a server in the client window.
If so, what are the relevant links I should be looking at to know more about it?
This is less commonly done than dedicated servers, so you won't have as much documentation.
That said, Godot's networking is expected to be improved for 2.2 and so will the documentation about it as well, probably.