Make sure your audio file is not set to be looped.
Select your audio file in the FileSystem dock. Then click on the Import tab (to the right-side of the Scene tab by default). Scroll down and make sure the “Loop” checkbox is not checked. Don’t forget to click on “Reimport” if you changed anything.