Yeah, you should only have one. What I generally do is have a separate game object with an AudioListener that will automatically position itself at Camera.main.transform.position in LateUpdate. I also mark this game object as DontDestroyOnLoad, and have it travel from scene to scene.
