Yes 100%, the quality of the headphones you use will effect your ability to accurately find the position of the sound so ultimately try to use good quality headphones. But other than that practice makes perfect, it'll become easier day by day.

Make sure your headphones are not summing the feed to mono. (mine has a setting for this)
Also check your computers audio settings.

Thanks a lot for the comments! I will definitely check the settings and practice my ass off!

Here's my trick with headphones. It's weird but works surprisingly well for me: Figure out which of my teeth the sound is coming from. I find it's about 1 tooth per .1 of panning. 0 is between my front two teeth, .3 is under my canines, and so on around to my molars. A big challenge is to figure out where center is (I started about .15 off all my answers to the left) but it comes with practice. Good luck!