So many posts asking about Atheist music (I object to the term atheist but I digress). May I put forward Roy Harper. A good mate of Led Zeppelin (hats off to Harper) and Jimmy Page. Also of Pink Floyd, he did the vocals on "Have a cigar".

His own works are rife with great atheist songs such as "The same old rock" ie. St Peter. "the Game".. (you Christians destroyed my tribe Ill fight you till I die). And an album that contains several called "Once" (heres to the one and only once).
If (but if it means going down on my knees well wheres the spirit gone. wheres the love your all talking of if we can't stand man to man) and controversially "Black cloud of Islam" (Your the worst of Jehovah's blind witlessnesis) and (I bet Salman Rushdie was just about right).

Do check him out and enjoy.