Artistic license again: I don't know that there are any songs that make me feel guilty. However, a song that I listened to a lot during a time when I was feeling rather guilty, that resonated with me a great deal, is Ani DiFranco's "Studying Stones." The visuals in the video below aren't great, but the sound quality is rather good, at least compared with a lot of live videos from YouTube. :)
Click to view