Hard drives being the size of Texas these days, I pretty much copy every CD I buy into iTunes. That, the occasional virtual shopping trip to the iTunes Music Store and my subscription to emusic mean I'm approaching 8,000 songs in my iTunes library.
I use iTunes and my tough little iPod Nano to help me manage my musical cornucopia. I found that the Nano will hold about 56 hours of music. So in iTunes, I created a smart playlist 56 hours long that picks a random selection from my library but only songs I haven't heard in the last 30 days. (I could have set this to 60 days, 90, a year, whatever.) I've also excluded some things like Christmas music and audio books. Now, whenever I connect my Nano to my computer, iTunes moves the songs I've listened to recently out and puts in new tunes to replace them. Slick.
I could have done this with my iPod Shuffle, but the screen in the Nano makes it better for such a thing since I don't necessarily recognize every song and I like to be able to see what and who are playing.