I feel this way about it too. I spend $5.00 a year on the lottery. Usually I'll bother with it if it's an unusually big jackpot. I like the dream, but I'm pretty sure I'll never win.
I think I've bought maybe $10 of lotto tickets in 5 years time. I drive by a big billboard that gives the day's millions, and so the thought of what I would do with all that pops up in my head, but it never really goes beyond that.