I think the reason AppleTV is still a hobby is because there is limited internet bandwidth in most homes.
One does not need broadband in order to play PSP or Wii.
Secondly Apple believed that the success of the iTunes store and music sales would naturally transition into a successful movie and TV shows sales bonanza. Please refer to my first statement regarding internet speed. If one has two wait 3 hours for a movie to download then they could easily go to the video store or rent 3 movies via netflix and get them in the mail. Plus the limited rental options do not help.
Apple was surprised to see how successful the iPod Touch was for gaming. They are transitioning that success to the iPad. I suspect that if that turns into a bigger gaming platform, it is only natural to see that move right into the big screen and true gaming console.
So I believe Apple will make the AppleTV into a gaming console, but we may have to wait a few more years.