This expression of surprise on their face always makes me feel like they first thought I was the last of the idiot.
Well, look at it from their perspective. How many non-Asian gaijin in Japan? How many of those speak Japanese? How may of those speak Japanese well? The first question starts at a very low percent, and then just goes lower and lower. Of course you're going to run into people that are surprised. Give it time... the novelty wears off after a while.

Funny that I haven't experienced anything similar in any other country
Not necessarily. What are the homogeneity figures of those countries? I couldn't imagine many European countries where one would be surprsied by a multilingual. Westerners are just not well known for learning Asian langauges. I don't think that's very strange to show a little interest/surprise in one speaking their native language.

I'd say that those the most likely to fit this generalisation are those who do not often meet foreigners (i.e. most Japanese) or have only met the "bad examples" of gaijin.
That's most likely the issue. I've bought items online through Yahoo many times and have often been told, "this is my first time to talk to a foreigner". I figure at the 5 year mark you get to the point where you're just immune to it. People ask if why you speak Japanese and you just tell them how many years you've lived there and go on with your day. You may even get confident/comfortable enough that you don't mind the odd acquantaince wanting to practice their elementary English on you.