I'm interpreting this question to be about things I personally find unpleasant or inconvenient when living in Japan, rather than matters of policy or suchlike (though I think the choices given straddle that).

My biggest complaint would be the climate. Japan certainly does have four seasons: a little too hot, a little too cold, way too hot and way too cold (^_^). But then, I'm from California so most anywhere seems a step down on that score.

I disliked the crowding and how cramped things are, and some of the things that flow from that. Even fairly small towns have a dense, urban mid-rise feel to them. I'm the first to admit that they're better than American cities in many ways, but concrete is concrete and traffic noise is traffic noise.

I also found the "one right way to do anything" attitude increasingly annoying the longer I stayed. As a foreigner, you're excused from a lot of it and your lapses often seen as more endearing than anything else. It can become a nasty issue if you get into closer relationships though, especially with the Japanese reticence to openly discuss disagreements.