Never enter a house with your shoes. This is one of the few rules for which Japanese will not make allowance just because you are a foreigner. This rule is also valid for some establishments like schools. Slippers are usually provided in the entrance hall. If slippers are provided for the toilet, use them instead of the one for the rest of the house.
When you are invited into a Japanese family, bring a small present or "omiyage" (souvenir, usually food). If you are coming straight from your country, it is preferable to bring some local culinary specialties from your home town/region.
Say "o-jama shimasu" (sorry for disturbing) while entering someone's house.
Some shops, cafes or department stores provide plastic covers for umbrellas. Make sure not to enter with a dripping wet umbrella without one.
Refrain from blowing your nose in front of other people. Japanese only use paper tissue for this. Like in other Asian countries, it is considered rude to blow you nose in a handkerchief and stuff it in your pocket afterward. Japanese are usually aware of this Western practice, although that might make them feel uncomfortable.
You should not eat while standing or walking in the street. Even inside a house, you should sit down to eat. The only exceptions are for eating at a counter (e.g. ramen) or for eating an ice-cream in the street. This custom is one of the most difficult to adapt to for many non-Japanese, as it doesn't seem to make much sense.
Do not point your finger, feet or chopsticks at people. If you have to indicate an object or direction to someone, wave your fingers with the palm downwards.
Avoid being expressing your opinion too directly. Japanese have what they call "honne" (real opinion) and "tatemae" (public opinion). They will express the latter in most situation so as not to disturb the group harmony. It is of course flexible and consist in agreeing with the people around you as much as possible. This is the reason why Japanese are so bad at debating serious issues in public (including the media). "Honne" is what you really think but do not say openly, or only to close friends or relatives.
Avoid interrupting people when they are speaking or thinking about an answer. Japanese do not mind short periods of silence in the middle of a discussion.
Avoid fixing someone in the eyes (for men, even, or especially beautiful girls sitting in front of you in the train).
Do not use your mobile phone in trains unless it is clearly allowed to do so. Using emails or SMS is fine though.
Money should be given in an envelope, but only about half the Japanese really take the trouble. Most men do not seem to care, except for formal situations. Never forget this rule for weddings. In addition, the number of banknotes given to the married couple should be a odd number, as superstitious people believe that the couple might separate if the number can be divided in two.
Do not stick your chopsticks vertically into a bowl of rice, as this is used in Buddhist funerary ceremony.
Do not pass food to someone else with your chopsticks for the same reason as above.
At a "nomikai" (e.g. while going drinking with colleagues at an Izakaya), you should (re)fill the glasses of people around you when they are empty, and they should do the same for you. If you want to refill you glass, start by serving other people. If you do not want a refill, do not empty you glass.
It is polite to say "itadakimasu" once before eating or drinking, and "gochisousama deshita" to your host or to the restaurant's staff after eating or when leaving the place.
Contrarily to Western manners, noodles can be and should be slurped. Likewise, bowls or plates should be brought up the the mouth rather than bending one's head toward it.
"Meishi" (business cards) are exchanged when meeting someone for the first time. They should be given and accepted with both hands in formal situations.
Make sure to observe it carefully and remember your opposite's name. Place the card on the table in front of you if you are sitting, or put it in your wallet. Do not put a proffered cards into your pocket or fold it in any way.
Japanese wash themselves before entering the bath, as they have a customs of sharing the bath water. This is true as well for public baths (sentō 銭湯) as for thermal spring (onsen 温泉) and bath in individual homes. The reason is that other people will use the same water after you (except if you live by yourself, of course). Therefore, you should not empty the bath after using it.
Japanese like bathing in (very) hot water (40 to 50 degrees celsius). If it is too hot for you, you can add a bit of cold water, but not as much as it becomes tepid, or the next person won't appreciate it.
In public baths, do not mistake men and women's changing rooms, as it is extremely impolite, even if you really mistook. The men's room are usually on the left, and normally has a blue curtain with "otoko" (男) or dono-sama (殿様) written on it. The women's room is usually on the right, with a red curtain reading "onna" (女). If you are not sure, ask !
Absolutely avoid bathing suits in public baths, as this could create incidents with Japanese customers and you could end up expelled from the premises.
Tattoos are banned in most public baths. If you have one, you should consult the staff at reception beforehand to avoid causing trouble.
In the most traditional families, you might have to do the following things.
Sit in the "seiza" position. This can be difficult and painful for Japanese themselves, especially taller people. It involves sitting on the floor with the legs folded under your body, with your back resting on your heels.
It is usually said that people should avoid the number "4" for gifts. Like in China and Korea, 4 is pronounced the same way as "death". However, it seems that very few Japanese people really care about this superstition nowadays.