It is true that there is not much of english spoken outside of the large cities. Even though, I never had issues with communicating with the locals. You will get along great just by pointing, using gestures and so on. Also there is a lot of adopted English words in Japanese (Japenglish) 😀
I use Google Translate for my immediate needs: Where ever there is internet, this is the easiest way to go. Google can translate kanji in pictures taken by your phone as well as anything spoken or typed into the smartphone in both ways.
For offline use, don’t forget to download Japanese and the language you wish to translate to on your device.
A few helpfull phrases:
Thank you. – Arigatou gosaimasu.
Full Tank please. – Mantan o shitte onegai shimasu.
Where is the toilet. – Toire wa doko desu ka.
The bill please. – okanjō onegai shimasu.
My favorite Kanji Trainer if you plan to seriously learn Japanese: