This is not determined by the timezone, but rather by locale settings.
I'm not sure if Xfce has an UI for specifying system locale in a detailed enough manner. But at the low level, there are several environment variables, which you can see by typing
locale. The main one is
$LANG, which sets the general system locale, and for specific areas there are separate
$LC_* variables – e.g.
$LC_MESSAGES for the translations used by programs, and
$LC_TIME for the date/time formats.
If you simply want everything to be in English, then the configuration is simple – select an English locale in your Xfce settings. (Typically "United States" aka en_US.UTF-8, though I use en_DK.UTF-8 for LC_TIME myself, as it gives ISO 8601 dates.) This will not affect time zones at all.
If you want specific settings for English and others for Japanese, you might have to manually "export" the specific variables in your
/etc/default/locale or ... wherever Ubuntu puts it. For example:
export LANG=... is necessary.)
Another thing to note that if you switch between two
*.utf-8 locales, it will not affect the character sets either – any UTF-8 locale still uses Unicode, so Japanese text will still work with
en_US.utf-8. (Not to be confused with the old
en_US which uses legacy ISO 8859.)