Yes, it is as safe as possible.
With a smaller open source project written almost exclusively by 1-2 people, there is some risk that they can hide something. But with big, well-known and well-tested software such as the Linux kernel, Firefox, and most packages which are a default part of a big distribution, you can be reasonably sure that lots of independent eyes have seen every line of code. You're probably more at risk of a rogue Microsoft employee sneaking in a keylogger into Windows just before a deadline.
The second point is, Linux isn't more secure just because there is fewer Linux malware written. It was designed with a better security model. With Windows, almost everybody ran as admin up to Win XP, and because of the UAC usability disaster in Vista, most people just continue this today, even on 7. The point of running as non-admin isn't to save you from yourself - if it is your own PC, it'll ask you for admin password for deleting System32 and you'll provide it - but that software processes started while you're running as non-admin don't have admin rights, so they are barred from anything the non-admin user is barred from. Besides, there are additional security mechansims built into operation systems, but many such were built into Windows later than their Linux counterparts, and recent studies show that third-party software doesn't use them. Link
And besides, you are not the first who thought of it. A big computer magazine in Germany, c't, actually regularly provides a "banking CD" with its magazine. It is just as you described, a LiveCD based on Ubuntu with some branding. I don't know if they provide specific tools on it too, but if you think that it is more secure than a "common" Ubuntu LiveCD, you could order a past issue with that CD from heise.de. Personally, I wouldn't bother and just take the LiveCD of some big distro.