It sounds like you are looking for MySQL Workbench:
It can access your MySQL server (directly or via SSH) from any remote desktop machine.
Most other Linux administration tasks are normally done in the console (i.e. via the command line). If you really want to access GUI tools on the remote host, though, you can look at running a remote X session. It generally doesn't perform very well, though, in my experience. You're probably better off just sticking with the command line.
By the way, the remote machine that you use to manage your Ubuntu servers doesn't have to be an Ubuntu machine. You can use another distribution of Linux, or a Windows or Mac machine.
On Windows, I know that MySQL Workbench runs just fine. Most Windows users use PuTTY to connect to a Linux machine. You can get PuTTY here: http://www.chiark.greenend.org.uk/~sgtatham/putty/