I want to upload all my project files to the server over SSH, I want to avoid guessing what files I need to upload.
I know GIT does this when remote pushing to Github but I need to upload it directly to my server instead of Github.
Set up a git repository at your server, then you can push stuff over there with no problems. Check the documentation at http://www.git-scm.com (there are several tutorials, and even a complete book). Check the documentation of git on how to configure remote branches.