Is there anyway for another user to change permissions of file without root permissions? If another user has access to my drive and they can theoretically change the user bit in the file via a binary or hex editor, could they change the permission?
The ownership and permissions are not stored in the file, but in the file system, which in turn lies on the hardware storage device (usually). Non-root users do not have access to the raw file system data / raw block device for obvious reasons, to prevent exactly this. If the file is on a removable storage medium which the other user can physically access, he can obviously just connect it to his computer, log in as root on his computer, mount the medium, and chmod/chown the file.