I know that there are tape backup systems available. I do not know how they work.
Can I use an audio tape recorder and audio tapes (like the 60/90 minute kind) and backup data somehow? (Pardon my n00bness, I've never done tape backups before.)
You can and that was very common in the 80th: Datasette (Wikipedia), but nowadays you won't be satisfied with a storage capacity in the order of 1MB. For a review of modern Tape Storage format, also see Wikipedia.
When I was starting out on my first computers back in the 1980's this is exactly what we did.
But programs were around 16-32 kB at the very most - yes that's kilo not even mega, certainly not giga!!
And, the backups failed to restore as often as not.
And, it took AGES to both backup and restore.
These days, disk space is very cheap and even Internet based storage is fairly cheap. So use those for backups. Keep at least 2 backups of all data and they must be both automatic and located in at least 2 different locations. Don't keep all backups onsite - fire, theft, etc. will see you loose all of your backups.
Personally, I have one backup to another machine locally in the house and a second to servers in the US (I'm in the UK). Oh, backups of data should also be locally encrypted as well. For reference, I use a tool called CrashPlan.
Great question though, welcome to SuperUser.