Talking about "data" is too broad, but it's pretty obvious you are actually talking about files. You need to be aware that files are usually stored in filesystems. Although there are differences, this generally means there's a tree structure of paths, representing either files or directories, and files are assigned space on the hard disks where its contents are stored.
Commands such as
rm simply remove the file, i.e. the entry in the tree: they mark the parent directory as having one less child. At the address where the file was, and where the file's contents were, nothing changes.
However, there are also commands such as
shred, which actually overwrite the file's contents, so that it (ideally) can't be restored. This is actually harder than the former, because it's not guaranteed that the same sectors are used for storing the overwriting data, and because physically, the magnetic data on the disk is not simply binary. Also, such tools do not simply zero out the data but overwrite it multiple times with patterns or random data, to make it physically harder to restore.
So, there are both options you mentioned, but the everyday deletion is simply removing the pointer to the actual data.