Traditionally unix systems (including Linux) track the date when a file was last read: its access time, often called atime for short. With these, you can detect that you haven't watched a video because its atime is the same as its mtime (modification time, the time you see in the ls -l output). On the command line, ls -lutr shows files with their access times. There are ways to locate files in a directory according to various time-related criteria, using find or zsh.
Ubuntu nowadays turns off access times, because there's a performance penalty. You can reenable them by editing /etc/fstab and changing the line for a filesystem to include the atime option, as in
/dev/sda1 / ext4 errors=remount-ro,atime 0 1
(You may see UUID=… in the first column.) This is a per-filesystem setting, you can't enable atimes only for a directory tree (unless you make it a separate filesystem).
You can also access your files through loggedfs
. This is a virtual filesystem that shows a view of the “real“ files underneath, and in addition logs all accesses. The documentation is a little scarce.