"What format are your video files in ? If they're DVD MPEG-2 then Handbrake can convert them to MP4/M4V or MKV, both of which support chapter markers.
If you already have MP4 files I believe you can use Subler to add chapters or something like Metadata Hootenanny."
Basically it is saying your MP4's needs to have the Chapters created before using handbrake. You may try Subler to add chapters. I have never tried Subler so I wouldn't be able to give you any specifics.