Okay, I give up. How do I size limit which files are downloaded, like say I don't want any files bigger than 2 MB?
The only limitation option I know which
I don't know what environment you're using, but crawler supports file size limitations with max-length-bytes and runs on the Java platform.
from their user manual:
If its about "downloading 2MB max" rather than "download files with max 2MB" you could just limit the output saved to disk.
(enough to see a "OK:$Message", not killing my /tmp with tons of error messages from the remote ;-))