Had the same question when I stumbled upon yours here.
After some research, the best I can come up with is this from Oracle:
Why don't I see the option to select Do not show this again for this
app in the security dialog for an unsigned application?
Java 7 Update 40, the option to select Do not show this again for this
app is no longer available. Unlike previous versions a user cannot
suppress the security dialog for an unsigned application and will have
to select the option, I accept the risk and want to run this app, each
time to run the unsigned application.
Prior to Java 7u40, there was a checkbox which said “Do not show again for this application” … which would make the dialog not appear again unless certificates were changed or a new version of the applet was deployed.
Interestingly, the specific release notes don’t mention this change: http://www.oracle.com/technetwork/java/javase/7u40-relnotes-2004172.html
Since this pre-dated Java 8, it's likely you were running an older Java 7 version and then started to notice this behaviour when you upgraded to Java 8.
I did poke through the Control Panel options and there does not seem to be a way to work around this.
So, if I've understood correctly, it is unavoidable to have this message display every the app is launched.