Google stupidly crippled the preferences UI so you can't edit the protocol handlers any longer that you established by answering the little dialog (screenshot) with "Always open ...". What I did to solve my problem (assuming Chrome "Default" profile):
End Chrome. Then edit this text file:
C:\Users\USERNAME\AppData\Local\Google\Chrome\User Data\Default\Preferences
It's all in one line, not meant for human editing. If your editor (e.g. Notepad++) features language highlighting, choose JSON.
I had to change the text
"protocol_handler":{"excluded_schemes":{"mql4buy":false}},
to
"protocol_handler":{"excluded_schemes":{}},
so I just removed
"mql4buy":false
This was for the custom protocol link mql4buy://(etc)
Since your problem is that your protocol is ignored, maybe you have an entry for your protocol saying ":true".