I don't think that's possible with Firefox Quantum anymore. I remember doing the same many years ago - editing XML file manually to adjust search settings (see here).
I've tried to find a file containing search engine settings, by looking for a phrase with one of my search engine's name recursively in Firefox profile directory:
grep -Ri 'some search engine name' /home/yahol/.mozilla/firefox/9s05hy1j.default/
None of the returned files referred to search engines.
There are some alternate solutions however, that will allow you generating a new search engine with custom:
I've tested Ready2Search and it works. When creating a search engine with an icon, make sure that image file has an extension (.gif, .jpeg, .png), otherwise it will spill errors about incorrect/unsupported file format. I think Mycroft Project might be more intuitive and user-friendly. And has great customization options.