4

The rsautl OpenSSL utility does not appear to have a setting wherein the hash for OAEP can be changed. My testing suggests that it just defaults to sha1 and that's it.

My question is... is there a way to change the hash?

The command I used to test it out:

openssl rsautl -inkey publickey.txt -encrypt -oaep -in plaintext.txt -out ciphertext.txt

migrated from crypto.stackexchange.com Dec 22 '15 at 10:38

This question came from our site for software developers, mathematicians and others interested in cryptography.

10

rsautl which is ancient, no. pkeyutl which was supposed to supersede rsautl in 2010, yes. -pkeyopt rsa_oaep_md:name or rsa_mgf1_md:name where name is a hash as in your Q on PSS.

Note for pkeyutl (edit) before 1.1.0 the order of options matters; you must do -encrypt then -inkey rsaprivate then -pkeyopt (not currently documented AFAICS, though the analogous cases in genpkey are) and you need to do rsa_padding_mode:oaep before rsa_{oaep,mgf1}_md:hash .

Same possible issue with FIPS mode.

  • 2
    I was trying this just now (maybe I tried it earlier too idk) and I had to do -pkeyopt before each option. eg. openssl pkeyutl -in ciphertext.txt -decrypt -inkey test.pem -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 -pkeyopt rsa_mgf1_md:sha1. So in that case I needed to do it once for each option - not once globally. – neubert Sep 17 '16 at 5:06
  • 1
    @neubert: yes you need -pkeyopt for each key-option; I thought that part was clear from the manpage and didn't repeat it. But the requirements on order were (and are) not all documented AFAICS, and such requirements are unusual for openssl, so I called them out. – dave_thompson_085 Sep 19 '16 at 6:16
  • 1
    The ordering doesn't appear to matter with OpenSSL 1.1.0f, but I couldn't find these options documented. Although -pkeyopt rsa_oaep_md:sha256 works, it is not mentioned on man pkeyutl. – starfry Jun 23 '17 at 9:04
  • @starfry: (don't know why I wasn't notified at the time) yes, 1.1.0, released 8 months after I wrote this answer, completely reimplemented option handling for all commandline operations; edited for anyone else who comes along. It's rsa_mgf1_md which applies to both OAEP and PSS -- and is documented in 1.1.1 which turnaboutly postdates your comment :-) – dave_thompson_085 Feb 5 at 2:25

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.