I have a pcap tracefile and private RSA key that should allow me to decrypt the HTTPS data in the pcap file.

With the exact same pcap file, private key and tshark command, I find that decryption fails in Oracle Linux 6 but succeeds in Ubuntu 18 and RHEL 8.

I would like to know what difference in environment/versions allows decryption to succeed on newer OSs but fail on the older OS.

tshark command is as follows (IP, private key path and pcap name simplified here):

tshark -V -n -o ssl.keys_list:,443,http,/path/to/key.pem -o ssl.desegment_ssl_records:TRUE -o ssl.desegment_ssl_application_data:TRUE -o ssl.debug_file:rsa_private.log -r ./trace.pcap

Client Hello TLS information (as seen on RHEL8):

TLS 1.2 (0x0303)
Extension: renegotiation_info (len=1)
Extension: server_name (len=24)
Extension: extended_master_secret (len=0)
Extension: signature_algorithms (len=20)
Extension: ec_point_formats (len=2)
Extension: supported_groups (len=8)

Server Hello TLS information (as seen on RHEL8):

Version: TLS 1.2 (0x0303)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Extension: renegotiation_info (len=1)
Extension: server_name (len=0)
Extension: extended_master_secret (len=0)

In my rsa_private.log log I find the following error messages on the old OS:

ssl_decrypt_record found padding 111 final len -48
ssl_decrypt_record wrong record len/padding outlen 96
ssl_decrypt_record: mac failed

Version information from different environments:

Oracle Linux 6 (fails to decrypt):

  • TShark 1.8.10
  • GLib 2.28.8
  • GnuTLS 2.12.23
  • Gcrypt 1.4.5
  • Linux 4.1.12-124.30.1.el6uek.x86_64
  • libpcap version 1.4.0

Ubuntu 18 (decrypts successfully):

  • TShark (Wireshark) 2.6.10
  • GLib 2.56.4
  • GnuTLS 3.5.18
  • Gcrypt 1.8.1
  • Linux 4.18.0-22-generic
  • libpcap version 1.8.1

RHEL8 (decrypts successfully):

  • TShark (Wireshark) 2.6.2
  • GLib 2.56.1
  • GnuTLS 3.6.4
  • Gcrypt 1.8.3
  • Linux 4.18.0-80.4.2.el8_0.x86_64
  • libpcap version 1.9.0-PRE-GIT

I presume something was added to libcrypt/openssl/gcrypt that allows the newer OS's to decrypt my traffic successfully, but I wonder what?

I have found other traffic using the same TLS version and cipher suite decrypt fine in the older OS.

EDIT: adding detail about TLS extension visibility in OL6 vs RHEL8:

OL6 shows the extension as follows (in both Hello's):

Extension: Extended Master Secret        
    Type: Extended Master Secret (0x0017)
    Length: 0

RHEL8 shows the extension as follows (in both Hello's):

Extension: extended_master_secret (len=0)
    Type: extended_master_secret (23)
    Length: 0

migrated from crypto.stackexchange.com Oct 10 at 17:27

This question came from our site for software developers, mathematicians and others interested in cryptography.

  • Welcome to crypto.stackexchange - Our site is really more about the mathematical details of cryptography than tasks such as this. This question is more suitable for a site like superuser - allow me to migrate this there for you. – Ella Rose Oct 10 at 17:27
  • Almost certainly it's extended_master_secret rfc7627 published in 2015-09, which changes the key derivation and thus the keys. Wireshark 1.8.10 was in 2013-09, 2.6.2 in 2018-07 and 2.6.10 just a few months ago in 2019-07. Does OL6/1.8.10 even show the hello extension decoded as EMS? (The crypto library versions won't have made any difference; RSA HMAC-SHA1 and AES-128-CBC haven't changed since about 2001.) – dave_thompson_085 Oct 10 at 23:21
  • Thanks for the insight - it would explain the failure to decrypt on OL6. I have added details about EMS appearance in OL6 vs RHEL8. I am developing a C application that uses OpenSSL 1.0.x to perform TLS decryption. My app is failing similar to tshark in OL6. Do you know if newer extensions like EMS require OpenSSL 1.1.x to read correctly, or can I get by with OpenSSL 1.0.x? stackoverflow.com/questions/46943850/… implies 1.0.x will not allow modern extensions to be read. – Neal Eastwood Oct 11 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.