Using ssldump

Posted on by

Earlier I’ve talked about various ways of observing and debugging SSL connections to the web server, such as acting as a client and acting as a proxy.

Both of these methods have their advantages and disadvantages. Today I’ll review using ssldump, which combines some of the advantages of both of the previous tools: you can observe real requests from a real client and you can also observe the actual application data (HTTP layer). The only drawback is that there is a bit of additonal setup work to do before this will work, but it is definitely worth it.

Unlike ssltap, ssldump observes network traffic directly. So it is not necessary to proxy the client requests through it, just point it at a network interface where the traffic of interest is visible. For the easiest use case you can simply run:

# ssldump -i bge0 -d port 8088
New TCP connection #1: myclient(34286) <-> myserver(8088)
1 1  0.0265 (0.0265)  C>S  Handshake
      ClientHello
        Version 3.0
        resume [32]=
          08 22 e7 bc cf 13 e7 7f 80 0d 62 43 24 4c 65 5b
          1e 19 69 ab 3c 51 0e 95 29 d9 79 9d 9f 79 04 92
        cipher suites
        Unknown value 0x39
        Unknown value 0x38
        Unknown value 0x35
        Unknown value 0x33
        Unknown value 0x32
        SSL_RSA_WITH_RC4_128_MD5
        SSL_RSA_WITH_RC4_128_SHA
        Unknown value 0x2f
        SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        Unknown value 0xfeff
        SSL_RSA_WITH_3DES_EDE_CBC_SHA
        SSL_DHE_DSS_WITH_DES_CBC_SHA
        Unknown value 0xfefe
        compression methods
                  NULL
1 2  0.0273 (0.0007)  S>C  Handshake
      ServerHello
        Version 3.0
        session_id[32]=
          08 22 e7 bc cf 13 e7 7f 80 0d 62 43 24 4c 65 5b
          1e 19 69 ab 3c 51 0e 95 29 d9 79 9d 9f 79 04 92
        cipherSuite         SSL_RSA_WITH_RC4_128_MD5
        compressionMethod                   NULL
1 3  0.0273 (0.0000)  S>C  ChangeCipherSpec
1 4  0.0273 (0.0000)  S>C  Handshake
1 5  0.0635 (0.0362)  C>S  ChangeCipherSpec
1 6  0.0635 (0.0000)  C>S  Handshake
1 7  0.0635 (0.0000)  C>S  application_data
1 8  0.0643 (0.0008)  S>C  application_data
1 9  30.1176 (30.0532)  S>C  Alert

Note that I ran ssldump as root since it needs to listen to the traffic on the network device bge0 (substitute the correct device for your system here – check with ifconfig if in doubt). My SSL-enabled web server listener is on port 8088 for this example, so I limited ssldump to tracking traffic on that port (I could’ve said “dst port 8088” as well), otherwise it would’ve dumped all network traffic on that device, making it difficult to follow the output.

Also check the first column in the output, which above is always “1”. Since ssldump observes the network traffic there may be multiple requests going to the server at once. Above there was only one:

New TCP connection #1: myclient(34286) <-> myserver(8088)

In the case where there are multiple connections observed, each will be numbered. The packets for each connection will most likely be interleaved in the output, but you can track each one by correlating these numbers. That can be quite useful – I’ve used ssldump to diagnose some SSL connectivity problems which were only reproducible when multiple concurrent requests hit the server all at once.

The example above is nice in that I didn’t have to proxy my browser traffic through it, I could just connect to my server as usual. But aside from that it is not so interesting. The output isn’t really any more useful than ssldump.

So, let’s make things more interesting.

If given access to the server private key, ssldump can decrypt the traffic to and from that server on the fly. That’s where it gets really useful. We’ll need to do a bit of prep work to set this up.

First, extract the private key from the server instance into a PKCS#12 format file using pk12util.

  • You’ll need to know the nickname of the server keypair/cert (see your server.xml) for the -n parameter.
  • I changed to the directory where the NSS *.db files live so I type “-d .”. Alternatively you could run the command from elsewhere by giving the right path.
  • Finally, I am runing JES Web Server 7.0 so there is no prefix to the NSS files, but if you are on 6.1 you’ll need to give a -P parameter with the right prefix for that instance.
% pk12util -o myserver.pk12 -n Server-Cert -d . -v
Enter Password or Pin for "NSS Certificate DB":
Enter password for PKCS12 file:
Re-enter password:
pk12util: PKCS12 EXPORT SUCCESSFUL

Perhaps needless to say, but if this was a production server I’d have to be very careful where I store this private key file. Keep that in mind.

Next I’ll just convert this to a format suitable for ssldump using openssl:

% openssl pkcs12 -in myserver.pk12 -out myserverkey
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

Ok.. now lets run ssldump and give it access to the server key material (the “PEM pass phrase” you need to type in below is the same you gave it above):

# ssldump -i bge0 -d -k /tmp/myserverkey port 8088
Enter PEM pass phrase:

Then I connect to my server from a browser:

New TCP connection #1: laptop(39699) <-> myserver(8088)
1 1  0.0853 (0.0853)  C>S SSLv2 compatible client hello
  Version 3.1
  cipher suites
  Unknown value 0x39
  Unknown value 0x38
  Unknown value 0x35
  Unknown value 0x33
  Unknown value 0x32
  TLS_RSA_WITH_RC4_128_MD5
  TLS_RSA_WITH_RC4_128_SHA
  Unknown value 0x2f
  TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
  TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
  Unknown value 0xfeff
  TLS_RSA_WITH_3DES_EDE_CBC_SHA
  TLS_DHE_RSA_WITH_DES_CBC_SHA
  TLS_DHE_DSS_WITH_DES_CBC_SHA
  Unknown value 0xfefe
  TLS_RSA_WITH_DES_CBC_SHA
  TLS_RSA_EXPORT1024_WITH_RC4_56_SHA
  TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA
  TLS_RSA_EXPORT_WITH_RC4_40_MD5
  TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5
1 2  0.0856 (0.0002)  S>C  Handshake
      ServerHello
        Version 3.1
        session_id[32]=
          08 22 c2 25 34 4e 85 61 dd 24 ba 9a 59 a2 dc b0
          77 a0 3f b7 ac c9 d3 ce 76 4a b5 42 cc 44 30 fb
        cipherSuite         TLS_RSA_WITH_RC4_128_MD5
        compressionMethod                   NULL
      Certificate
      ServerHelloDone
1 3  6.1870 (6.1013)  C>S  Handshake
      ClientKeyExchange
1 4  6.1870 (0.0000)  C>S  ChangeCipherSpec
1 5  6.1870 (0.0000)  C>S  Handshake
      Finished
1 6  6.1931 (0.0061)  S>C  ChangeCipherSpec
1 7  6.1931 (0.0000)  S>C  Handshake
      Finished
1 8  6.2852 (0.0921)  C>S  application_data
    ---------------------------------------------------------------
    GET / HTTP/1.1
    Host: myserver:8088
    User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5)
 Gecko/20041217
    Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/
plain;q=0.8,image/png,*/*;q=0.5
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 300
    Connection: keep-alive

    ---------------------------------------------------------------
1 9  6.2859 (0.0007)  S>C  application_data
    ---------------------------------------------------------------
    HTTP/1.1 403 Forbidden
    Server: Sun-Java-System-Web-Server/7.0
    Date: Thu, 27 Apr 2006 08:12:15 GMT
    Content-length: 142
    Content-type: text/html

    Forbidden



Forbidden




    Your client is not allowed to access the requested object.
    --------------------------------------------------------------
-
1 10 12.0328 (5.7468)  C>S  Alert
    level           warning
    value           close_notify
1    12.0360 (0.0032)  C>S  TCP FIN
1    12.0361 (0.0000)  S>C  TCP FIN

Nice! Now I can watch all the application data traffic in plaintext. As you can imagine, this can be very useful for diagnosing all kinds of problems with an SSL-enabled server.

As a final note, when building ssldump be sure to build it with openssl support or the traffic decryption will not work. Here’s the configure options I gave it on my Solaris 10 machine:

./configure --with-pcap=/opt/sfw --with-openssl=/usr/sfw