Earlier I’ve talked about various ways of observing and debugging SSL connections to the web server, such as acting as a client and acting as a proxy.
Both of these methods have their advantages and disadvantages. Today I’ll review using ssldump, which combines some of the advantages of both of the previous tools: you can observe real requests from a real client and you can also observe the actual application data (HTTP layer). The only drawback is that there is a bit of additonal setup work to do before this will work, but it is definitely worth it.
Unlike ssltap, ssldump observes network traffic directly. So it is not necessary to proxy the client requests through it, just point it at a network interface where the traffic of interest is visible. For the easiest use case you can simply run:
# ssldump -i bge0 -d port 8088 New TCP connection #1: myclient(34286) <-> myserver(8088) 1 1 0.0265 (0.0265) C>S Handshake ClientHello Version 3.0 resume [32]= 08 22 e7 bc cf 13 e7 7f 80 0d 62 43 24 4c 65 5b 1e 19 69 ab 3c 51 0e 95 29 d9 79 9d 9f 79 04 92 cipher suites Unknown value 0x39 Unknown value 0x38 Unknown value 0x35 Unknown value 0x33 Unknown value 0x32 SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA Unknown value 0x2f SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA Unknown value 0xfeff SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA Unknown value 0xfefe compression methods NULL 1 2 0.0273 (0.0007) S>C Handshake ServerHello Version 3.0 session_id[32]= 08 22 e7 bc cf 13 e7 7f 80 0d 62 43 24 4c 65 5b 1e 19 69 ab 3c 51 0e 95 29 d9 79 9d 9f 79 04 92 cipherSuite SSL_RSA_WITH_RC4_128_MD5 compressionMethod NULL 1 3 0.0273 (0.0000) S>C ChangeCipherSpec 1 4 0.0273 (0.0000) S>C Handshake 1 5 0.0635 (0.0362) C>S ChangeCipherSpec 1 6 0.0635 (0.0000) C>S Handshake 1 7 0.0635 (0.0000) C>S application_data 1 8 0.0643 (0.0008) S>C application_data 1 9 30.1176 (30.0532) S>C Alert
Note that I ran ssldump as root since it needs to listen to the traffic on the network device bge0 (substitute the correct device for your system here – check with ifconfig if in doubt). My SSL-enabled web server listener is on port 8088 for this example, so I limited ssldump to tracking traffic on that port (I could’ve said “dst port 8088” as well), otherwise it would’ve dumped all network traffic on that device, making it difficult to follow the output.
Also check the first column in the output, which above is always “1”. Since ssldump observes the network traffic there may be multiple requests going to the server at once. Above there was only one:
New TCP connection #1: myclient(34286) <-> myserver(8088)
In the case where there are multiple connections observed, each will be numbered. The packets for each connection will most likely be interleaved in the output, but you can track each one by correlating these numbers. That can be quite useful – I’ve used ssldump to diagnose some SSL connectivity problems which were only reproducible when multiple concurrent requests hit the server all at once.
The example above is nice in that I didn’t have to proxy my browser traffic through it, I could just connect to my server as usual. But aside from that it is not so interesting. The output isn’t really any more useful than ssldump.
So, let’s make things more interesting.
If given access to the server private key, ssldump can decrypt the traffic to and from that server on the fly. That’s where it gets really useful. We’ll need to do a bit of prep work to set this up.
First, extract the private key from the server instance into a PKCS#12 format file using pk12util.
- You’ll need to know the nickname of the server keypair/cert (see your server.xml) for the -n parameter.
- I changed to the directory where the NSS *.db files live so I type “-d .”. Alternatively you could run the command from elsewhere by giving the right path.
- Finally, I am runing JES Web Server 7.0 so there is no prefix to the NSS files, but if you are on 6.1 you’ll need to give a -P parameter with the right prefix for that instance.
% pk12util -o myserver.pk12 -n Server-Cert -d . -v Enter Password or Pin for "NSS Certificate DB": Enter password for PKCS12 file: Re-enter password: pk12util: PKCS12 EXPORT SUCCESSFUL
Perhaps needless to say, but if this was a production server I’d have to be very careful where I store this private key file. Keep that in mind.
Next I’ll just convert this to a format suitable for ssldump using openssl:
% openssl pkcs12 -in myserver.pk12 -out myserverkey Enter Import Password: MAC verified OK Enter PEM pass phrase: Enter PEM pass phrase: Verifying - Enter PEM pass phrase:
Ok.. now lets run ssldump and give it access to the server key material (the “PEM pass phrase” you need to type in below is the same you gave it above):
# ssldump -i bge0 -d -k /tmp/myserverkey port 8088 Enter PEM pass phrase:
Then I connect to my server from a browser:
New TCP connection #1: laptop(39699) <-> myserver(8088) 1 1 0.0853 (0.0853) C>S SSLv2 compatible client hello Version 3.1 cipher suites Unknown value 0x39 Unknown value 0x38 Unknown value 0x35 Unknown value 0x33 Unknown value 0x32 TLS_RSA_WITH_RC4_128_MD5 TLS_RSA_WITH_RC4_128_SHA Unknown value 0x2f TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA Unknown value 0xfeff TLS_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_RSA_WITH_DES_CBC_SHA TLS_DHE_DSS_WITH_DES_CBC_SHA Unknown value 0xfefe TLS_RSA_WITH_DES_CBC_SHA TLS_RSA_EXPORT1024_WITH_RC4_56_SHA TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA TLS_RSA_EXPORT_WITH_RC4_40_MD5 TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 1 2 0.0856 (0.0002) S>C Handshake ServerHello Version 3.1 session_id[32]= 08 22 c2 25 34 4e 85 61 dd 24 ba 9a 59 a2 dc b0 77 a0 3f b7 ac c9 d3 ce 76 4a b5 42 cc 44 30 fb cipherSuite TLS_RSA_WITH_RC4_128_MD5 compressionMethod NULL Certificate ServerHelloDone 1 3 6.1870 (6.1013) C>S Handshake ClientKeyExchange 1 4 6.1870 (0.0000) C>S ChangeCipherSpec 1 5 6.1870 (0.0000) C>S Handshake Finished 1 6 6.1931 (0.0061) S>C ChangeCipherSpec 1 7 6.1931 (0.0000) S>C Handshake Finished 1 8 6.2852 (0.0921) C>S application_data --------------------------------------------------------------- GET / HTTP/1.1 Host: myserver:8088 User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5) Gecko/20041217 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/ plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive --------------------------------------------------------------- 1 9 6.2859 (0.0007) S>C application_data --------------------------------------------------------------- HTTP/1.1 403 Forbidden Server: Sun-Java-System-Web-Server/7.0 Date: Thu, 27 Apr 2006 08:12:15 GMT Content-length: 142 Content-type: text/html ForbiddenForbidden
Your client is not allowed to access the requested object. -------------------------------------------------------------- - 1 10 12.0328 (5.7468) C>S Alert level warning value close_notify 1 12.0360 (0.0032) C>S TCP FIN 1 12.0361 (0.0000) S>C TCP FIN
Nice! Now I can watch all the application data traffic in plaintext. As you can imagine, this can be very useful for diagnosing all kinds of problems with an SSL-enabled server.
As a final note, when building ssldump be sure to build it with openssl support or the traffic decryption will not work. Here’s the configure options I gave it on my Solaris 10 machine:
./configure --with-pcap=/opt/sfw --with-openssl=/usr/sfw