TLS 1.2 Deep Dive: Complete Handshake Breakdown

Article 2 of 6: Understanding TLS Security Series

In the previous article, we covered cryptographic primitives. Now we examine how TLS 1.2 combines these building blocks to establish secure connections.

TLS 1.2, standardized in August 2008 (RFC 5246), remains deployed as a fallback for legacy clients. This article provides a complete breakdown of the handshake, key derivation, certificate trust model, and Wireshark analysis.

2026 Update: TLS 1.2 is now legacy fallback only. TLS 1.3 handles 90%+ of connections on modern infrastructure. Keep TLS 1.2 enabled only for compatibility with older clients, and always prefer TLS 1.3.

TLS 1.2 Overview

Property	TLS 1.2
RFC	RFC 5246 (August 2008)
Handshake	2-RTT (two round trips)
Forward Secrecy	Optional (requires ECDHE cipher suites)
Certificate Privacy	NO - transmitted in plaintext
Key Derivation	PRF (Pseudo Random Function) with HMAC-SHA256
Cipher Suites	37+ available (many weak/legacy)

1. TLS 1.2 Handshake (2-RTT)

Theory: How It Works

The TLS 1.2 handshake is a 2-RTT process that negotiates cryptographic parameters, authenticates the server, and derives shared session keys.

CLIENT                                          SERVER
  |                                               |
  |  1. Client Hello                              |
  |  ──────────────────────────────────────────►   |
  |  TLS versions, cipher suites, client random   |
  |                                               |
  |                          2. Server Hello      |
  |  ◄──────────────────────────────────────────   |
  |  Selected version, cipher, server random      |
  |                                               |
  |            3. Certificate (PLAINTEXT!)         |
  |  ◄──────────────────────────────────────────   |
  |  X.509 chain - VISIBLE to observers!          |
  |                                               |
  |            4. Server Key Exchange              |
  |  ◄──────────────────────────────────────────   |
  |  ECDHE public key (signed by cert)            |
  |                                               |
  |            5. Server Hello Done               |
  |  ◄──────────────────────────────────────────   |
  |                                               |
  |  6. Client Key Exchange                       |
  |  ──────────────────────────────────────────►   |
  |  Client ECDHE public key                      |
  |                                               |
  |  ═══════ ENCRYPTION STARTS ═══════            |
  |                                               |
  |  7. Change Cipher Spec + Finished             |
  |  ──────────────────────────────────────────►   |
  |  Encrypted with session keys                  |
  |                                               |
  |  8. Change Cipher Spec + Finished             |
  |  ◄──────────────────────────────────────────   |
  |  Handshake complete                           |
  |                                               |
  |  Application Data (Encrypted)                 |
  |  ◄────────────────────────────────────────►   |

              Total: 2 Round Trips (2-RTT)

Handshake Steps Explained:

1. Client Hello: Client sends supported TLS versions, cipher suites (in preference order), 32-byte random, session ID, and extensions (SNI, ALPN, supported curves)
2. Server Hello: Server selects TLS version and cipher suite, sends 32-byte server random and session ID
3. Certificate: Server sends X.509 certificate chain IN PLAINTEXT. This is a major privacy issue. Any observer can see which site you are connecting to
4. Server Key Exchange: For ECDHE, server sends ephemeral public key signed with certificate private key
5. Server Hello Done: Signals server has finished its part
6. Client Key Exchange: Client sends its ECDHE public key. Both can now compute Pre-Master Secret
7. Change Cipher Spec + Finished: Both signal switch to encrypted mode and send encrypted verification

Practical: Real-World Implementation

Capture TLS 1.2 Handshake with OpenSSL:

# Connect and show handshake details
openssl s_client -connect example.com:443 -tls1_2 -state -debug

# Show certificate chain
openssl s_client -connect example.com:443 -showcerts

# Show negotiated cipher suite
openssl s_client -connect example.com:443 -tls1_2 | grep 'Cipher'

Wireshark Filters:

Filter	Shows
tls.handshake	All handshake messages
tls.handshake.type == 1	Client Hello only
tls.handshake.type == 2	Server Hello only
tls.handshake.type == 11	Certificate message
tls.handshake.type == 12	Server Key Exchange
tls.handshake.type == 16	Client Key Exchange

Reality: Security Risks

Risk	Impact	Mitigation
Plaintext Certificate	Passive observer sees which site you visit	Use TLS 1.3 (encrypted cert)
Downgrade Attack	MITM forces weak cipher/version	TLS_FALLBACK_SCSV, disable old versions
Weak Cipher Selection	Server picks vulnerable cipher	Configure server cipher preference
RSA Key Exchange	No forward secrecy, past traffic exposed	Use only ECDHE cipher suites
Session Resumption	Session tickets can be stolen	Rotate session ticket keys frequently

2. Key Derivation

Theory: How It Works

TLS 1.2 derives encryption keys using a PRF (Pseudo Random Function) based on HMAC-SHA256.

Key Derivation Process:

1. Pre-Master Secret: Computed from ECDHE key exchange (or RSA encrypted)
2. Master Secret: PRF(pre_master_secret, "master secret", ClientRandom + ServerRandom)
3. Key Block: PRF(master_secret, "key expansion", ServerRandom + ClientRandom)

Key Block Contains:

• client_write_MAC_key / server_write_MAC_key (integrity)
• client_write_key / server_write_key (encryption)
• client_write_IV / server_write_IV (initialization vectors)

Practical: Real-World Implementation

PRF Formula:

master_secret = PRF(pre_master_secret, "master secret",
                     ClientHello.random + ServerHello.random)[0..47]

Decrypt TLS with Wireshark (for debugging):

# Set SSLKEYLOGFILE environment variable
export SSLKEYLOGFILE=/tmp/keys.log

# In Wireshark: Edit > Preferences > Protocols > TLS
# Set (Pre)-Master-Secret log filename to /tmp/keys.log

Reality: Security Risks

Risk	Impact	Mitigation
RSA Pre-Master	If RSA key stolen, decrypt all past sessions	Use ECDHE (ephemeral keys)
Weak Random	Predictable random = predictable keys	Use cryptographic RNG (e.g., /dev/urandom)
Key Logging	SSLKEYLOGFILE exposes all session keys	Disable in production, audit access

3. Certificate Trust Model

Theory: How It Works

X.509 certificates bind a public key to an identity (domain name). The trust model uses a hierarchy of Certificate Authorities (CAs).

Certificate Chain (3 Levels):

1. Root CA Certificate: Self-signed, pre-installed in browsers/OS trust store
2. Intermediate CA Certificate: Signed by Root CA, signs server certificates
3. Server (Leaf) Certificate: Contains server public key, domain name, validity period

Certificate Validation Steps:

1. Verify signature chain up to trusted root
2. Check certificate not expired (validity period)
3. Verify domain name matches (CN or SAN)
4. Check revocation status (CRL or OCSP)

Practical: Real-World Implementation

Certificate Types:

Type	Validation	Use Case
Self-Signed	None (Issuer = Subject)	Testing, dev only
DV	Domain ownership (HTTP/DNS challenge)	Basic websites
OV	Organization identity verified	Business sites
EV	Extended verification of legal entity	Banks, e-commerce

OpenSSL Certificate Commands:

# View certificate details
openssl x509 -in cert.pem -text -noout

# Verify certificate chain
openssl verify -CAfile ca-bundle.crt cert.pem

# Check certificate expiry
openssl x509 -in cert.pem -noout -dates

# Generate self-signed certificate (testing only)
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

Reality: Certificate Attacks

Risk	Real Incident	Defense
Rogue CA	DigiNotar 2011, fake Google certs	Certificate Transparency (CT)
CA Compromise	Comodo 2011, fraudulent certs issued	CAA DNS records, CT monitoring
Misissued Cert	Symantec 2017, trust removed	CT logs, browser policies
Expired Cert	Ericsson 2018, 32M users offline	Automated renewal (Let’s Encrypt)
Self-Signed Prod	Users trained to click through warnings	Never use self-signed in production

4. Wireshark Packet Analysis

TLS 1.2 with Self-Signed Certificate

When analyzing a TLS 1.2 connection with a self-signed certificate in Wireshark:

• Wireshark Filter: tls.handshake.type == 11
• Visible Packets: Client Hello, Server Hello, Certificate, Server Key Exchange, Client Key Exchange, Finished
• Certificate Details: Issuer = Subject (self-signed indicator), no CA chain, full cert visible in plaintext
• Browser Behavior: Security warning: “Your connection is not private” (NET::ERR_CERT_AUTHORITY_INVALID)

Key Observations:

• Expand the Certificate packet to show Issuer = Subject
• Full certificate details are readable by any network observer
• Negotiated cipher visible (e.g., TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384)
• Encrypted communication but NO identity verification

TLS 1.2 with CA-Issued Certificate

• Wireshark Filter: tls.handshake.type == 11
• Visible Packets: Client Hello, Server Hello, Certificate (with chain), Server Key Exchange, Finished
• Certificate Details: Full chain visible: Server Cert → R3 (Intermediate) → ISRG Root X1
• Browser Behavior: Padlock icon, no warnings. Trusted connection

Key Observations:

• Issuer field shows trusted CA (e.g., “CN=R3, O=Let’s Encrypt”)
• Subject field shows actual domain
• Certificate chain transmitted IN PLAINTEXT (privacy concern)

5. TLS 1.2 Security Considerations

Limitation	Impact
Certificate in plaintext	Passive observers see which site you visit
Forward secrecy optional	Must explicitly choose ECDHE cipher suites
Legacy cipher support	RC4, 3DES, CBC modes still available
2-RTT handshake	Higher latency than TLS 1.3
Complex cipher negotiation	37+ cipher suites, misconfiguration risk

6. Best Practices for TLS 1.2 Deployment

• Use only ECDHE cipher suites for forward secrecy
• Disable RSA key exchange entirely
• Use only AEAD ciphers (AES-GCM, ChaCha20-Poly1305)
• Disable CBC mode (padding oracle vulnerabilities)
• Use strong key sizes (RSA 2048+, ECDSA P-256+)
• Enable HSTS to prevent downgrade attacks
• Enable OCSP Stapling for efficient revocation checking
• Plan migration to TLS 1.3 where client compatibility allows

Key Takeaways

• TLS 1.2 uses 2-RTT handshake, slower than TLS 1.3
• Certificate is transmitted in PLAINTEXT, a privacy concern
• Forward secrecy requires ECDHE and is not automatic
• Key derivation uses PRF with HMAC-SHA256
• Disable CBC, RSA key exchange, and legacy ciphers
• Use CA-issued certificates, never self-signed in production

---

Previous: Article 1: Cryptography Fundamentals

Next: Article 3: TLS 1.3: The Modern Standard

This is part of our 6-article series on Understanding TLS Security.

We test TLS configuration as part of every Web Application Pentest and IoT Pentest. Weak ciphers, protocol downgrade risks, and certificate issues are caught during our assessment. Get a free security snapshot or view pricing.