Chapter 4 · Network Security· Intermediate· ~16 min

Networking Refresher

Down a layer: TCP flags and state, congestion control, NAT mechanics, and BGP's unsecured trust model.

You will learn

  • The TCP three-way handshake and the flags that matter for security
  • How SYN cookies defeat SYN flood at the cost of a few features
  • What NAT actually does at the packet level, and why it confuses logging
  • Why BGP is a security disaster waiting to happen, and what RPKI is

The three-way handshake

Every TCP connection starts with three packets:

  1. Client to server: SYN with initial sequence number A.
  2. Server to client: SYN+ACK with sequence number B, acknowledging A+1. Server allocates a TCB (Transmission Control Block) at this point.
  3. Client to server: ACK with B+1. Connection established.

Three security-relevant TCP flags worth knowing: SYN (new connection), RST (abort), FIN (graceful close). A packet with no flags set is illegal but used in scanning. A SYN flood spams step 1 to consume server TCB memory.

SYN cookies

The classic SYN flood defense: instead of allocating a TCB at SYN, the server encodes the connection state into the initial sequence number it returns. The client's final ACK echoes that sequence number back; the server decodes it and only then allocates state. Memory pressure goes to zero. Trade-off: TCP options (window scaling, timestamps, SACK) cannot be remembered through the cookie, so very high-performance flows take a hit. Most servers use cookies only when SYN queue depth crosses a threshold.

Congestion control, briefly

TCP throttles itself to avoid filling the queue. Two parameters matter:

  • cwnd (congestion window): how many unacknowledged bytes the sender will allow.
  • rwnd (receive window): how many bytes the receiver has buffer for.

Classic algorithms (Reno, CUBIC, BBR) differ in how cwnd grows and how it reacts to loss. From a security standpoint, the relevant fact is that congestion control reacts to loss as a signal of congestion. Selective ACK loss can be weaponized to slow down a victim (Slowloris-style attacks on the transport layer).

NAT at the packet level

NAT (Network Address Translation) rewrites packet headers as they cross the boundary. The most common form, NAPT (Network Address and Port Translation, sometimes called PAT or "IP masquerading"):

  • Outbound packet from 10.0.0.42:51234 to 8.8.8.8:443 becomes 203.0.113.5:51234 to 8.8.8.8:443. Router records the mapping.
  • Inbound reply from 8.8.8.8:443 to 203.0.113.5:51234 is rewritten back to 10.0.0.42:51234 using the table.
  • Port can be remapped if two internal hosts use the same source port.

Security implications: logging at the public edge sees only the public IP and the remapped port. Tying back to the internal host requires correlating with the NAT table at that moment. Most consumer routers do not log NAT mappings.

BGP and the trust assumption

The Border Gateway Protocol carries internet routing announcements between autonomous systems (ASes). An AS announces "I have a path to 198.51.100.0/24" and neighbors propagate it. Other ASes pick the best path by policy (typically shortest AS-path with tie-breakers).

The original BGP has no authentication. A malicious or misconfigured AS can announce any prefix it likes, including ones that belong to someone else. Real incidents:

  • Pakistan Telecom's 2008 announcement of YouTube's /24 took YouTube offline globally for hours.
  • The 2018 hijack of Amazon Route 53 redirected MyEtherWallet traffic and stole crypto.
  • Repeated leaks via misconfiguration where a small ISP accidentally advertises routes for the entire internet.

The defense in slow deployment is RPKI (Resource Public Key Infrastructure): cryptographically signed assertions of which AS may originate which prefix. Adoption is patchy. Newer schemes (BGPsec, ASPA) add path validation but are even less deployed. The pragmatic posture today is to publish ROAs for your prefixes, monitor for unexpected announcements with tools like RIPEstat or BGPmon, and not assume anyone else is doing this right.

Where security lives at each layer

  • Link (Ethernet/Wi-Fi): WPA2/3, 802.1X, MAC filtering (weak), port security.
  • Internet (IP): IPsec (in tunnels). Otherwise the layer is famously unauthenticated.
  • Transport (TCP/UDP): TLS for TCP, DTLS or QUIC for UDP, both add their own auth and crypto.
  • Application (HTTP, DNS, etc.): per-protocol auth.

Most production security work happens at link, transport, and application. The internet layer is mostly left as best-effort, with security pushed up.

Take the quiz Back to the beginner version