In this post I will discuss TCP SYN attacks, and how to some extent SCTP protocol is safe from these attacks, I will discuss this primarily from Linux perspective.
What are TCP SYN attacks?
TCP is a connection oriented protocol, In order to establish connection, TCP uses three-way handshake mechanism, i.e client and the server exchange three messages before a connection is established properly.
Client Server SYN -----------------------> SYN, ACK <------------------------ ACK ------------------------>
Server after receiving SYN, from client will sending the SYN-ACK message, and keeps this half established connection in a queue. Once clients responds with a ACK message this half established connection will be moved into full established state, and passed into socket buffer from where user process can use accept() system call to get the connection.
Suppose if the client don’t send ACK then, the connection is not properly established, and server can either retry by sending SYN-ACK again or it might free the connection form the queue after timeout.
If the attacker sends a flood a SYN requests, and keeps quiet by not responding with ACK’s, after receiving SYN-ACK, then the queue in server will be filled up by these half open connections. As the connection queue is full with requests generated by attacker, legitimate users are denied from establishing connection. This is a form of denial of service attack.
What is the queue size in Linux ?
In Linux the backlog queue is controlled by sysctl command and the value passed to listen() system call.
[lamadala@Cent net]$ sysctl -a |grep backlog net.core.netdev_max_backlog = 1000 net.ipv4.tcp_max_syn_backlog = 512 [lamadala@Cent net]$
On my machine the backlog queue size is 512, When a user process calls listen() for a server program the kernel will create a queue to store the backlog connections.
listen() -> inet_listen() -> inet_csk_listen_start() -> reqsk_queue_alloc()
reqsk_queue_alloc(struct request_sock_queue *queue, unsigned int nr_table_entries)
{
................................
nr_table_entries = min_t(u32, nr_table_entries, sysctl_max_syn_backlog);
nr_table_entries = max_t(u32, nr_table_entries, 8);
nr_table_entries = roundup_pow_of_two(nr_table_entries + 1);
lopt_size += nr_table_entries * sizeof(struct request_sock *);
...................................
See that the queue size is derived from the system level config (sysctl) and the user arguments passed to listen(). As you can see that the size of queue is finite, suppose if the attacker is able to fill the queue with invalid requests, then legitimate users are denied.
What are the solutions?
One of the solutions Linux came up to solve this issue is, to reserve one quarter of backlog buffer for half open connections from known party. If the connection request is legitimate, then the client will surely respond with a ACK message, this makes the peer known.
Only a attacker won’t be responding with ACK messages, Hence if the peer is not known then use only three quarters of queue, This way known peers can always stay connected to the server and the attacker can only fill 75% of backlog queue.
else if (!sysctl_tcp_syncookies &&
(sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
(sysctl_max_syn_backlog >> 2)) &&
!tcp_peer_is_proven(req, dst, false)) {
LIMIT_NETDEBUG(KERN_DEBUG pr_fmt("drop open request from %pI4/%u\n"),
&saddr, ntohs(tcp_hdr(skb)->source));
goto drop_and_release;
}
Observe that the connection request is dropped and is NOT saved in the backlog queue if the peer is unknown and the queue is three fourth full.
Another solution is to use the concept of cookies, Generate a cookie using the most basic data comprising the connection state; Use this cookie as a sequence number in SYN-ACK message, A legitimate connection request will respond back with an ACK segment, then the basic TCB data can be regenerated.
if (want_cookie) {
isn = cookie_v4_init_sequence(sk, skb, &req->mss);
req->cookie_ts = tmp_opt.tstamp_ok;
}
One major problem with this approach is, we won’t be able to regenerate the SYN-ACK’s when there is a timeout as no state information is stored.
This looks like a good Idea, any protocol using cookies?
Similar to TCP there is one more protocol called Stream Control Transmission Protocol (SCTP), which is also connection oriented protocol. Connection is established by way of a four-way handshake, and cookies are used to protect the connection queue in the server.
Client Server INIT -----------------------> INIT-ACK(Cookie) <------------------------ Cookie-Echo ------------------------> Cookie-Ack <------------------------
Client sends a Init message to server, Server responds with a Init-Ack containing the cookie, at this stage server doesn’t retain any client information. When the client responds with a Cookie-Echo message, only then the server will construct a TCB block in memory for the request and responds back with Cookie-Ack.
Is it completely safe from buffer attacks?
The problem with three-way handshake mechanism is, before establishing the validity of connection request from client, a TCB is created for half-open connection on the queue by the server, Exploiting this loop-hole a attacker can fill the queue.
The four-way handshake method offers us the possibility of delaying TCB creation until the client validity is established. The Cookie-Echo message from client at-least confirms us validity of client IP address.
The important point to note here is, in four-way handshake mechanism by the time TCB is created, kernel knows that it is a valid IP and the control can be immediately given to user process, which access the socket using accept() system call. When user process finds no data at the socket it can immediately close the socket and free the buffer.
In three-way handshake, TCB is created for a half-open connection, as the socket is not completely established, control can’t be given to user process to decide on closing or retaining the connection. This way the kernel has to decide on the number of backlog connections in the queue and this can cause issues.
Are we completely safe? No, we are not completely safe, because the attacker may still choose not to respond after receiving Cookie-Ack from server, This can still fill server buffers; But we are saved by the fact that decision on closing or retaining a socket can be passed to user process immediately.
So the concept of cookies can be of great help in ensuring that only a legitimate client is allowed to establish a connection, and the user process can immediately act on the request either by accepting or closing the socket.
My Linux and Telecom Experiences 