Rare TCP state

TCP State Machine

Normal State

Ref : TCP/IP State Transition Diagram

Rare State1

Circumstance

One-way communication

• Server only read from socket, no response.

• Client only write data into socket, ignore response.

• Server will close an connection after been idle for Xmins

Demo

func handleRequest(conn net.Conn) {
    buf := make([]byte, 1024)

    //defer conn.(*net.TCPConn).CloseWrite()  // ShutDown(SHUT_WR)
    //defer conn.(*net.TCPConn).CloseRead() // ShutDown(SHUT_RD)
    defer conn.Close()
    for {
        conn.SetReadDeadline(time.Now().Add(10 * time.Second))
        reqLen, err := conn.Read(buf)
        if err != nil {
            DoLog("Error in reading[%v]", err.Error())
            return
        } else {
            DoLog("INFO read[%v] Message[%v]", reqLen, string(buf))
        }   

        //conn.Write([]byte("Message received."))
    }   
}

Result

We got a data loss here.

Rare State1 CLOSE_WAITE

• do conn.Write() with a []byte -> it runs fine without error!

• it takes another conn.Write to get the error: broken pipe

• [Data Loss]()

what we expected :

[fin_wait2 (Server)| close_wait (Client)]()
| (DATA) <————-|———-(DATA)

Why only one data lost?

21:13:03.505439 IP 127.0.0.1.5555 > 127.0.0.1.6882: Flags [F.], seq 1, ack 4, win 256, options [nop,nop,TS val 1996918705 ecr 1996913703], length 0
21:13:03.506316 IP 127.0.0.1.6882 > 127.0.0.1.5555: Flags [.], ack 2, win 257, options [nop,nop,TS val 1996918706 ecr 1996918705], length 0
21:13:06.783940 IP 127.0.0.1.6882 > 127.0.0.1.5555: Flags [P.], seq 4:5, ack 2, win 257, options [nop,nop,TS val 1996921983 ecr 1996918705], length 1
21:13:06.783975 IP 127.0.0.1.5555 > 127.0.0.1.6882: Flags [R], seq 4031687754, win 0, length 0

Close vs. Shutdown

The normal way to terminate a network connection is to call the close function. But, there are two limitations with close that can be avoided with shutdown:

Close() terminates both directions of data transfer, reading and writing. 

Since a TCP connection is full-duplex , there are times when we want to tell the other end that we have finished sending, even though that end might have more data to send us.

 Close() only terminates socket when the fd reference is 0

close() decrements the descriptor’s reference count and closes the socket only if the count reaches 0. shutdown() breaks the connection for all processes sharing the socketid. Those who try to read will detect EOF, and those who try to write will reseive SIGPIPE,

REF shutdown Function

NOTE: A shutdown will not close a socket.

It’s important to note that shutdown() doesn’t actually close the file descriptor—it just changes its usability. To free a socket descriptor, you need to use close().

shutdown, close and linger

The effect of an setsockopt(…, SO_LINGER,…) depends on what the values in the linger structure (the third parameter passed to setsockopt()) are:

Case 1:  linger->l_onoff is zero (linger->l_linger has no meaning): 

This is the default.
On close(), the underlying stack attempts to gracefully shutdown the connection after ensuring all unsent data is sent. In the case of connection-oriented protocols such as TCP, the stack also ensures that sent data is acknowledged by the peer. The stack will perform the above-mentioned graceful shutdown in the background (after the call to close() returns), regardless of whether the socket is blocking or non-blocking.

Case 2: linger->l_onoff is non-zero and linger->l_linger is zero:

A close() returns immediately. The underlying stack discards any unsent data, and, in the case of connection-oriented protocols such as TCP, sends a RST (reset) to the peer (this is termed a hard or abortive close). All subsequent attempts by the peer’s application to read()/recv() data will result in an ECONNRESET.

Case 3: linger->l_onoff is non-zero and linger->l_linger is non-zero:

A close() will either block (if a blocking socket) or fail with EWOULDBLOCK (if non-blocking) until a graceful shutdown completes or the time specified in linger->l_linger elapses (time-out). Upon time-out the stack behaves as in case 2 above.

How to get one-way communication:

CloseWrite (Shutdown) instead of Close()
Demo

REF close linger
REF Writing to a closed, local TCP socket not failing
REF When should I use shutdown()
REF Beej’s Guide to Network Programming
REF close vs shutdown socket?
REF The ultimate SO_LINGER page, or: why is my tcp not reliable

Go语言TCP Socket编程
socket链接的关闭close和shutdown的区别_TIME_WAIT和CLOSE_WAIT什么时刻出现_如何处理
socket关闭: close()和shutdown()的差异

How to prevent the data loss

• Check Connection Closed before write.

• Socket Read in golang is block (underlying socket in Go Runtime is Non-block socket + epool)
  The Go scheduler

func CheckFdCloseWait(conn *net.TCPConn) (flag bool) {
    fileDesc, errFile := conn.File()
    if nil != errFile {
        return false
    }   
    msg := make([]byte, 0)
    nRead, _, err := syscall.Recvfrom(int(fileDesc.Fd()), msg, syscall.MSG_DONTWAIT)

    DoLog("CheckFdCloseWait nRead[%v] err[%v]", nRead, err)
    if nil == err && nRead == 0 { 
        return true
    }   
    return false
}

DEMO