8.1 Sockets

8.1 Sockets

Some network application developers say that the lower application layers are all about socket programming. This may not be true for all cases, but many modern web applications do indeed use sockets to their advantage. Have you ever wondered how browsers communicate with web servers when you are surfing the internet? Or How MSN connects you and your friends together in a chatroom, relaying each message in real-time? Many services like these use sockets to transfer data. As you can see, sockets occupy an important position in network programming today, and we’re going to learn about using sockets in Go in this section.

What is a socket？

Sockets originate from Unix, and given the basic “everything is a file” philosophy of Unix, everything can be operated on with “open -> write/read -> close”. Sockets are one implementation of this philosophy. Sockets have a function call for opening a socket just like you would open a file. This returns an int descriptor of the socket which can then be used for operations like creating connections, transferring data, etc.

Two types of sockets that are commonly used are stream sockets (SOCK_STREAM) and datagram sockets (SOCK_DGRAM). Stream sockets are connection-oriented like TCP, while datagram sockets do not establish connections, like UDP.

Socket communication

Before we understand how sockets communicate with one another, we need to figure out how to make sure that every socket is unique, otherwise establishing a reliable communication channel is already out of the question. We can give every process a unique PID which serves our purpose locally, however that’s not able to work over a network. Fortunately, TCP/IP helps us solve this problem. The IP addresses of the network layer are unique in a network of hosts, and “protocol + port” is also unique among host applications. So, we can use these principles to make sockets which are unique.

Figure 8.1 network protocol layers

Applications that are based on TCP/IP all use socket APIs in their code in one way or another. Given that networked applications are becoming more and more prevalent in the modern day, it’s no wonder some developers are saying that “everything is about sockets”.

Socket basic knowledge

We know that sockets have two types, which are TCP sockets and UDP sockets. TCP and UDP are protocols and, as mentioned, we also need an IP address and port number to have a unique socket.

IPv4

The global internet uses TCP/IP as its protocol, where IP is the network layer and a core part of TCP/IP. IPv4 signifies that its version is 4; infrastructure development to date has spanned over 30 years.

The number of bits in an IPv4 address is 32, which means that 2^32 devices are able to uniquely connect to the internet. Due to the rapid develop of the internet, IP addresses are already running out of stock in recent years.

Address format:127.0.0.1, 172.122.121.111.

IPv6

IPv6 is the next version or next generation of the internet. It’s being developed for solving many of the problems inherent with IPv4. Devices using IPv6 have an address that’s 128 bits long, so we’ll never need to worry about a shortage of unique addresses. To put this into perspective, you could have more than 1000 IP addresses for every square meter on earth with IPv6. Other problems like peer to peer connection, service quality (QoS), security, multiple broadcast, etc., are also be improved.

Address format: 2002:c0e8:82e7:0:0:0:c0e8:82e7.

IP types in Go

The net package in Go provides many types, functions and methods for network programming. The definition of IP as follows:

type IP []byte

Function ParseIP(s string) IP is to convert the IPv4 or IPv6 format to an IP:

package main
import (
    "net"
    "os"
    "fmt"
)
func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s ip-addr\n", os.Args[0])
        os.Exit(1)
    }
    name := os.Args[1]
    addr := net.ParseIP(name)
    if addr == nil {
        fmt.Println("Invalid address")
    } else {
        fmt.Println("The address is ", addr.String())
    }
    os.Exit(0)
}

It returns the corresponding IP format for a given IP address.

TCP socket

What can we do when we know how to visit a web service through a network port? As a client, we can send a request to an appointed network port and gets its response; as a server, we need to bind a service to an appointed network port, wait for clients’ requests and supply a response.

In Go’s net package, there’s a type called TCPConn that facilitates this kind of clients/servers interaction. This type has two key functions:

func (c *TCPConn) Write(b []byte) (n int, err os.Error)
func (c *TCPConn) Read(b []byte) (n int, err os.Error)

TCPConn can be used by either client or server for reading and writing data.

We also need a TCPAddr to represent TCP address information:

type TCPAddr struct {
    IP IP
    Port int
}

We use the ResolveTCPAddr function to get a TCPAddr in Go:

func ResolveTCPAddr(net, addr string) (*TCPAddr, os.Error)

Arguments of net can be one of “tcp4”, “tcp6” or “tcp”, which each signify IPv4-only, IPv6-only, and either IPv4 or IPv6, respectively.
addr can be a domain name or IP address, like “www.google.com:80” or “127.0.0.1:22”.

TCP client

Go clients use the DialTCP function in the net package to create a TCP connection, which returns a TCPConn object; after a connection is established, the server has the same type of connection object for the current connection, and client and server can begin exchanging data with one another. In general, clients send requests to servers through a TCPConn and receive information from the server response; servers read and parse client requests, then return feedback. This connection will remain valid until either the client or server closes it. The function for creating a connection is as follows:

func DialTCP(net string, laddr, raddr *TCPAddr) (c *TCPConn, err os.Error)

Arguments of net can be one of “tcp4”, “tcp6” or “tcp”, which each signify IPv4-only, IPv6-only, and either IPv4 or IPv6, respectively.
laddr represents the local address, set it to nil in most cases.
raddr represents the remote address.

Let’s write a simple example to simulate a client requesting a connection to a server based on an HTTP request. We need a simple HTTP request header:

"HEAD / HTTP/1.0\r\n\r\n"

Server response information format may look like the following:

HTTP/1.0 200 OK
ETag: "-9985996"
Last-Modified: Thu, 25 Mar 2010 17:51:10 GMT
Content-Length: 18074
Connection: close
Date: Sat, 28 Aug 2010 00:43:48 GMT
Server: lighttpd/1.4.23

Client code:

package main
import (
    "fmt"
    "io/ioutil"
    "net"
    "os"
)
func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s host:port ", os.Args[0])
        os.Exit(1)
    }
    service := os.Args[1]
    tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
    checkError(err)
    conn, err := net.DialTCP("tcp", nil, tcpAddr)
    checkError(err)
    _, err = conn.Write([]byte("HEAD / HTTP/1.0\r\n\r\n"))
    checkError(err)
    result, err := ioutil.ReadAll(conn)
    checkError(err)
    fmt.Println(string(result))
    os.Exit(0)
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
        os.Exit(1)
    }
}

In the above example, we use user input as the service argument of net.ResolveTCPAddr to get a tcpAddr. Passing tcpAddr to the DialTCP function, we create a TCP connection, conn. We can then use conn to send request information to the server. Finally, we use ioutil.ReadAll to read all the content from conn, which contains the server response.

TCP server

We have a TCP client now. We can also use the net package to write a TCP server. On the server side, we need to bind our service to a specific inactive port and listen for any incoming client requests.

func ListenTCP(net string, laddr *TCPAddr) (l *TCPListener, err os.Error)
func (l *TCPListener) Accept() (c Conn, err os.Error)

The arguments required here are identical to those required by the DialTCP function we used earlier. Let’s implement a time syncing service using port 7777:

package main
import (
    "fmt"
    "net"
    "os"
    "time"
)
func main() {
    service := ":7777"
    tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
    checkError(err)
    listener, err := net.ListenTCP("tcp", tcpAddr)
    checkError(err)
    for {
        conn, err := listener.Accept()
        if err != nil {
            continue
        }
        daytime := time.Now().String()
        conn.Write([]byte(daytime)) // don't care about return value
        conn.Close()                // we're finished with this client
    }
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
        os.Exit(1)
    }
}

After the service is started, it waits for client requests. When it receives a client request, it Accepts it and returns a response to the client containing information about the current time. It’s worth noting that when errors occur in the for loop, the service continues running instead of exiting. Instead of crashing, the server will record the error to a server error log.

The above code is still not good enough, however. We didn’t make use of goroutines, which would have allowed us to accept simultaneous requests. Let’s do this now:

package main
import (
    "fmt"
    "net"
    "os"
    "time"
)
func main() {
    service := ":1200"
    tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
    checkError(err)
    listener, err := net.ListenTCP("tcp", tcpAddr)
    checkError(err)
    for {
        conn, err := listener.Accept()
        if err != nil {
            continue
        }
        go handleClient(conn)
    }
}
func handleClient(conn net.Conn) {
    defer conn.Close()
    daytime := time.Now().String()
    conn.Write([]byte(daytime)) // don't care about return value
    // we're finished with this client
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
        os.Exit(1)
    }
}

By separating out our business process from the handleClient function, and by using the go keyword, we’ve already implemented concurrency in our service. This is a good demonstration of the power and simplicity of goroutines.

Some of you may be thinking the following: this server does not do anything meaningful. What if we needed to send multiple requests for different time formats over a single connection? How would we do that?

package main
import (
    "fmt"
    "net"
    "os"
    "time"
    "strconv"
)
func main() {
    service := ":1200"
    tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
    checkError(err)
    listener, err := net.ListenTCP("tcp", tcpAddr)
    checkError(err)
    for {
        conn, err := listener.Accept()
        if err != nil {
            continue
        }
        go handleClient(conn)
    }
}
func handleClient(conn net.Conn) {
    conn.SetReadDeadline(time.Now().Add(2 * time.Minute)) // set 2 minutes timeout
    request := make([]byte, 128) // set maximum request length to 128B to prevent flood based attacks
    defer conn.Close()  // close connection before exit
    for {
        read_len, err := conn.Read(request)
        if err != nil {
            fmt.Println(err)
            break
        }
        if read_len == 0 {
            break // connection already closed by client
        } else if string(request[:read_len]) == "timestamp" {
            daytime := strconv.FormatInt(time.Now().Unix(), 10)
            conn.Write([]byte(daytime))
        } else {
            daytime := time.Now().String()
            conn.Write([]byte(daytime))
        }
    }
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
        os.Exit(1)
    }
}

In this example, we use conn.Read() to constantly read client requests. We cannot close the connection because clients may issue more than one request. Due to the timeout we set using conn.SetReadDeadline(), the connection closes automatically after our allotted time period. When the expiry time has elapsed, our program breaks from the for loop. Notice that request needs to be created with a max size limitation in order to prevent flood attacks.

Controlling TCP connections

Controlling TCP functions:

func DialTimeout(net, addr string, timeout time.Duration) (Conn, error)

Setting the timeout of connections. These are suitable for use on both clients and servers:

func (c *TCPConn) SetReadDeadline(t time.Time) error
func (c *TCPConn) SetWriteDeadline(t time.Time) error

Setting the write/read timeout of one connection:

func (c *TCPConn) SetKeepAlive(keepalive bool) os.Error

It’s worth taking some time to think about how long you want your connection timeouts to be. Long connections can reduce the amount of overhead involved in creating connections and are good for applications that need to exchange data frequently.

For more detailed information, just look up the official documentation for Go’s net package .

UDP sockets

The only difference between a UDP socket and a TCP socket is the processing method for dealing with multiple requests on server side. This arises from the fact that UDP does not have a function like Accept. All of the other functions have UDP counterparts; just replace TCP with UDP in the functions mentioned above.

func ResolveUDPAddr(net, addr string) (*UDPAddr, os.Error)
func DialUDP(net string, laddr, raddr *UDPAddr) (c *UDPConn, err os.Error)
func ListenUDP(net string, laddr *UDPAddr) (c *UDPConn, err os.Error)
func (c *UDPConn) ReadFromUDP(b []byte) (n int, addr *UDPAddr, err os.Error
func (c *UDPConn) WriteToUDP(b []byte, addr *UDPAddr) (n int, err os.Error)

UDP client code sample:

package main
import (
    "fmt"
    "net"
    "os"
)
func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s host:port", os.Args[0])
        os.Exit(1)
    }
    service := os.Args[1]
    udpAddr, err := net.ResolveUDPAddr("udp4", service)
    checkError(err)
    conn, err := net.DialUDP("udp", nil, udpAddr)
    checkError(err)
    _, err = conn.Write([]byte("anything"))
    checkError(err)
    var buf [512]byte
    n, err := conn.Read(buf[0:])
    checkError(err)
    fmt.Println(string(buf[0:n]))
    os.Exit(0)
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error ", err.Error())
        os.Exit(1)
    }
}

UDP server code sample:

package main
import (
    "fmt"
    "net"
    "os"
    "time"
)
func main() {
    service := ":1200"
    udpAddr, err := net.ResolveUDPAddr("udp4", service)
    checkError(err)
    conn, err := net.ListenUDP("udp", udpAddr)
    checkError(err)
    for {
        handleClient(conn)
    }
}
func handleClient(conn *net.UDPConn) {
    var buf [512]byte
    _, addr, err := conn.ReadFromUDP(buf[0:])
    if err != nil {
        return
    }
    daytime := time.Now().String()
    conn.WriteToUDP([]byte(daytime), addr)
}
func checkError(err error) {
    if err != nil {
        fmt.Fprintf(os.Stderr, "Fatal error ", err.Error())
        os.Exit(1)
    }
}

Summary

Through describing and coding some simple programs using TCP and UDP sockets, we can see that Go provides excellent support for socket programming, and that they are fun and easy to use. Go also provides many functions for building high performance socket applications.

8.1. Sockets