7.2 net/http包使用及工作原理

一个web应用从客户端（浏览器）发起请求（request）到服务端（服务器）,服务端从HTTP Request中提取请求路径（URL）并找到对应的处理程序（Handler）处理请求，最后返回结果。以下讲解为http服务端实现。

http包的运行机制

服务端的几个概念

• Request：用户请求的信息，用来解析用户的请求信息，包括post，get，Cookie，url等信息。

• Response:服务器需要反馈给客户端的信息。

• Conn：用户的每次请求链接。

• Handle:处理请求和生成返回信息的处理逻辑。

Go实现web服务的流程

• 创建Listen Socket，监听指定的端口，等待客户端请求到来。
• Listen Socket接受客户端的请求，得到Client Socket，接下来通过Client Socket与客户端通信。
• 处理客户端请求，首先从Client Socket读取HTTP请求的协议头，如果是POST方法，还可能要读取客户端提交的数据，然后交给相应的handler处理请求，handler处理完，将数据通过Client Socket返回给客户端。

http 执行流程

源码分析（net/http 源码 或本地 src/net/http/server.go）

Go 语言中处理 HTTP 请求主要跟两个东西相关：ServeMux 和 Handler。

ServrMux 本质上是一个 HTTP 请求路由器（或者叫多路复用器，Multiplexor）。它把收到的请求与一组预先定义的 URL 路径列表做对比，然后在匹配到路径的时候调用关联的处理器（Handler）。

处理器（Handler）负责输出HTTP响应的头和正文。任何满足了http.Handler接口的对象都可作为一个处理器。通俗的说，对象只要有个如下签名的ServeHTTP方法即可：

一、注册路由

1、先调用 http.HandleFunc(“/“, sayHelloHandler)，方法如下：

func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
    DefaultServeMux.HandleFunc(pattern, handler)
}

2、使用默认 ServeMux

// HandleFunc registers the handler function for the given pattern.
func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
   mux.Handle(pattern, HandlerFunc(handler))
}

3、注册路由策略 DefaultServeMux

// Handle registers the handler for the given pattern.
// If a handler already exists for pattern, Handle panics.
func (mux *ServeMux) Handle(pattern string, handler Handler) {
    mux.mu.Lock()//涉及并发，需要枷锁
    defer mux.mu.Unlock()
    if pattern == "" {
        panic("http: invalid pattern")
    }
    if handler == nil {
        panic("http: nil handler")
    }
    if _, exist := mux.m[pattern]; exist {
        panic("http: multiple registrations for " + pattern)
    }
    if mux.m == nil {
        mux.m = make(map[string]muxEntry)
    }
    //pattern   url 匹配正则  
    mux.m[pattern] = muxEntry{h: handler, pattern: pattern}
    if pattern[0] != '/' {
        mux.hosts = true
    }
}

二、服务监听

1、调用 http.ListenAndServe(“:8080”, nil) 监听 ,方法代码如下：

// ListenAndServe always returns a non-nil error.
func ListenAndServe(addr string, handler Handler) error {
   server := &Server{Addr: addr, Handler: handler}
   return server.ListenAndServe()
}

创建一个 Server 对象，并调用 Server 的 ListenAndServe()

2、监听tcp端口 , server.ListenAndServe 对应代码：

// ListenAndServe listens on the TCP network address srv.Addr and then
// calls Serve to handle requests on incoming connections.
// Accepted connections are configured to enable TCP keep-alives.
// If srv.Addr is blank, ":http" is used.
// ListenAndServe always returns a non-nil error.
func (srv *Server) ListenAndServe() error {
   addr := srv.Addr
   if addr == "" {
      addr = ":http"
   }
   ln, err := net.Listen("tcp", addr)
   if err != nil {
      return err
   }
   return srv.Serve(tcpKeepAliveListener{ln.(*net.TCPListener)})
}

3、接收请求，srv.Serve（）对应代码：

// Serve always returns a non-nil error. After Shutdown or Close, the
// returned error is ErrServerClosed.
func (srv *Server) Serve(l net.Listener) error {
   defer l.Close()
   if fn := testHookServerServe; fn != nil {
      fn(srv, l)
   }
   var tempDelay time.Duration // how long to sleep on accept failure
   if err := srv.setupHTTP2_Serve(); err != nil {
      return err
   }
   srv.trackListener(l, true)
   defer srv.trackListener(l, false)
   baseCtx := context.Background() // base is always background, per Issue 16220
   ctx := context.WithValue(baseCtx, ServerContextKey, srv)
   for {
      rw, e := l.Accept() //接收请求
      if e != nil {
         select {
         case <-srv.getDoneChan():
            return ErrServerClosed
         default:
         }
         if ne, ok := e.(net.Error); ok && ne.Temporary() {
            if tempDelay == 0 {
               tempDelay = 5 * time.Millisecond
            } else {
               tempDelay *= 2
            }
            if max := 1 * time.Second; tempDelay > max {
               tempDelay = max
            }
            srv.logf("http: Accept error: %v; retrying in %v", e, tempDelay)
            time.Sleep(tempDelay)
            continue
         }
         return e
      }
      tempDelay = 0
      c := srv.newConn(rw)  创建 *conn
      c.setState(c.rwc, StateNew) // before Serve can return
      go c.serve(ctx)   //新启一个goroutine，将请求数据做为参数传给 conn，由这个新的goroutine 来处理这次请求
   }
}

Go为了实现高并发和高性能, 使用了goroutines来处理Conn的读写事件, 这样每个请求都能保持独立，相互不会阻塞，可以高效的响应网络事件。

4、goroutine 处理请求

// Serve a new connection.
func (c *conn) serve(ctx context.Context) {
   c.remoteAddr = c.rwc.RemoteAddr().String()
   ctx = context.WithValue(ctx, LocalAddrContextKey, c.rwc.LocalAddr())
   defer func() {
      if err := recover(); err != nil && err != ErrAbortHandler {
         const size = 64 << 10
         buf := make([]byte, size)
         buf = buf[:runtime.Stack(buf, false)]
         c.server.logf("http: panic serving %v: %v\n%s", c.remoteAddr, err, buf)
      }
      if !c.hijacked() {
         c.close()
         c.setState(c.rwc, StateClosed)
      }
   }()
   if tlsConn, ok := c.rwc.(*tls.Conn); ok {
      if d := c.server.ReadTimeout; d != 0 {
         c.rwc.SetReadDeadline(time.Now().Add(d))
      }
      if d := c.server.WriteTimeout; d != 0 {
         c.rwc.SetWriteDeadline(time.Now().Add(d))
      }
      if err := tlsConn.Handshake(); err != nil {
         c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
         return
      }
      c.tlsState = new(tls.ConnectionState)
      *c.tlsState = tlsConn.ConnectionState()
      if proto := c.tlsState.NegotiatedProtocol; validNPN(proto) {
         if fn := c.server.TLSNextProto[proto]; fn != nil {
            h := initNPNRequest{tlsConn, serverHandler{c.server}}
            fn(c.server, tlsConn, h)
         }
         return
      }
   }
   // HTTP/1.x from here on.
   ctx, cancelCtx := context.WithCancel(ctx)
   c.cancelCtx = cancelCtx
   defer cancelCtx()
   c.r = &connReader{conn: c}
   c.bufr = newBufioReader(c.r)
   c.bufw = newBufioWriterSize(checkConnErrorWriter{c}, 4<<10)
   for {
      w, err := c.readRequest(ctx)
      if c.r.remain != c.server.initialReadLimitSize() {
         // If we read any bytes off the wire, we're active.
         c.setState(c.rwc, StateActive)
      }
      if err != nil {
         const errorHeaders = "\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n"
         if err == errTooLarge {
            // Their HTTP client may or may not be
            // able to read this if we're
            // responding to them and hanging up
            // while they're still writing their
            // request. Undefined behavior.
            const publicErr = "431 Request Header Fields Too Large"
            fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
            c.closeWriteAndWait()
            return
         }
         if isCommonNetReadError(err) {
            return // don't reply
         }
         publicErr := "400 Bad Request"
         if v, ok := err.(badRequestError); ok {
            publicErr = publicErr + ": " + string(v)
         }
         fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
         return
      }
      // Expect 100 Continue support
      req := w.req
      if req.expectsContinue() {
         if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
            // Wrap the Body reader with one that replies on the connection
            req.Body = &expectContinueReader{readCloser: req.Body, resp: w}
         }
      } else if req.Header.get("Expect") != "" {
         w.sendExpectationFailed()
         return
      }
      c.curReq.Store(w)
      if requestBodyRemains(req.Body) {
         registerOnHitEOF(req.Body, w.conn.r.startBackgroundRead)
      } else {
         if w.conn.bufr.Buffered() > 0 {
            w.conn.r.closeNotifyFromPipelinedRequest()
         }
         w.conn.r.startBackgroundRead()
      }
      // HTTP cannot have multiple simultaneous active requests.[*]
      // Until the server replies to this request, it can't read another,
      // so we might as well run the handler in this goroutine.
      // [*] Not strictly true: HTTP pipelining. We could let them all process
      // in parallel even if their responses need to be serialized.
      // But we're not going to implement HTTP pipelining because it
      // was never deployed in the wild and the answer is HTTP/2.
      serverHandler{c.server}.ServeHTTP(w, w.req)    //处理请求
      w.cancelCtx()
      if c.hijacked() {
         return
      }
      w.finishRequest()
      if !w.shouldReuseConnection() {
         if w.requestBodyLimitHit || w.closedRequestBodyEarly() {
            c.closeWriteAndWait()
         }
         return
      }
      c.setState(c.rwc, StateIdle)
      c.curReq.Store((*response)(nil))
      if !w.conn.server.doKeepAlives() {
         // We're in shutdown mode. We might've replied
         // to the user without "Connection: close" and
         // they might think they can send another
         // request, but such is life with HTTP/1.1.
         return
      }
      if d := c.server.idleTimeout(); d != 0 {
         c.rwc.SetReadDeadline(time.Now().Add(d))
         if _, err := c.bufr.Peek(4); err != nil {
            return
         }
      }
      c.rwc.SetReadDeadline(time.Time{})
   }
}

5、处理请求 ， serverHandler{c.server}.ServeHTTP(w, w.req)对应代码：

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
   handler := sh.srv.Handler
   if handler == nil {
      handler = DefaultServeMux
   }
   if req.RequestURI == "*" && req.Method == "OPTIONS" {
      handler = globalOptionsHandler{}
   }
   handler.ServeHTTP(rw, req)
}

6、handler.ServeHTTP(rw, req)

// ServeHTTP dispatches the request to the handler whose
// pattern most closely matches the request URL.
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
   if r.RequestURI == "*" {
      if r.ProtoAtLeast(1, 1) {
         w.Header().Set("Connection", "close")
      }
      w.WriteHeader(StatusBadRequest)
      return
   }
   h, _ := mux.Handler(r)
   h.ServeHTTP(w, r)
}

7、执行处理

// The HandlerFunc type is an adapter to allow the use of
// ordinary functions as HTTP handlers. If f is a function
// with the appropriate signature, HandlerFunc(f) is a
// Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)
// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
   f(w, r)
}

三、 Handler 函数接受的两个参数：http.Request 和 http.ResponseWriter

1、http.ResponseWriter

ResponseWriter 是一个接口，定义了三个方法：

• Header()：返回一个 Header 对象，可以通过它的 Set() 方法设置头部，注意最终返回的头部信息可能和你写进去的不完全相同，因为后续处理还可能修改头部的值（比如设置 Content-Length、Content-type 等操作）
• Write()： 写 response 的主体部分，比如 html 或者 json 的内容就是放到这里的
• WriteHeader()：设置 status code，如果没有调用这个函数，默认设置为 http.StatusOK， 就是 200 状态码

// A ResponseWriter interface is used by an HTTP handler to
// construct an HTTP response.
//
// A ResponseWriter may not be used after the Handler.ServeHTTP method
// has returned.
type ResponseWriter interface {
   // Header returns the header map that will be sent by
   // WriteHeader. The Header map also is the mechanism with which
   // Handlers can set HTTP trailers.
   //
   // Changing the header map after a call to WriteHeader (or
   // Write) has no effect unless the modified headers are
   // trailers.
   //
   // There are two ways to set Trailers. The preferred way is to
   // predeclare in the headers which trailers you will later
   // send by setting the "Trailer" header to the names of the
   // trailer keys which will come later. In this case, those
   // keys of the Header map are treated as if they were
   // trailers. See the example. The second way, for trailer
   // keys not known to the Handler until after the first Write,
   // is to prefix the Header map keys with the TrailerPrefix
   // constant value. See TrailerPrefix.
   //
   // To suppress implicit response headers (such as "Date"), set
   // their value to nil.
   Header() Header
   // Write writes the data to the connection as part of an HTTP reply.
   //
   // If WriteHeader has not yet been called, Write calls
   // WriteHeader(http.StatusOK) before writing the data. If the Header
   // does not contain a Content-Type line, Write adds a Content-Type set
   // to the result of passing the initial 512 bytes of written data to
   // DetectContentType.
   //
   // Depending on the HTTP protocol version and the client, calling
   // Write or WriteHeader may prevent future reads on the
   // Request.Body. For HTTP/1.x requests, handlers should read any
   // needed request body data before writing the response. Once the
   // headers have been flushed (due to either an explicit Flusher.Flush
   // call or writing enough data to trigger a flush), the request body
   // may be unavailable. For HTTP/2 requests, the Go HTTP server permits
   // handlers to continue to read the request body while concurrently
   // writing the response. However, such behavior may not be supported
   // by all HTTP/2 clients. Handlers should read before writing if
   // possible to maximize compatibility.
   Write([]byte) (int, error)
   // WriteHeader sends an HTTP response header with the provided
   // status code.
   //
   // If WriteHeader is not called explicitly, the first call to Write
   // will trigger an implicit WriteHeader(http.StatusOK).
   // Thus explicit calls to WriteHeader are mainly used to
   // send error codes.
   //
   // The provided code must be a valid HTTP 1xx-5xx status code.
   // Only one header may be written. Go does not currently
   // support sending user-defined 1xx informational headers,
   // with the exception of 100-continue response header that the
   // Server sends automatically when the Request.Body is read.
   WriteHeader(statusCode int)
}

在response中是可以看到

func (w *response) Header() Header
func (w *response) WriteHeader(code ``int``)
func (w *response) Write(data []``byte``) (n ``int``, err error)

所以说response实现了ResponseWriter接口。

2、Request

Request 就是封装好的客户端请求，包括 URL，method，header 等等所有信息，以及一些方便使用的方法：

在源代码 src/net/http/request.go

// The field semantics differ slightly between client and server
// usage. In addition to the notes on the fields below, see the
// documentation for Request.Write and RoundTripper.
type Request struct {
   // Method specifies the HTTP method (GET, POST, PUT, etc.).
   // For client requests an empty string means GET.
   //
   // Go's HTTP client does not support sending a request with
   // the CONNECT method. See the documentation on Transport for
   // details.
   Method string
   // URL specifies either the URI being requested (for server
   // requests) or the URL to access (for client requests).
   //
   // For server requests the URL is parsed from the URI
   // supplied on the Request-Line as stored in RequestURI.  For
   // most requests, fields other than Path and RawQuery will be
   // empty. (See RFC 2616, Section 5.1.2)
   //
   // For client requests, the URL's Host specifies the server to
   // connect to, while the Request's Host field optionally
   // specifies the Host header value to send in the HTTP
   // request.
   URL *url.URL
   // The protocol version for incoming server requests.
   //
   // For client requests these fields are ignored. The HTTP
   // client code always uses either HTTP/1.1 or HTTP/2.
   // See the docs on Transport for details.
   Proto      string // "HTTP/1.0"
   ProtoMajor int    // 1
   ProtoMinor int    // 0
   // Header contains the request header fields either received
   // by the server or to be sent by the client.
   //
   // If a server received a request with header lines,
   //
   // Host: example.com
   // accept-encoding: gzip, deflate
   // Accept-Language: en-us
   // fOO: Bar
   // foo: two
   //
   // then
   //
   // Header = map[string][]string{
   //    "Accept-Encoding": {"gzip, deflate"},
   //    "Accept-Language": {"en-us"},
   //    "Foo": {"Bar", "two"},
   // }
   //
   // For incoming requests, the Host header is promoted to the
   // Request.Host field and removed from the Header map.
   //
   // HTTP defines that header names are case-insensitive. The
   // request parser implements this by using CanonicalHeaderKey,
   // making the first character and any characters following a
   // hyphen uppercase and the rest lowercase.
   //
   // For client requests, certain headers such as Content-Length
   // and Connection are automatically written when needed and
   // values in Header may be ignored. See the documentation
   // for the Request.Write method.
   Header Header
   // Body is the request's body.
   //
   // For client requests a nil body means the request has no
   // body, such as a GET request. The HTTP Client's Transport
   // is responsible for calling the Close method.
   //
   // For server requests the Request Body is always non-nil
   // but will return EOF immediately when no body is present.
   // The Server will close the request body. The ServeHTTP
   // Handler does not need to.
   Body io.ReadCloser
   // GetBody defines an optional func to return a new copy of
   // Body. It is used for client requests when a redirect requires
   // reading the body more than once. Use of GetBody still
   // requires setting Body.
   //
   // For server requests it is unused.
   GetBody func() (io.ReadCloser, error)
   // ContentLength records the length of the associated content.
   // The value -1 indicates that the length is unknown.
   // Values >= 0 indicate that the given number of bytes may
   // be read from Body.
   // For client requests, a value of 0 with a non-nil Body is
   // also treated as unknown.
   ContentLength int64
   // TransferEncoding lists the transfer encodings from outermost to
   // innermost. An empty list denotes the "identity" encoding.
   // TransferEncoding can usually be ignored; chunked encoding is
   // automatically added and removed as necessary when sending and
   // receiving requests.
   TransferEncoding []string
   // Close indicates whether to close the connection after
   // replying to this request (for servers) or after sending this
   // request and reading its response (for clients).
   //
   // For server requests, the HTTP server handles this automatically
   // and this field is not needed by Handlers.
   //
   // For client requests, setting this field prevents re-use of
   // TCP connections between requests to the same hosts, as if
   // Transport.DisableKeepAlives were set.
   Close bool
   // For server requests Host specifies the host on which the
   // URL is sought. Per RFC 2616, this is either the value of
   // the "Host" header or the host name given in the URL itself.
   // It may be of the form "host:port". For international domain
   // names, Host may be in Punycode or Unicode form. Use
   // golang.org/x/net/idna to convert it to either format if
   // needed.
   //
   // For client requests Host optionally overrides the Host
   // header to send. If empty, the Request.Write method uses
   // the value of URL.Host. Host may contain an international
   // domain name.
   Host string
   // Form contains the parsed form data, including both the URL
   // field's query parameters and the POST or PUT form data.
   // This field is only available after ParseForm is called.
   // The HTTP client ignores Form and uses Body instead.
   Form url.Values
   // PostForm contains the parsed form data from POST, PATCH,
   // or PUT body parameters.
   //
   // This field is only available after ParseForm is called.
   // The HTTP client ignores PostForm and uses Body instead.
   PostForm url.Values
   // MultipartForm is the parsed multipart form, including file uploads.
   // This field is only available after ParseMultipartForm is called.
   // The HTTP client ignores MultipartForm and uses Body instead.
   MultipartForm *multipart.Form
   // Trailer specifies additional headers that are sent after the request
   // body.
   //
   // For server requests the Trailer map initially contains only the
   // trailer keys, with nil values. (The client declares which trailers it
   // will later send.)  While the handler is reading from Body, it must
   // not reference Trailer. After reading from Body returns EOF, Trailer
   // can be read again and will contain non-nil values, if they were sent
   // by the client.
   //
   // For client requests Trailer must be initialized to a map containing
   // the trailer keys to later send. The values may be nil or their final
   // values. The ContentLength must be 0 or -1, to send a chunked request.
   // After the HTTP request is sent the map values can be updated while
   // the request body is read. Once the body returns EOF, the caller must
   // not mutate Trailer.
   //
   // Few HTTP clients, servers, or proxies support HTTP trailers.
   Trailer Header
   // RemoteAddr allows HTTP servers and other software to record
   // the network address that sent the request, usually for
   // logging. This field is not filled in by ReadRequest and
   // has no defined format. The HTTP server in this package
   // sets RemoteAddr to an "IP:port" address before invoking a
   // handler.
   // This field is ignored by the HTTP client.
   RemoteAddr string
   // RequestURI is the unmodified Request-URI of the
   // Request-Line (RFC 2616, Section 5.1) as sent by the client
   // to a server. Usually the URL field should be used instead.
   // It is an error to set this field in an HTTP client request.
   RequestURI string
   // TLS allows HTTP servers and other software to record
   // information about the TLS connection on which the request
   // was received. This field is not filled in by ReadRequest.
   // The HTTP server in this package sets the field for
   // TLS-enabled connections before invoking a handler;
   // otherwise it leaves the field nil.
   // This field is ignored by the HTTP client.
   TLS *tls.ConnectionState
   // Cancel is an optional channel whose closure indicates that the client
   // request should be regarded as canceled. Not all implementations of
   // RoundTripper may support Cancel.
   //
   // For server requests, this field is not applicable.
   //
   // Deprecated: Use the Context and WithContext methods
   // instead. If a Request's Cancel field and context are both
   // set, it is undefined whether Cancel is respected.
   Cancel <-chan struct{}
   // Response is the redirect response which caused this request
   // to be created. This field is only populated during client
   // redirects.
   Response *Response
   // ctx is either the client or server context. It should only
   // be modified via copying the whole Request using WithContext.
   // It is unexported to prevent people from using Context wrong
   // and mutating the contexts held by callers of the same request.
   ctx context.Context
}

Handler 需要知道关于请求的任何信息，都要从这个对象中获取.

net/http包代码执行流程总结

首先调用Http.HandleFunc 按顺序做了几件事: 1 调用了DefaultServeMux的HandleFunc 2 调用了DefaultServeMux的Handle 3 往DefaultServeMux的map[string]muxEntry中增加对应的handler和路由规则

其次调用http.ListenAndServe(“:8080”, nil) 按顺序做了几件事情: 1 示例化Server 2 调用Server的ListenAndServe() 3 调用net.Listen(“tcp”, addr)监听端口 4 启动一个for循环,在循环体中Accept请求 5 对每个请求示例化一个Conn,并且开启一个goroutine为这个请求进行服务go c.serve() 6 读取每个请求的内容w, err := c.readRequest() 7 判断handler是否为空,如果没有设置handler(这个例子就没有设置handler),handler就设置为DefaultServeMux 8 调用handler的ServeHttp 9 在这个例子中,下面就进入到DefaultServeMux.ServeHttp 10 根据request选择handler,并且进入到这个handler的ServeHTTP mux.handler(r).ServeHTTP(w, r) 11 选择handler: A 判断是否有路由能满足这个request(循环遍历ServerMux的muxEntry) B 如果有路由满足,调用这个路由handler的ServeHttp C 如果没有路由满足,调用NotFoundHandler的ServeHttp

http连接处理流程图

links

• 目录
• 上一节：Go搭建一个简单web服务
• 下一节：http客户端