tip

The complete microservice features and related components are provided starting from version v2.4.

Introduction

The GoFrame framework supports microservice mode development, providing commonly used microservice components, development tools, and development tutorials to help teams quickly transition to microservices.

Simple Example

The GoFrame microservice components feature low coupling and generic design, supporting most microservice communication protocols. In the official documentation, we use examples of HTTP and GRPC protocols to introduce microservice development and the use of component tools. Since HTTP Web development has a relatively rich and comprehensive independent chapter introduction, most of the microservice chapter is introduced with a focus on GRPC.

HTTP Microservice Example

https://github.com/gogf/gf/tree/master/example/registry/file

server.go

package main
import (
    "github.com/gogf/gf/contrib/registry/file/v2"
    "github.com/gogf/gf/v2/frame/g"
    "github.com/gogf/gf/v2/net/ghttp"
    "github.com/gogf/gf/v2/net/gsvc"
    "github.com/gogf/gf/v2/os/gfile"
)
func main() {
    gsvc.SetRegistry(file.New(gfile.Temp("gsvc")))
    s := g.Server(`hello.svc`)
    s.BindHandler("/", func(r *ghttp.Request) {
        g.Log().Info(r.Context(), `request received`)
        r.Response.Write(`Hello world`)
    })
    s.Run()
}

As you can see, an HTTP microservice end and a regular Web Server end have no significant differences, but there is an additional line of code at the top:

gsvc.SetRegistry(file.New(gfile.Temp("gsvc")))

This line of code is used to enable and register the registration and discovery component used by the current service. In this example, file.New(gfile.Temp("gsvc")) is a service registration and discovery component based on local system files, where gfile.Temp("gsvc") specifies the path to store service files, for example, in Linux/MacOS systems, it points to the /tmp/gsvc directory. File system-based registration discovery is only used for local microservice examples and cannot be used for cross-node communication. In production environments, we often use other service registration and discovery components, such as etcd, polaris, zookeeper, etc. The community components of the framework already provide implementations of commonly used service registration and discovery components.

Secondly, in this example, we set a name hello.svc for the Server, which represents the name of the microservice bound by this Server. The service name serves as a unique identifier for microservices, used for identification communication between services. When the service registration component registration is enabled, the HTTP Server will register its access address to the service registration component at runtime, making it easier for other services to access it through the same component by service name.

client.go

package main
import (
    "time"
    "github.com/gogf/gf/contrib/registry/file/v2"
    "github.com/gogf/gf/v2/frame/g"
    "github.com/gogf/gf/v2/net/gsvc"
    "github.com/gogf/gf/v2/os/gctx"
    "github.com/gogf/gf/v2/os/gfile"
)
func main() {
    gsvc.SetRegistry(file.New(gfile.Temp("gsvc")))
    client := g.Client()
    for i := 0; i < 10; i++ {
        ctx := gctx.New()
        res, err := client.Get(ctx, `http://hello.svc/`)
        if err != nil {
            panic(err)
        }
        g.Log().Debug(ctx, res.ReadAllString())
        res.Close()
        time.Sleep(time.Second)
    }
}

The client creates an HTTP Client through g.Client() and accesses the server through the address http://hello.svc/, where hello.svc is the microservice name bound by the Server end previously. When the client accesses through the microservice name, the service registration and discovery component will perform retrieval at the underlying level and find the corresponding server address for communication.

Execution Results

First, run the server.go server to run a simple service, and then execute client.go to request the service by service name.

After execution, the client outputs:

$ go run client.go
2023-03-14 20:22:10.006 [DEBU] {8054f3a48c484c1760fb416bb3df20a4} Hello world
2023-03-14 20:22:11.007 [DEBU] {6831cae08c484c1761fb416b9d4df851} Hello world
2023-03-14 20:22:12.008 [DEBU] {9035761c8d484c1762fb416b1e648b81} Hello world
2023-03-14 20:22:13.011 [DEBU] {a05a32588d484c1763fb416bc19ff667} Hello world
2023-03-14 20:22:14.012 [DEBU] {40fdea938d484c1764fb416b8459fc43} Hello world
2023-03-14 20:22:15.014 [DEBU] {686c9acf8d484c1765fb416b3697d369} Hello world
2023-03-14 20:22:16.015 [DEBU] {906a470b8e484c1766fb416b85b9867e} Hello world
2023-03-14 20:22:17.017 [DEBU] {28c7fd468e484c1767fb416b86e5557f} Hello world
2023-03-14 20:22:18.018 [DEBU] {90d2ad828e484c1768fb416bfcde738f} Hello world
2023-03-14 20:22:19.019 [DEBU] {d05559be8e484c1769fb416baad06f23} Hello world

The server outputs:

$ go run server.go
2023-03-14 20:20:06.364 [INFO] pid[96421]: http server started listening on [:61589]
2023-03-14 20:20:06.364 [INFO] openapi specification is disabled
2023-03-14 20:20:06.364 [DEBU] service register: &{Head: Deployment: Namespace: Name:hello.svc Version: Endpoints:10.35.12.81:61589 Metadata:map[insecure:true protocol:http]}
   SERVER   | DOMAIN  | ADDRESS | METHOD | ROUTE |                             HANDLER                             |    MIDDLEWARE
------------|---------|---------|--------|-------|-----------------------------------------------------------------|--------------------
  hello.svc | default | :61589  | ALL    | /     | main.main.func1                                                 |
------------|---------|---------|--------|-------|-----------------------------------------------------------------|--------------------
  hello.svc | default | :61589  | ALL    | /*    | github.com/gogf/gf/v2/net/ghttp.internalMiddlewareServerTracing | GLOBAL MIDDLEWARE
------------|---------|---------|--------|-------|-----------------------------------------------------------------|--------------------
2023-03-14 20:22:10.006 [INFO] {8054f3a48c484c1760fb416bb3df20a4} request received
2023-03-14 20:22:11.007 [INFO] {6831cae08c484c1761fb416b9d4df851} request received
2023-03-14 20:22:12.008 [INFO] {9035761c8d484c1762fb416b1e648b81} request received
2023-03-14 20:22:13.010 [INFO] {a05a32588d484c1763fb416bc19ff667} request received
2023-03-14 20:22:14.012 [INFO] {40fdea938d484c1764fb416b8459fc43} request received
2023-03-14 20:22:15.013 [INFO] {686c9acf8d484c1765fb416b3697d369} request received
2023-03-14 20:22:16.015 [INFO] {906a470b8e484c1766fb416b85b9867e} request received
2023-03-14 20:22:17.016 [INFO] {28c7fd468e484c1767fb416b86e5557f} request received
2023-03-14 20:22:18.017 [INFO] {90d2ad828e484c1768fb416bfcde738f} request received
2023-03-14 20:22:19.019 [INFO] {d05559be8e484c1769fb416baad06f23} request received

GRPC Microservice Example

https://github.com/gogf/gf/tree/master/example/rpc/grpcx/basic

helloworld.proto

The main difference between grpc and http protocols is that grpc requires protobuf to define API interfaces and data structures.

syntax = "proto3";
package protobuf;
option go_package = "github.com/gogf/gf/grpc/example/helloworld/protobuf";
// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}
// The response message containing the greetings
message HelloReply {
  string message = 1;
}

The above protobuf file is compiled with the following command (please install the protoc tool in advance):

gf gen pb

It will generate the corresponding proto go data structure file and grpc interface file:

helloworld.pb.go
helloworld_grpc.pb.go

controller.go

The controller is used to implement the interface methods defined in proto (if using the framework’s standardized engineering directory structure, this controller code file is also automatically generated by the framework’s gf gen pb tool, and developers only need to fill in the corresponding method’s specific implementation):

type Controller struct {
    protobuf.UnimplementedGreeterServer
}
func Register(s *grpcx.GrpcServer) {
    protobuf.RegisterGreeterServer(s.Server, &Controller{})
}
// SayHello implements helloworld.GreeterServer
func (s *Controller) SayHello(ctx context.Context, in *protobuf.HelloRequest) (*protobuf.HelloReply, error) {
    return &protobuf.HelloReply{Message: "Hello " + in.GetName()}, nil
}

config.yaml

The server configuration file specifies that the service’s name is demo. The microservice name is used as the unique identification mark for service communication. When the server’s listening port is not explicitly specified, the server will randomly listen on an available local port. In microservice mode, since communication is conducted using the service name, the server port often does not need to be explicitly specified, and random listening is sufficient.

grpc:
  name:            "demo"
  logPath:          "./log"
  logStdout:        true
  errorLogEnabled:  true
  accessLogEnabled: true
  errorStack:       true

server.go

The grpc server does not explicitly specify the service registration and discovery component used by the server by default uses the system file registration discovery component, which is only used for single-machine testing. The controller.Register call registers the specific interface implementation into the server via the controller registration method generated by our tool.

package main
import (
    "github.com/gogf/gf/contrib/rpc/grpcx/v2"
    "github.com/gogf/gf/example/rpc/grpcx/basic/controller"
)
func main() {
    s := grpcx.Server.New()
    controller.Register(s)
    s.Run()
}

client.go

The grpc client needs to specify the specific name of the server service when creating the connection. Here, the server service name is demo, which is the microservice name mentioned above. When the client does not explicitly specify the service registration and discovery component used, it defaults to the system file registration discovery component for single-machine testing.

package main
import (
    "github.com/gogf/gf/contrib/rpc/grpcx/v2"
    "github.com/gogf/gf/example/rpc/grpcx/basic/protobuf"
    "github.com/gogf/gf/v2/frame/g"
    "github.com/gogf/gf/v2/os/gctx"
)
func main() {
    var (
        ctx    = gctx.New()
        conn   = grpcx.Client.MustNewGrpcClientConn("demo")
        client = protobuf.NewGreeterClient(conn)
    )
    res, err := client.SayHello(ctx, &protobuf.HelloRequest{Name: "World"})
    if err != nil {
        g.Log().Error(ctx, err)
        return
    }
    g.Log().Debug(ctx, "Response:", res.Message)
}

Execution Results

The server output: You can see the server output some DEBU debug information to indicate some details of the service registration. At the same time, because the server’s listening port was not explicitly specified, a local port 64517 was randomly listened on here.

$ go run server.go
2023-03-14 20:50:58.465 [DEBU] set default registry using file registry as no custom registry set
2023-03-14 20:50:58.466 [DEBU] service register: &{Head: Deployment: Namespace: Name:demo Version: Endpoints:10.35.12.81:64517 Metadata:map[protocol:grpc]}
2023-03-14 20:50:58.466 [INFO] pid[98982]: grpc server started listening on [:64517]
2023-03-14 20:52:37.059 {9898c809364a4c17da79e47f3e6c3b8f} /protobuf.Greeter/SayHello, 0.003ms, name:"World", message:"Hello World"

The client output: The client accessed through the microservice name and received a return from the server. Note that in the client’s log and the server’s log, the chain tracing TraceID is the same (9898c809364a4c17da79e47f3e6c3b8f), indicating logs generated from the same request. The GoFrame microservice feature enables traceability capabilities by default.

$ go run client.go
2023-03-14 20:52:37.060 [DEBU] {9898c809364a4c17da79e47f3e6c3b8f} Response: Hello World

Documentation

📄️ PreparationPrepare the environment for microservices development, mainly including the installation guide of GRPC protocol dependencies and GoFrame framework CLI tools. Learners need to ensure the installation of GRPC related tools and master the basic application of GRPC protocol in the microservices development of the GoFrame framework.

📄️ Project StructureStandard project directory structure for microservices development using the GoFrame framework, including management of protocol and interface files. It describes in detail how to use the GoFrame framework’s development tools to generate protobuf files corresponding to database table structures, and how to compile protocol files to generate interfaces and controllers. Additionally, it explains the specific steps for service startup and interface implementation, and introduces methods for tag injection and data validation plugins.

📄️ Scaffold ComponentsThe scaffold module in the GoFrame framework provides convenient implementations of GRPC server and client, including features like context, load balancing, and service discovery. With the grpcx component, users can easily build and manage microservice applications, supporting a variety of load balancing strategies and etcd as a registry center, simplifying inter-service communication and data transfer.

📄️ GRPC Server ConfigurationServer configuration methods, including how to read and manage configuration files through the GoFrame Framework. Provides a complete configuration template example covering service name, service listening address, log storage directory, error log recording, and access log recording settings. The configuration aligns with the framework’s automatic reading logic, ensuring convenient service deployment and efficient log management, as well as how to set up and use parameter log component configurations for independent grpc server log management.

📄️ GRPC Interceptor ComponentMethods for using GRPC interceptors in the GoFrame framework, detailing the implementation of server and client interceptors, including features such as link tracing, error handling, and automatic error validation. By using the grpcx component, users can flexibly add and customize interceptors to enhance the scalability and stability of GRPC services and clients.

📄️ Service Registry and DiscoveryThe use and implementation of the service registry and discovery components within the GoFrame framework, managed by the gsvc component, supports multiple implementations such as etcd, zookeeper, and polaris. Proper configuration enables global service registry and discovery, enhancing the flexibility and scalability of service calls, which is an important guide for developing microservice architectures.

📄️ Service Load BalancingThe load balancing component in the GoFrame framework, the gsel component, provides multiple built-in load balancing strategies including LeastConnection, Random, RoundRobin, and Weight. Developers can choose the appropriate strategy based on their needs or implement a custom one, with practical application examples of load balancing strategies shown through HTTP and GRPC.

📄️ Service ConfigurationUsing the configuration management component in the GoFrame framework, it supports various third-party configuration centers like Polaris, Apollo, Nacos, Consul, etc., through decoupled design. Detailed code is provided to showcase how to initialize and enable the Polaris configuration client, including usage examples and error handling methods.