Simple Go Microservice Architecture to begin your journey with Go!

Part 1

I started learning Go a year ago as I wanted to develop a microservice for Kugelblitz for providing various financial services to their clients in go which was clean and simple to understand with not much complexity in the structure or data flow of the service. And after researching through many online articles I came up with a structure which in my opinion is simple enough to understand. I am gonna divide this article into parts to explain each segment that I have added to the architecture. I will be discussing making connections with relational and non-relational databases, adding an encryption layer to jwt authentication, connecting with AWS services like S3, and so on as we progress in this series.

Let's dive in without wasting any more time!

Pre-Requisites

1. go version 1.11 or higher installed

Setup

1. Create a folder for your go project

$ mkdir product

$ cd product

2. Enable dependency for tracking your code

$ go mod init api/product

Note :In order to add any go module, just add the code and import and then run `go mod tidy`.

Project Structure

The project structure is going to be fairly simple!

Project-

--------Auth

--------Data

--------Handlers

--------Main.go

Now let's dive into the explanations and content of these folders!

Auth

Here we will add files regarding our authentication for the service. For the scope of this article, we will add the logic for JWT authentication and encryption logic will be added here later in the series.

Now create a token.go file and add the following code

package auth

import (
	"fmt"
	"time"

	jwt "github.com/dgrijalva/jwt-go"
)

var mySigningKey = []byte("4zx8u44md43f696217b41rn969fac8c1")

func GetJWT(audience string) (map[string]string, error) {
	token := jwt.New(jwt.SigningMethodHS256)

	claims := token.Claims.(jwt.MapClaims)

	claims["authorized"] = true
	claims["client"] = "access"
	claims["aud"] = audience
	claims["iss"] = "jwtgo.io"
	claims["exp"] = time.Now().Add(time.Hour * 1).Unix()

	tokenString, err := token.SignedString(mySigningKey)

	if err != nil {
		fmt.Errorf("Something Went Wrong: %s", err.Error())
		return nil, err
	}

	refreshToken := jwt.New(jwt.SigningMethodHS256)
	rtClaims := refreshToken.Claims.(jwt.MapClaims)
	rtClaims["aud"] = audience
	rtClaims["client"] = "refresh"
	rtClaims["exp"] = time.Now().Add(time.Hour * 24).Unix()

	rt, err := refreshToken.SignedString(mySigningKey)
	if err != nil {
		fmt.Errorf("Something Went Wrong: %s", err.Error())
		return nil, err
	}

	return map[string]string{
		"access_token":  tokenString,
		"refresh_token": rt,
	}, nil
}
        

Here we are generating access and refresh tokens with claims like audience, issuer, and expiry for the token.

Data

Here we will add all our data models and all the operation related to them.

Let's look at the example, create a file products.go

package data

import (
	"encoding/json"
	"fmt"
	"io"
	"regexp"
	"time"

	"github.com/go-playground/validator"
)

//Product defines the structure for an api product
type Product struct {
	// the id for the product
	//
	// required: false
	// min: 1
	ID int `json:"id"`
	// the name for this poduct
	//
	// required: true
	// max length: 255
	Name string `json:"name" validate:"required"`
	// the description for this poduct
	//
	// required: false
	// max length: 10000
	Description string `json:"description"`
	// the price for the product
	//
	// required: true
	// min: 0.01
	Price float32 `json:"price" validate:"gt=0"`
	// the SKU for the product
	//
	// required: true
	// pattern: [a-z]+-[a-z]+-[a-z]+
	SKU       string `json:"sku" validate:"required,sku"`
	CreatedOn string `json:"_"`
	UpdatedOn string `json:"_"`
	DeletedOn string `json:"_"`
}        

Here we have a structure of a Product which we will be playing with.

Now let's add validations to the product and logic to convert the struct to json or the opposite,

func (p *Product) Validate() error {
	validate := validator.New()
	validate.RegisterValidation("sku", validateSKU)

	return validate.Struct(p)
}

func validateSKU(fl validator.FieldLevel) bool {
	//sku is of format abc-sdf-fvdv
	re := regexp.MustCompile(`[a-z]+-[a-z]+-[a-z]+`)
	matches := re.FindAllString(fl.Field().String(), -1)

	if len(matches) != 1 {
		return false
	}

	return true
}

func (p *Product) ToJSON(w io.Writer) error {
	e := json.NewEncoder(w)
	return e.Encode(p)
}

func (p *Product) FromJSON(r io.Reader) error {
	e := json.NewDecoder(r)
	return e.Decode(p)
}

Now add some dummy data to play with,

var productList = []*Product{
	&Product{
		ID:          1,
		Name:        "Latte",
		Description: "Frothy milky cofee",
		Price:       2.45,
		SKU:         "abc323",
		CreatedOn:   time.Now().UTC().String(),
		UpdatedOn:   time.Now().UTC().String(),
	},
	&Product{
		ID:          2,
		Name:        "Espresso",
		Description: "Short and strong cofee without milk",
		Price:       1.99,
		SKU:         "fjd34",
		CreatedOn:   time.Now().UTC().String(),
		UpdatedOn:   time.Now().UTC().String(),
	},
}        

So now let's write some operation logic for the data,

Get Product:

var ErrProductNotFound = fmt.Errorf("Product not found")

func FindProduct(id int) (*Product, int, error) {
	for i, p := range productList {
		if p.ID == id {
			return p, i, nil
		}
	}

	return nil, -1, ErrProductNotFound
}        

Add Product:

func getNextId() int {
	lp := productList[len(productList)-1]
	return lp.ID + 1
}

func AddProducts(p *Product) {
	p.ID = getNextId()
	productList = append(productList, p)
}        

Update Product:

func UpdateProduct(id int, p *Product) error {
	_, pos, err := FindProduct(id)
	if err != nil {
		return err
	}

	p.ID = id
	productList[pos] = p
	return nil
}        

Delete Product:

func DeleteProduct(id int) error {
	_, i, err := FindProduct(id)

	if err != nil {
		return err
	}

	if i == -1 {
		return ErrProductNotFound
	}

	productList = append(productList[:i], productList[i+1])

	return nil
}        

Handlers

As the name suggests, here you will write your API request handlers.

Now Let's create handlers in order to manage the above data and its operations

create a file handlers.go

Here we will create a Product wrapper with the go lang's default logger

type Products struct {
	l *log.Logger
}

//NewProducts creates a product handler with the given logger
func NewProducts(l *log.Logger) *Products {
	return &Products{l}
}

type KeyProduct struct{}        

Now, create another file and name it middleware.go

Here we will add the middleware logic where we will validate and sanitize the data coming in the request.

func (p *Products) MiddlewareProductValidation(next http.Handler) http.Handler {
	return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
		prod := data.Product{}

		err := prod.FromJSON(r.Body)
		if err != nil {
			http.Error(rw, "Unable to unmarshal json", http.StatusBadRequest)
			return
		}

		//validate product
		err = prod.Validate()
		if err != nil {
			http.Error(
				rw,
				fmt.Sprintf("Error validating product: %s", err),
				http.StatusBadRequest,
			)
			return
		}

		ctx := context.WithValue(r.Context(), KeyProduct{}, prod)
		req := r.WithContext(ctx)

		next.ServeHTTP(rw, req)
	})
}        

The Above code will take the request body and will try to parse the body into the product structure that we have defined in the data package.

And after successful parsing, it will validate the data based on the conditions we have written in product tags.

Now, we will add our JWT implementation here too as every request needs to go through the same security check.

var MySigningKey = []byte("4zx8u44md43f696217b41rn969fac8c1")
func IsAuthorized(endpoint func(http.ResponseWriter, *http.Request)) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.Header["Token"] != nil {

			clientId := r.Header["Client_id"][0]

			jtoken := r.Header["Token"][0]

			token, err := jwt.Parse(jtoken, func(token *jwt.Token) (interface{}, error) {
				if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
					return nil, fmt.Errorf(("Invalid Signing Method"))
				}
				aud := clientId
				checkAudience := token.Claims.(jwt.MapClaims).VerifyAudience(aud, true)
				if !checkAudience {
					return nil, fmt.Errorf(("invalid aud"))
				}
				// verify iss claim
				iss := "jwtgo.io"
				checkIss := token.Claims.(jwt.MapClaims).VerifyIssuer(iss, true)
				if !checkIss {
					return nil, fmt.Errorf(("invalid issuer"))
				}

				// verify token access
				client := "access"
				checkClient := token.Claims.(jwt.MapClaims)["client"].(string)
				if client != checkClient {
					return nil, fmt.Errorf(("invalid access"))
				}

				return MySigningKey, nil
			})
			if err != nil {
				fmt.Fprintf(w, err.Error())
				return
			}

			if token.Valid {
				endpoint(w, r)
			}
		} else {
			fmt.Fprintf(w, "No Authorization Token provided")
			return
		}
	})
}        

It's a pretty simple boilerplate code for verifying the jwt token present in the request header.

And then we will add the handler for JWT tokens

func (p *Products) GetAdminToken(rw http.ResponseWriter, r *http.Request) {

	p.l.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent(), "Token")

	validToken, err := auth.GetJWT(r,Header["Audience"][0])
	if err != nil {
		p.l.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent(), http.StatusInternalServerError, "Could not generate token")
		return
	}

	tokenKeys, err := json.Marshal(validToken)
	if err != nil {
		p.l.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent(), http.StatusInternalServerError, err.Error())
		return
	}
	p.l.Println(r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent(), http.StatusOK, "OK")
	rw.Header().Add("Content-Type", "application/json")
	rw.WriteHeader(http.StatusOK)
	rw.Write(tokenKeys)
}
        

The above handler will return access and refresh tokens as a response.

Now everything is set for all the handlers related to your product API requests, create a file product.go

Here I am just adding Get and Post, I am sure you will get the gist of it.

// Create handles POST requests to add new products
func (p *Products) AddProducts(rw http.ResponseWriter, r *http.Request) {
	p.l.Println("Handle POST Products")

	prod := r.Context().Value(KeyProduct{}).(data.Product)

	p.l.Println(&prod)
	data.AddProducts(&prod)
}

// ListSingle handles GET requests
func (p *Products) ListSingle(rw http.ResponseWriter, r *http.Request) {

	vars := mux.Vars(r)
	id, err := strconv.Atoi(vars["id"])
	if err != nil {
		http.Error(rw, "Unable to covert id", http.StatusBadRequest)
		return
	}

	p.l.Println("[DEBUG] get record id", id)

	prod, _, err := data.FindProduct(id)

	switch err {
	case nil:

	case data.ErrProductNotFound:
		p.l.Println("[ERROR] fetching product", err)

		rw.WriteHeader(http.StatusNotFound)
		return
	default:
		p.l.Println("[ERROR] fetching product", err)

		rw.WriteHeader(http.StatusInternalServerError)
		return
	}

	err = prod.ToJSON(rw)
	if err != nil {
		p.l.Println("[ERROR] serializing product", err)
	}
}        

Till now we have finished the data, its handlers, request authentication, now all we have to do is set up the server and manage the endpoints,

which we will be doing in main.go.

Add the following code in your main.go

package main

import (
	"context"
	"log"
	"net/http"
	"os"
	"os/signal"
	"time"

	"github.com/go-openapi/runtime/middleware"
	"github.com/vermavashish/try/handlers"

	"github.com/gorilla/mux"
)

func main() {

	l := log.New(os.Stdout, "product-api", log.LstdFlags)

	//Create the handlers
	ph := handlers.NewProducts(l)

	//create a new server mux and register the handlers
	sm := mux.NewRouter()

	getRouter := sm.Methods("GET").Subrouter()
	getRouter.Handle("/product/{id:[0-9]+}", handlers.IsAuthorized(ph.ListSingle))

	postRouter := sm.Methods("POST").Subrouter()
	postRouter.Handle("/", handlers.IsAuthorized(ph.AddProducts))
	postRouter.Use(ph.MiddlewareProductValidation)

	//create a new server
	s := &http.Server{
		Addr:         ":9000",
		Handler:      sm,
		ErrorLog:     l,
		IdleTimeout:  120 * time.Second,
		ReadTimeout:  5 * time.Second,
		WriteTimeout: 10 * time.Second,
	}

	//Start the server
	go func() {
		err := s.ListenAndServe()
		if err != nil {
			l.Fatal(err)
		}
	}()

	//trap the sigterm or interupt and gracefully shutdown the server
	sigChan := make(chan os.Signal)
	signal.Notify(sigChan, os.Interrupt)
	signal.Notify(sigChan, os.Kill)

	sig := <-sigChan

	l.Println("Received Terminate, Graceful Shutdown", sig)

	tc, _ := context.WithTimeout(context.Background(), 30*time.Second)
	s.Shutdown(tc)
}        

Here We have created an instance for our logger, and then we have configured our endpoints with their respective handlers and middlewares.

And finally, we are done with our small demonstration of the go structure.

I started with this very same architecture and slowly was able to build a microservice with a logging system, connection with Postgres and MongoDB database, encryption layer over Jwt token, and other stuff which I will be sharing in the upcoming part.

Hope this will be useful for you!

This content was originally published on Kugelblitz's Blog.