Go Re-Usability - Interface, Generic, Reflection, Dynamic Type, and Code Generation
Jingshun Chen
Senior Fullstack Software Engineer - asp.net Core, Kafka, MongoDB, SqlServer, Postgres, React
In Go programming, maximizing the usability of functions often involves exploring various techniques and strategies. This post delves into several approaches to enhance the versatility and flexibility of functions, covering the utilization of interfaces, generics, reflection, dynamic types, and code generation. By adopting these methodologies, developers can create more adaptable and scalable code that caters to a broader range of use cases.
Interface
Suppose we aim to create a function to sort posts. We can achieve this by implementing the following code snippet:
package main
import "sort"
type Post struct {
ID int
Title string
}
func SortPost(entities []Post) {
sort.Slice(entities, func(i, j int) bool {
return entities[i].ID < entities[j].ID
})
}
Later, if we introduce another entity, such as a user, that also requires sorting, a simplistic approach would involve duplicating the SortPost() function as SortUser(). However, such duplication makes code maintenance cumbersome. Furthermore, if we decide to switch to a more optimized sorting algorithm, we'd need to update every instance of the Sort*** functions.
A more sophisticated approach involves leveraging Go's interfaces. Initially, our sorting function can accept an array of an interface instead of a specific struct array.
type IDable interface {
GetID() int
}
func Sort(entities []IDable) {
sort.Slice(entities, func(i, j int) bool {
return entities[i].GetID() < entities[j].GetID()
})
}
Subsequently, both the Post and User structs need to implement the GetID() method. This method can be placed in a BaseEntity struct and then embedded into the Post and User structs.
type BaseEntity struct { ID int }
func (b BaseEntity) GetID() int { return b.ID }
type Post struct {
BaseEntity
Title string
}
type User struct {
BaseEntity
Name string
}
The entire code is provided below:
package main
import (
"fmt"
"sort"
)
type BaseEntity struct { ID int }
func (b BaseEntity) GetID() int { return b.ID }
type Post struct {
BaseEntity
Title string
}
type User struct {
BaseEntity
Name string
}
type IDable interface {
GetID() int
}
func Sort(entities []IDable) {
sort.Slice(entities, func(i, j int) bool {
return entities[i].GetID() < entities[j].GetID()
})
}
func main() {
var posts []IDable
posts = append(posts, Post{
Title: "first post",
BaseEntity: BaseEntity{ID: 2},
})
posts = append(posts, Post{
Title: "second post",
BaseEntity: BaseEntity{ID: 1},
})
fmt.Println("Before sort:", posts)
Sort(posts)
fmt.Println("After sort", posts)
}
Generic
The release of Go 1.18 marks a significant milestone with the introduction of long-awaited support for generics, ushering in a new era of enhanced type safety and code flexibility. To illustrate the power of generics, consider a simple yet powerful function capable of converting any value to its pointer:
func Ptr[T any](t T) *T {
return &t
}
This utility function streamlines caller functions, making them shorter and more elegant:
func main() {
//no generic
ts := time.Now()
timePtr := &ts
//with generic
timePtr1 := Ptr(time.Now())
fmt.Println(*timePtr, *timePtr1)
}
However, the true potential of generics is unveiled with more complex applications. Take, for instance, the implementation of a TTL (time to live) Cache, leveraging generics to accommodate a wide range of data types:
package main
import (
"fmt"
"golang.org/x/exp/constraints"
"sync"
"time"
)
type TTLEntity[T any] struct {
Item T
TS time.Time
}
type TTLCache[K constraints.Ordered, T any] struct {
cache sync.Map
duration time.Duration
}
func NewTTLCache[K constraints.Ordered, T any](duration time.Duration) *TTLCache[K, T] {
return &TTLCache[K, T]{
cache: sync.Map{},
duration: duration,
}
}
func (c *TTLCache[K, T]) Set(k K, v T) {
entity := TTLEntity[T]{
Item: v,
TS: time.Now(),
}
c.cache.Swap(k, entity)
}
func (c *TTLCache[K, T]) Get(k K) (T, bool) {
var empty T
item, ok := c.cache.Load(k)
if !ok {
return empty, false
}
ttlEntity := item.(TTLEntity[T])
if ttlEntity.TS.Add(c.duration).Before(time.Now()) {
return empty, false
}
return ttlEntity.Item, true
}
type Post struct {
ID int
Title string
}
func main() {
postStore := NewTTLCache[int, Post](time.Second * 10)
p := Post{
ID: 1,
Title: "post one",
}
store.Set(p.ID, p)
fmt.Println(store.Get(p.ID))
time.Sleep(time.Second * 11)
fmt.Println(store.Get(p.ID))
}
To ensure goroutine safety, we utilize sync.Map to store objects. We attach system time to each entity, forming TTLEntity, and then store it in the map. When retrieving TTLEntity from the map, we check for expiration, only returning valid results to the caller.
This implementation leverages generics to offer type-safe caching functionality. With the ability to define generic structs, such as TTLEntity, the code abstracts away the complexities of managing TTL logic, allowing for cleaner and more maintainable code. Overall, the integration of generics in Go 1.18 empowers developers with enhanced type safety and code reusability, paving the way for more robust and scalable software solutions.
Reflection
Reflection in Go is a double-edged sword. While it provides the ability to inspect and generate code dynamically at runtime, it comes with performance overhead and sacrifices type safety. Despite these drawbacks, reflection plays a crucial role in creating versatile tools, libraries, and frameworks for introspecting and manipulating Go type.
Consider a practical scenario where we need to manage database connection settings. We define a struct, DbConnectionConfig, to hold these settings, setting default values for connection and idle limits. However, we want the flexibility to override these defaults dynamically.
Enter the MergeDefaults function. Using reflection, this function iterates over the fields of the provided newItem, copying non-empty values to the corresponding fields of the defaultItem. This dynamic merging capability enables us to manage configurations without explicit handling of each field.
Here's a glimpse of how it works:
领英推荐
func MergeDefaults[T any](defaultItem, newItem T) T {
defaultValues := reflect.ValueOf(&defaultItem).Elem()
types := reflect.TypeOf(newItem)
for i := 0; i < types.NumField(); i++ {
newValue := reflect.ValueOf(newItem).Field(i)
empty := reflect.Zero(types.Field(i).Type).Interface()
if !reflect.DeepEqual(newValue.Interface(), empty) {
targetValue := defaultValues.Field(i)
targetValue.Set(newValue)
}
}
return defaultItem
}
Using this function, we merge a customized configuration with the default one, ensuring seamless flexibility in managing database connections.
defaultConfig := DbConnectionConfig{
MaxConnection: 100,
MaxIdleConnection: 10,
}
config := MergeDefaults(defaultConfig, DbConnectionConfig{
Host: "127.0.0.1",
DB: "test",
Username: "test",
Password: "test",
})
This dynamic approach enhances code maintainability and flexibility. However, it's crucial to tread carefully, considering the performance implications and potential risks, particularly in performance-critical or security-sensitive scenarios. Reflection should be used judiciously, weighing its benefits against its costs.
Dynamic type
In scenarios where flexibility is paramount, Go's static typing might seem too restrictive. Consider a scenario where you're building a gateway to fetch data from multiple URLs and merge them dynamically. Rather than defining specific structs for each data type, we opt for a more dynamic approach using maps.
The mergedItems function exemplifies this dynamic style, where we fetch data from two APIs and merge them together based on a common identifier.
package main
import (
"encoding/json"
"fmt"
"net/http"
)
func GetAPIResult[T any](url string) (T, error) {
var empty T
// Send HTTP GET request to the API endpoint
response, err := http.Get(url)
if err != nil {
return empty, err
}
defer response.Body.Close()
// Check if the response status code is not OK (200)
if response.StatusCode != http.StatusOK {
return empty, fmt.Errorf("HTTP request failed with status code %d", response.StatusCode)
}
// Parse the JSON response body into a slice of map[string]interface{}
var results T
err = json.NewDecoder(response.Body).Decode(&results)
if err != nil {
return empty, err
}
return results, nil
}
func main() {
items, err := mergedItems(
"https://jsonplaceholder.typicode.com/posts",
"https://jsonplaceholder.typicode.com/users/%v",
"userId",
"user")
if err != nil {
panic(err)
}
if bs, err := json.Marshal(items); err != nil {
panic(err)
} else {
fmt.Println(string(bs))
}
}
type Item map[string]any
func mergedItems(mainEntityURL, lookupEntityURL, localIDField, lookupField string) ([]Item, error) {
items, err := GetAPIResult[[]Item](mainEntityURL)
if err != nil {
return nil, err
}
for _, item := range items {
localID := fmt.Sprintf("%v", item[localIDField])
url := fmt.Sprintf(lookupEntityURL, localID)
lookupEntity, err := GetAPIResult[Item](url)
if err != nil {
return nil, err
}
item[localIDField] = lookupEntity
}
return items, nil
}
By utilizing maps instead of predefined structs, we provide the flexibility to merge any two entities from different APIs together. This approach emphasizes flexibility and simplicity, enabling dynamic handling of data without the constraints of static types. However, it's crucial to weigh the benefits against the potential risks, such as decreased type safety and increased complexity.
Code Generation
Generating Go code can be very useful for tasks like creating boilerplate code, defining configuration files, or generating code based on certain specifications.
A typical use case of code generation is ORM(Object-Relational Mapping). The following code demonstrate how to build a simple ORM using Code Generation.
Our target Post ORM
package models
import (
"context"
"database/sql"
)
type Post struct{
userId float64
id float64
title string
body string
}
func (e Post) Insert(ctx context.Context, db *sql.DB) (int64, error) {
result, err := db.ExecContext(ctx,
"insert into Post (userId,id,title,body) values(?,?,?,?)",
[]any{e.userId,e.id,e.title,e.body})
if err != nil {
return 0, err
}
return result.LastInsertId()
}
To generate target Post ORM, we first define a template
var ormTemplate = `package models
import (
"context"
"database/sql"
)
type [entityName] struct{
[structFields]
}
func (e Post) Insert(ctx context.Context, db *sql.DB) (int64, error) {
result, err := db.ExecContext(ctx,
"insert into Post ([insertFields]) values([insertPlaceHolders])",
[]any{[insertParams]})
if err != nil {
return 0, err
}
return result.LastInsertId()
}
`
To fill [structFields], [insertFields], [insetPlaceHolders], [insertParams], we need to get database schema's metadata, we omit that process by simply provide a map[string]any variable, here's our generate code function.
type Item map[string]any
func GenerateORM(item Item, entityName string) error {
structFields, insertFields, insertParams, insertPlaceHolders := "", "", "", ""
for s, a := range item {
structFields += fmt.Sprintf("\n %s %s", s, reflect.TypeOf(a).Name())
insertFields += fmt.Sprintf(",%s", s)
insertParams += fmt.Sprintf(",e.%s", s)
insertPlaceHolders += ",?"
}
code := strings.ReplaceAll(ormTemplate, "[entityName]", entityName)
code = strings.ReplaceAll(code, "[structFields]", structFields[1:])
code = strings.ReplaceAll(code, "[insertFields]", insertFields[1:])
code = strings.ReplaceAll(code, "[insertPlaceHolders]", insertPlaceHolders[1:])
code = strings.ReplaceAll(code, "[insertParams]", insertParams[1:])
fileName := fmt.Sprintf("./models/%s.go", entityName)
return os.WriteFile(fileName, []byte(code), 0644)
}
To test our ORM generator, we use json placeholder's data again.
func TestGenerateModel(t *testing.T) {
item, err := GetAPIResult[Item]("https://jsonplaceholder.typicode.com/posts/1")
if err != nil {
panic(err)
}
if err := GenerateORM(item, "Post"); err != nil {
panic(err)
}
}
Generating ORM code has the following advantage.
Conclusion:
In this post, we've explored diverse methodologies for maximizing function usability in Go.