GoLang Power Tools: Reflection, Unsafe, Templates & CLI Utilities
A guide to the 'dark arts' of Go (Reflection & Unsafe) and the essential utilities for building production tools. Learn to manipulate types at runtime, handle complex text generation, and implement structured logging.
Reflection
reflect package
The reflect package implements runtime reflection, allowing inspection and modification of types and values dynamically. Use sparingly—it's slow, bypasses type safety, and complicates code. Common in serialization, ORMs, and frameworks.
import "reflect" var x float64 = 3.4 // Get type and value t := reflect.TypeOf(x) // reflect.Type v := reflect.ValueOf(x) // reflect.Value fmt.Println(t) // float64 fmt.Println(t.Kind()) // float64 fmt.Println(v) // 3.4 fmt.Println(v.Float()) // 3.4 // Common use cases: // - JSON/XML marshaling // - ORM field mapping // - Dependency injection // - Generic programming (pre-generics) // - Test frameworks
reflect.Type
reflect.Type describes a Go type including its name, kind, methods, and struct fields. Get it with reflect.TypeOf(value). Use for type inspection without the value.
type Person struct { Name string `json:"name"` Age int `json:"age"` } t := reflect.TypeOf(Person{}) t.Name() // "Person" t.PkgPath() // "main" t.Kind() // reflect.Struct t.Size() // Size in bytes t.NumField() // 2 t.NumMethod() // Number of methods // Get field by index field := t.Field(0) // StructField field.Name // "Name" field.Type // reflect.Type of string field.Tag // `json:"name"` // Get field by name field, ok := t.FieldByName("Age") // Get method method, ok := t.MethodByName("String")
reflect.Value
reflect.Value holds a value and provides methods to read, modify, or call it. Get it with reflect.ValueOf(value). Pass a pointer to modify the original value.
x := 3.4 v := reflect.ValueOf(x) v.Kind() // reflect.Float64 v.Type() // float64 v.Float() // 3.4 v.CanSet() // false (need pointer) v.Interface() // interface{} containing 3.4 // To modify, pass pointer v = reflect.ValueOf(&x).Elem() v.CanSet() // true v.SetFloat(7.2) // x is now 7.2 // Struct value p := Person{Name: "Alice", Age: 30} v = reflect.ValueOf(&p).Elem() v.Field(0).SetString("Bob") v.FieldByName("Age").SetInt(25)
TypeOf and ValueOf
reflect.TypeOf and reflect.ValueOf are the entry points to reflection. TypeOf returns nil for nil interfaces; ValueOf returns zero Value. Both accept any type through the empty interface.
// TypeOf var i interface{} = 42 t := reflect.TypeOf(i) // int var nilPtr *int t = reflect.TypeOf(nilPtr) // *int (not nil!) var nilIface interface{} t = reflect.TypeOf(nilIface) // nil! // ValueOf v := reflect.ValueOf(42) v.Int() // 42 v = reflect.ValueOf(nilPtr) v.IsNil() // true v.IsValid() // true v = reflect.ValueOf(nilIface) v.IsValid() // false (zero Value) // Convert back to interface{} original := v.Interface() num := original.(int) // Type assert back
Kind method
Kind returns the base type category, distinguishing between alias types and composites. Use Kind for type-independent logic (e.g., "iterate over any slice") rather than exact type matching.
type MyInt int type MySlice []int v1 := reflect.ValueOf(MyInt(42)) v2 := reflect.ValueOf([]int{1, 2}) v3 := reflect.ValueOf(MySlice{1, 2}) v1.Type().String() // "main.MyInt" v1.Kind() // reflect.Int v2.Kind() // reflect.Slice v3.Kind() // reflect.Slice (same!) // All kinds // Invalid, Bool, Int, Int8...Int64 // Uint, Uint8...Uint64, Uintptr // Float32, Float64, Complex64, Complex128 // Array, Chan, Func, Interface, Map // Pointer, Slice, String, Struct, UnsafePointer // Switch on kind switch v.Kind() { case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: fmt.Println(v.Int()) case reflect.String: fmt.Println(v.String()) case reflect.Struct: // Handle struct }
Struct field inspection
Access struct fields by index or name, including unexported fields (read-only). Each field has type, name, tag, offset, and embedded status information.
type User struct { ID int `json:"id" db:"user_id"` Name string `json:"name"` email string // unexported *Profile // embedded } t := reflect.TypeOf(User{}) // Iterate fields for i := 0; i < t.NumField(); i++ { field := t.Field(i) fmt.Printf("%s: %s\n", field.Name, field.Type) } // Get specific field field, found := t.FieldByName("Name") if found { fmt.Println(field.Index) // [1] fmt.Println(field.Anonymous) // false } // Nested field access field, _ := t.FieldByName("Profile") field.Anonymous // true (embedded) // Check exported field.PkgPath == "" // Exported if empty
Struct tag reading
Struct tags are key-value pairs in backticks. Parse them with reflect.StructTag.Get(key) or Lookup(key) for existence checking. Common uses: JSON field names, validation rules, ORM mapping.
type User struct { ID int `json:"id" db:"user_id" validate:"required"` Name string `json:"name,omitempty" validate:"min=2,max=50"` } t := reflect.TypeOf(User{}) field, _ := t.FieldByName("Name") tag := field.Tag // reflect.StructTag // Get tag value jsonTag := tag.Get("json") // "name,omitempty" dbTag := tag.Get("db") // "" (not present) // Check existence val, ok := tag.Lookup("validate") if ok { // val = "min=2,max=50" } // Parse JSON tag parts := strings.Split(jsonTag, ",") name := parts[0] // "name" hasOmitempty := len(parts) > 1 && parts[1] == "omitempty"
Setting values with reflection
To modify values via reflection, you need a settable Value (addressable, exported). Pass a pointer and call .Elem() to get the settable underlying value.
// ❌ Can't set - not addressable x := 3.4 v := reflect.ValueOf(x) v.CanSet() // false // ✅ Pass pointer, use Elem() v = reflect.ValueOf(&x).Elem() v.CanSet() // true v.SetFloat(7.2) // Struct fields type Person struct { Name string age int // unexported } p := Person{Name: "Alice", age: 30} v = reflect.ValueOf(&p).Elem() // Set exported field v.FieldByName("Name").SetString("Bob") // Can't set unexported field v.FieldByName("age").CanSet() // false // Set by index v.Field(0).SetString("Charlie") // Create and set slice slice := reflect.MakeSlice(reflect.TypeOf([]int{}), 3, 5) slice.Index(0).SetInt(1)
Method invocation
Call methods dynamically using reflect.Value.Method or MethodByName. Pass arguments as []reflect.Value and receive results the same way.
type Calculator struct{} func (c Calculator) Add(a, b int) int { return a + b } func (c Calculator) Greet(name string) string { return "Hello, " + name } calc := Calculator{} v := reflect.ValueOf(calc) // Get method by name method := v.MethodByName("Add") // Prepare arguments args := []reflect.Value{ reflect.ValueOf(3), reflect.ValueOf(4), } // Call method results := method.Call(args) sum := results[0].Int() // 7 // Check if method exists method = v.MethodByName("Unknown") if !method.IsValid() { // Method not found } // Variadic methods // Use CallSlice for variadic functions
Creating instances dynamically
Create new instances of types dynamically using reflect.New (returns pointer), reflect.Zero (returns zero value), and reflect.MakeSlice/Map/Chan for composites.
// Create new instance (returns pointer) t := reflect.TypeOf(Person{}) ptrValue := reflect.New(t) // *Person instance := ptrValue.Elem() // Person // Set fields instance.FieldByName("Name").SetString("Alice") // Get as interface person := ptrValue.Interface().(*Person) // Zero value zeroVal := reflect.Zero(t) // Zero Person{} // Create slice sliceType := reflect.TypeOf([]int{}) slice := reflect.MakeSlice(sliceType, 5, 10) // Create map mapType := reflect.TypeOf(map[string]int{}) m := reflect.MakeMap(mapType) m.SetMapIndex(reflect.ValueOf("key"), reflect.ValueOf(42)) // Create channel chanType := reflect.TypeOf(make(chan int)) ch := reflect.MakeChan(chanType, 10)
Reflection performance considerations
Reflection is 10-100x slower than direct code and allocates heavily. Cache type information, avoid in hot paths, and prefer generics (Go 1.18+) or code generation when performance matters.
// ❌ Slow: reflects on every call func slowSet(obj interface{}, field string, value interface{}) { v := reflect.ValueOf(obj).Elem() v.FieldByName(field).Set(reflect.ValueOf(value)) } // ✅ Better: cache field index type fieldCache struct { index []int typ reflect.Type } var cache sync.Map func fastSet(obj interface{}, field string, value interface{}) { // Cache lookup... } // ✅ Best: use generics (Go 1.18+) func Set[T any](obj *T, setter func(*T)) { setter(obj) } // ✅ Or: code generation //go:generate stringer -type=Status type Status int
Unsafe Package
unsafe.Pointer
unsafe.Pointer is a special pointer that can be converted to/from any pointer type and uintptr. It bypasses Go's type safety, enabling low-level memory operations. Handle with extreme care.
import "unsafe" // Convert between pointer types var f float64 = 3.14 p := unsafe.Pointer(&f) i := (*uint64)(p) fmt.Printf("bits: %016x\n", *i) // Conversion rules: // 1. *T → unsafe.Pointer (any pointer type) // 2. unsafe.Pointer → *T (any pointer type) // 3. unsafe.Pointer → uintptr (for arithmetic) // 4. uintptr → unsafe.Pointer (dangerous!) // ⚠️ uintptr doesn't keep object alive! // GC can collect the object while you hold uintptr // Always convert back to unsafe.Pointer immediately
Pointer arithmetic
Pointer arithmetic uses uintptr for offset calculations. Calculate offsets with unsafe.Offsetof, but convert back to unsafe.Pointer immediately—uintptr values aren't updated by GC.
type Data struct { a int32 b int64 c int32 } d := Data{a: 1, b: 2, c: 3} // Access field b via pointer arithmetic ptr := unsafe.Pointer(&d) bPtr := (*int64)(unsafe.Pointer(uintptr(ptr) + unsafe.Offsetof(d.b))) fmt.Println(*bPtr) // 2 // ⚠️ DANGEROUS pattern - uintptr can become stale: // u := uintptr(unsafe.Pointer(&x)) // // ... GC might run here and move x! // p := unsafe.Pointer(u) // INVALID! // ✅ Safe: single expression p := unsafe.Pointer(uintptr(ptr) + offset)
Size and alignment (unsafe.Sizeof, unsafe.Alignof, unsafe.Offsetof)
These functions return memory layout information at compile time. Use them for understanding struct layout, optimizing memory, and interfacing with C code.
type Example struct { a bool // 1 byte b float64 // 8 bytes c int32 // 4 bytes } fmt.Println(unsafe.Sizeof(Example{})) // 24 (with padding) fmt.Println(unsafe.Alignof(Example{})) // 8 var e Example fmt.Println(unsafe.Offsetof(e.a)) // 0 fmt.Println(unsafe.Offsetof(e.b)) // 8 (padded for alignment) fmt.Println(unsafe.Offsetof(e.c)) // 16 // Memory layout: // [a:1][pad:7][b:8][c:4][pad:4] = 24 bytes // // Reordered for efficiency: type Better struct { b float64 // 8 c int32 // 4 a bool // 1 } // Size: 16 bytes (with 3 bytes padding at end)
String and slice internals
Strings and slices have internal header structures. Understanding them enables zero-copy conversions, but modifications can corrupt data or crash the program.
// String header (reflect.StringHeader) type StringHeader struct { Data uintptr Len int } // Slice header (reflect.SliceHeader) type SliceHeader struct { Data uintptr Len int Cap int } // Zero-copy string to []byte (READ-ONLY!) func stringToBytes(s string) []byte { return unsafe.Slice(unsafe.StringData(s), len(s)) } // Zero-copy []byte to string func bytesToString(b []byte) string { return unsafe.String(unsafe.SliceData(b), len(b)) } // ⚠️ The resulting []byte from stringToBytes is READ-ONLY // Modifying it causes undefined behavior (possibly crash) // ⚠️ If b is modified, string changes too (violates string immutability)
When to use unsafe (rarely)
Use unsafe only when absolutely necessary: syscall interfaces, C interop, extreme performance optimization, or low-level runtime work. Most Go code never needs it.
// ✅ Valid use cases: // 1. Syscall/CGO integration // 2. Memory-mapped I/O // 3. Implementing sync primitives // 4. Extreme optimization (after profiling!) // 5. Binary protocol parsing (zero-copy) // ❌ Don't use for: // - "Performance" without benchmarks // - Bypassing the type system for convenience // - Generic programming (use generics instead) // - Normal application code // Before using unsafe, ask: // 1. Is there a safe alternative? // 2. Have I benchmarked to prove it's needed? // 3. Is the code heavily tested? // 4. Am I prepared for it to break in future Go versions?
Unsafe patterns
Common unsafe patterns include zero-copy conversions, struct field access, and atomic operations on arbitrary types. All bypass type safety and require careful validation.
// Pattern 1: Type punning (reinterpret bits) func float64bits(f float64) uint64 { return *(*uint64)(unsafe.Pointer(&f)) } // Pattern 2: Zero-copy slice conversion func intsToBytes(s []int64) []byte { return unsafe.Slice( (*byte)(unsafe.Pointer(unsafe.SliceData(s))), len(s)*8, ) } // Pattern 3: Access unexported field (very bad idea!) type hidden struct { secret int } h := &hidden{secret: 42} secretPtr := (*int)(unsafe.Pointer(h)) fmt.Println(*secretPtr) // 42 // Pattern 4: noescape for stack allocation //go:noescape func noescape(p unsafe.Pointer) unsafe.Pointer
CGO and unsafe
CGO requires unsafe for passing data between Go and C. C pointers become unsafe.Pointer, and you must manage memory lifetime carefully. CGO has significant overhead.
/* #include <stdlib.h> #include <string.h> void process(char* data, int len) { // C code } */ import "C" import "unsafe" func callC(data []byte) { // Convert Go slice to C pointer cdata := C.CBytes(data) // Allocates C memory! defer C.free(cdata) // Must free! C.process((*C.char)(cdata), C.int(len(data))) } // Direct pointer (no copy, data must not move) func directCall(data []byte) { if len(data) == 0 { return } C.process( (*C.char)(unsafe.Pointer(&data[0])), C.int(len(data)), ) // ⚠️ data must not be modified until C returns }
Text Processing
text/template
text/template provides data-driven templates for text output. Define templates with {{actions}}, execute with data, and compose with named templates. Thread-safe for concurrent execution.
import "text/template" const tmpl = ` Hello, {{.Name}}! Your items: {{range .Items}}- {{.}} {{end}} Total: {{len .Items}} items ` data := struct { Name string Items []string }{ Name: "Alice", Items: []string{"apple", "banana", "cherry"}, } t := template.Must(template.New("test").Parse(tmpl)) t.Execute(os.Stdout, data) // Output: // Hello, Alice! // Your items: // - apple // - banana // - cherry // Total: 3 items
Template syntax
Template actions use {{}} delimiters with expressions, control structures, and function calls. Pipelines (|) chain operations, and . represents the current context.
/* Actions: {{.Field}} - Access struct field {{.Method}} - Call method (no args, returns value or value,error) {{$var := .Field}} - Variable assignment {{$var}} - Variable access {{if .Cond}}...{{end}} - Conditional {{if .A}}...{{else if .B}}...{{else}}...{{end}} {{range .Items}}...{{end}} - Loop (. is current element) {{range $i, $v := .Items}}...{{end}} - With index {{with .Field}}...{{end}} - With scope (. = .Field) {{template "name" .}} - Include template {{block "name" .}}...{{end}} - Define and include Pipelines: {{.Name | printf "%s!"}} - Pipe to function {{.Items | len}} - Built-in function */ t := template.Must(template.New("").Parse(` {{if .Enabled}} {{.Message | printf "STATUS: %s"}} {{else}} Disabled {{end}} `))
Template execution
Execute templates with Execute (single output), ExecuteTemplate (named template), or Lookup (get template). Handle errors properly—partial output may be written before error.
t := template.Must(template.New("main").Parse(` {{define "header"}}=== {{.Title}} ==={{end}} {{define "item"}}- {{.}}{{end}} {{template "header" .}} {{range .Items}} {{template "item" .}} {{end}} `)) data := map[string]interface{}{ "Title": "My List", "Items": []string{"one", "two", "three"}, } // Execute main template err := t.Execute(os.Stdout, data) // Execute named template err = t.ExecuteTemplate(os.Stdout, "header", data) // To buffer (check for errors before output) var buf bytes.Buffer if err := t.Execute(&buf, data); err != nil { log.Fatal(err) } os.Stdout.Write(buf.Bytes())
Template functions
Templates include built-in functions and support custom functions via Funcs. Register functions before parsing. Functions can return one or two values (value, optional error).
// Built-in functions: // and, or, not, eq, ne, lt, le, gt, ge // len, index, slice, printf, print, println // html, js, urlquery (escaping) // call (call function) // Custom functions funcs := template.FuncMap{ "upper": strings.ToUpper, "add": func(a, b int) int { return a + b }, "safe": func(s string) template.HTML { return template.HTML(s) // Skip escaping }, "formatDate": func(t time.Time) string { return t.Format("2006-01-02") }, } t := template.Must( template.New("").Funcs(funcs).Parse(` Name: {{.Name | upper}} Sum: {{add 1 2}} Date: {{.Created | formatDate}} `), )
Template composition
Compose templates using define, template, and block. Parse multiple files with ParseFiles or ParseGlob. Use Clone for variants without modifying the original.
// Base template with blocks const base = ` <!DOCTYPE html> <html> <head>{{block "head" .}}<title>{{.Title}}</title>{{end}}</head> <body> {{block "content" .}}Default content{{end}} </body> </html> ` // Page template overriding blocks const page = ` {{define "head"}}<title>{{.Title}} - MySite</title>{{end}} {{define "content"}} <h1>{{.Title}}</h1> <p>{{.Body}}</p> {{end}} ` t := template.Must(template.New("base").Parse(base)) t = template.Must(t.Parse(page)) t.ExecuteTemplate(w, "base", data) // Parse from files t, err := template.ParseFiles("base.html", "page.html") t, err := template.ParseGlob("templates/*.html")
html/template (XSS protection)
html/template is identical to text/template but automatically escapes output based on context (HTML, JS, CSS, URL). Always use it for web output to prevent XSS attacks.
import "html/template" // Same API as text/template t := template.Must(template.New("page").Parse(` <h1>{{.Title}}</h1> <p>{{ .Body}}</p> <a href="/search?q={{.Query}}">Search</a> <script>var name = "{{.Name}}";</script> `)) data := struct { Title string Body string Query string Name string }{ Title: "<script>alert('xss')</script>", Body: "Hello & welcome", Query: "a=1&b=2", Name: "O'Reilly", } t.Execute(os.Stdout, data) // Output (auto-escaped by context): // <h1><script>alert('xss')</script></h1> // <p>Hello & welcome</p> // <a href="/search?q=a%3d1%26b%3d2">Search</a> // <script>var name = "O\x27Reilly";</script> // To bypass escaping (dangerous!): // {{.TrustedHTML | safe}} // with safe func returning template.HTML
text/scanner
text/scanner provides lexical scanning of UTF-8 text, extracting tokens like identifiers, numbers, and strings. Configure which tokens to recognize with the Mode field.
import "text/scanner" var s scanner.Scanner s.Init(strings.NewReader("hello 123 + 45.6 'world'")) // Configure what to scan s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanFloats | scanner.ScanStrings for tok := s.Scan(); tok != scanner.EOF; tok = s.Scan() { switch tok { case scanner.Ident: fmt.Println("Identifier:", s.TokenText()) case scanner.Int: fmt.Println("Integer:", s.TokenText()) case scanner.Float: fmt.Println("Float:", s.TokenText()) case scanner.String, scanner.RawString: fmt.Println("String:", s.TokenText()) default: fmt.Println("Other:", s.TokenText()) } } // Identifier: hello // Integer: 123 // Other: + // Float: 45.6 // String: 'world'
text/tabwriter
text/tabwriter aligns text in columns using tabs. Write tab-separated data, then Flush to compute column widths and produce aligned output. Great for CLI table formatting.
import "text/tabwriter" w := tabwriter.NewWriter(os.Stdout, 0, // minwidth 0, // tabwidth 2, // padding ' ', // padchar 0, // flags ) fmt.Fprintln(w, "Name\tAge\tCity") fmt.Fprintln(w, "----\t---\t----") fmt.Fprintln(w, "Alice\t30\tNew York") fmt.Fprintln(w, "Bob\t25\tLos Angeles") fmt.Fprintln(w, "Charlie\t35\tChicago") w.Flush() // Output: // Name Age City // ---- --- ---- // Alice 30 New York // Bob 25 Los Angeles // Charlie 35 Chicago // Flags: // tabwriter.AlignRight - Right-align columns // tabwriter.Debug - Show column boundaries // tabwriter.DiscardEmptyColumns
Sorting and Searching
sort package
The sort package provides primitives for sorting slices and user-defined collections. It includes convenience functions for common types and interfaces for custom sorting.
import "sort" // Package provides: // - Functions for []int, []float64, []string // - sort.Slice for any slice with less function // - sort.Interface for custom types // - Binary search functions // - Stable sort variants // Quick check if sorted sort.IntsAreSorted([]int{1, 2, 3}) // true sort.StringsAreSorted([]string{"a", "b"}) // true sort.SliceIsSorted(slice, less) // generic
sort.Ints, sort.Strings, sort.Float64s
These convenience functions sort slices of basic types in ascending order in-place. They modify the original slice and use optimized comparison functions.
// Sort integers ints := []int{3, 1, 4, 1, 5, 9, 2, 6} sort.Ints(ints) fmt.Println(ints) // [1 1 2 3 4 5 6 9] // Sort strings (lexicographic) strs := []string{"banana", "apple", "cherry"} sort.Strings(strs) fmt.Println(strs) // [apple banana cherry] // Sort floats floats := []float64{3.14, 1.41, 2.72} sort.Float64s(floats) fmt.Println(floats) // [1.41 2.72 3.14] // Check if sorted sort.IntsAreSorted(ints) // true sort.StringsAreSorted(strs) // true // Note: NaN handling in Float64s // NaN values are sorted to the beginning
sort.Slice
sort.Slice sorts any slice using a provided less function, eliminating the need for a custom type implementing sort.Interface. The less function receives indices, not values.
people := []struct { Name string Age int }{ {"Alice", 30}, {"Bob", 25}, {"Charlie", 35}, } // Sort by age sort.Slice(people, func(i, j int) bool { return people[i].Age < people[j].Age }) // [{Bob 25} {Alice 30} {Charlie 35}] // Sort by name sort.Slice(people, func(i, j int) bool { return people[i].Name < people[j].Name }) // Multiple criteria sort.Slice(people, func(i, j int) bool { if people[i].Age != people[j].Age { return people[i].Age < people[j].Age } return people[i].Name < people[j].Name })
sort.SliceStable
sort.SliceStable preserves the original order of equal elements. Use it when the relative order of equal elements matters, like sorting by one field while maintaining previous sort order.
type Record struct { Name string Priority int Order int // Original order } records := []Record{ {"A", 1, 1}, {"B", 2, 2}, {"C", 1, 3}, {"D", 2, 4}, } // Stable sort by priority - original order preserved within groups sort.SliceStable(records, func(i, j int) bool { return records[i].Priority < records[j].Priority }) // [{A 1 1} {C 1 3} {B 2 2} {D 2 4}] // Priority 1: A, C (original order maintained) // Priority 2: B, D (original order maintained) // Unstable sort might give: [{C 1 3} {A 1 1} {B 2 2} {D 2 4}]
Custom sorting with sort.Interface
Implement sort.Interface with Len, Less, and Swap methods for full control over sorting behavior. Required for older Go code or when you need both Sort and Search on the same type.
type ByAge []Person func (a ByAge) Len() int { return len(a) } func (a ByAge) Less(i, j int) bool { return a[i].Age < a[j].Age } func (a ByAge) Swap(i, j int) { a[i], a[j] = a[j], a[i] } people := []Person{{"Alice", 30}, {"Bob", 25}} sort.Sort(ByAge(people)) // Reverse sort sort.Sort(sort.Reverse(ByAge(people))) // Generic wrapper for multiple sort orders type People []Person func (p People) Len() int { return len(p) } func (p People) Swap(i, j int) { p[i], p[j] = p[j], p[i] } type ByField struct { People less func(a, b *Person) bool } func (b ByField) Less(i, j int) bool { return b.less(&b.People[i], &b.People[j]) }
Binary search
sort.Search performs binary search on a sorted slice, returning the smallest index where the condition is true. Use type-specific functions like sort.SearchInts for convenience.
// Generic binary search // Returns smallest i where f(i) is true // Slice must be sorted so f(i) == true implies f(i+1) == true data := []int{1, 3, 5, 7, 9, 11, 13} // Find index of value 7 i := sort.Search(len(data), func(i int) bool { return data[i] >= 7 }) if i < len(data) && data[i] == 7 { fmt.Println("Found at index", i) } // Convenience functions i := sort.SearchInts(data, 7) // Find 7 in []int i := sort.SearchStrings(strs, "foo") i := sort.SearchFloat64s(floats, 3.14) // Insert position (for missing values) i := sort.SearchInts(data, 6) // Returns 3 (where 6 would go)
Reverse sorting
Reverse any sort order using sort.Reverse wrapper or negate the Less comparison. Works with sort.Interface types and the Slice functions.
// Reverse with sort.Interface nums := []int{3, 1, 4, 1, 5} sort.Sort(sort.Reverse(sort.IntSlice(nums))) // [5 4 3 1 1] strs := []string{"c", "a", "b"} sort.Sort(sort.Reverse(sort.StringSlice(strs))) // ["c" "b" "a"] // Reverse with sort.Slice - just flip comparison sort.Slice(nums, func(i, j int) bool { return nums[i] > nums[j] // > instead of < }) // Descending multi-criteria sort.Slice(people, func(i, j int) bool { if people[i].Age != people[j].Age { return people[i].Age > people[j].Age // Descending age } return people[i].Name < people[j].Name // Ascending name })
Math
math package
The math package provides mathematical constants and functions for float64 operations. For integers, use operators directly or math/bits. Functions follow IEEE 754 for special values.
import "math" // Package provides: // - Mathematical constants (Pi, E, Phi) // - Basic operations (Abs, Max, Min) // - Exponential and logarithmic // - Trigonometric // - Rounding // - Special value handling (Inf, NaN) // Special values math.IsNaN(x) math.IsInf(x, 1) // +Inf math.IsInf(x, -1) // -Inf math.IsInf(x, 0) // Either Inf x := math.NaN() y := math.Inf(1) // +Inf z := math.Inf(-1) // -Inf
Math constants
The package exports common mathematical constants with maximum float64 precision. Use these instead of approximations for accuracy.
math.Pi // 3.141592653589793 math.E // 2.718281828459045 (Euler's number) math.Phi // 1.618033988749895 (Golden ratio) math.Sqrt2 // 1.4142135623730951 math.SqrtE // 1.6487212707001282 math.SqrtPi // 1.7724538509055159 math.SqrtPhi // 1.272019649514069 math.Ln2 // 0.693147... (natural log of 2) math.Ln10 // 2.302585... (natural log of 10) math.Log2E // 1/Ln2 math.Log10E // 1/Ln10 math.MaxFloat64 // 1.7976931348623157e+308 math.SmallestNonzeroFloat64 // 5e-324 math.MaxInt // Platform dependent math.MaxInt64 // 9223372036854775807
Basic operations (Min, Max, Abs)
Basic math functions for float64. Note that Go 1.21+ added generic min and max built-in functions for any ordered type.
// Float64 operations math.Abs(-3.14) // 3.14 math.Max(3.0, 5.0) // 5.0 math.Min(3.0, 5.0) // 3.0 math.Mod(7.0, 3.0) // 1.0 (remainder) math.Remainder(7.0, 3.0) // 1.0 (IEEE 754 remainder) math.Dim(5.0, 3.0) // 2.0 (max(x-y, 0)) math.Copysign(3.0, -1.0) // -3.0 (magnitude of x, sign of y) // Go 1.21+ built-in min/max (any ordered type) min(3, 5) // 3 max(3, 5) // 5 min("a", "b") // "a" min(3.14, 2.72) // 2.72 // For integers before Go 1.21 func minInt(a, b int) int { if a < b { return a } return b }
Rounding functions
Multiple rounding functions handle different use cases. Round uses banker's rounding (round half to even), while Floor and Ceil always round down/up.
x := 3.7 math.Floor(x) // 3.0 (round toward -∞) math.Ceil(x) // 4.0 (round toward +∞) math.Trunc(x) // 3.0 (round toward zero) math.Round(x) // 4.0 (round to nearest, ties to even) math.RoundToEven(x) // 4.0 (explicit banker's rounding) // Negative numbers math.Floor(-3.7) // -4.0 math.Ceil(-3.7) // -3.0 math.Trunc(-3.7) // -3.0 math.Round(-3.5) // -4.0 // Round to N decimal places func roundTo(x float64, places int) float64 { shift := math.Pow(10, float64(places)) return math.Round(x*shift) / shift } roundTo(3.14159, 2) // 3.14
Trigonometric functions
Standard trig functions work in radians. Use math.Pi for conversions. Inverse functions return radians. All handle special values per IEEE 754.
// Basic trig (radians) math.Sin(math.Pi / 2) // 1.0 math.Cos(0) // 1.0 math.Tan(math.Pi / 4) // ~1.0 // Inverse trig (returns radians) math.Asin(1.0) // Pi/2 math.Acos(0.0) // Pi/2 math.Atan(1.0) // Pi/4 math.Atan2(y, x) // Angle of point (x,y) // Hyperbolic math.Sinh(x) math.Cosh(x) math.Tanh(x) math.Asinh(x) math.Acosh(x) math.Atanh(x) // Convert degrees ↔ radians func degToRad(d float64) float64 { return d * math.Pi / 180 } func radToDeg(r float64) float64 { return r * 180 / math.Pi } math.Sin(degToRad(90)) // 1.0
Exponential and logarithmic functions
Exponential and log functions for mathematical and statistical computations. Handles special values correctly, returning Inf or NaN as appropriate.
// Exponential math.Exp(1) // e^1 = 2.718... math.Exp2(10) // 2^10 = 1024 math.Expm1(x) // e^x - 1 (accurate for small x) math.Pow(2, 10) // 2^10 = 1024 math.Pow10(3) // 10^3 = 1000 // Square/cube root math.Sqrt(16) // 4 math.Cbrt(27) // 3 // Logarithms math.Log(math.E) // 1.0 (natural log) math.Log10(1000) // 3.0 (base 10) math.Log2(1024) // 10.0 (base 2) math.Log1p(x) // log(1+x) (accurate for small x) // Arbitrary base func logBase(x, base float64) float64 { return math.Log(x) / math.Log(base) } logBase(81, 3) // 4.0
Random numbers (math/rand)
math/rand provides pseudo-random numbers for simulations and non-security uses. Seed it once at startup. For cryptographic use, use crypto/rand instead.
import "math/rand" // Go 1.20+: auto-seeded, no need to seed manually // Before Go 1.20: rand.Seed(time.Now().UnixNano()) rand.Int() // Random int rand.Intn(100) // 0 <= n < 100 rand.Int63() // Random int64 rand.Float64() // 0.0 <= f < 1.0 rand.Float32() // 0.0 <= f < 1.0 // Random in range [min, max) func randRange(min, max int) int { return rand.Intn(max-min) + min } // Shuffle slice rand.Shuffle(len(slice), func(i, j int) { slice[i], slice[j] = slice[j], slice[i] }) // Custom source for reproducibility src := rand.NewSource(42) // Fixed seed r := rand.New(src) r.Intn(100) // Reproducible sequence
Crypto-secure random (crypto/rand)
crypto/rand provides cryptographically secure random numbers using OS entropy. Use for keys, tokens, salts—anything security-related. Never use math/rand for security.
import "crypto/rand" import "math/big" // Random bytes buf := make([]byte, 32) _, err := rand.Read(buf) // Random integer [0, max) max := big.NewInt(1000) n, err := rand.Int(rand.Reader, max) fmt.Println(n.Int64()) // Random token (URL-safe) func generateToken(length int) (string, error) { b := make([]byte, length) if _, err := rand.Read(b); err != nil { return "", err } return base64.URLEncoding.EncodeToString(b), nil } // Random UUID v4 func generateUUID() string { uuid := make([]byte, 16) rand.Read(uuid) uuid[6] = (uuid[6] & 0x0f) | 0x40 // Version 4 uuid[8] = (uuid[8] & 0x3f) | 0x80 // Variant return fmt.Sprintf("%x-%x-%x-%x-%x", uuid[0:4], uuid[4:6], uuid[6:8], uuid[8:10], uuid[10:16]) }
big package (arbitrary-precision arithmetic)
The math/big package provides arbitrary-precision integers (Int), rationals (Rat), and floats (Float). Use for cryptography, financial calculations, or when native types overflow.
import "math/big" // Big integers a := big.NewInt(1) b := big.NewInt(2) c := new(big.Int) c.Add(a, b) // c = a + b c.Sub(a, b) // c = a - b c.Mul(a, b) // c = a * b c.Div(a, b) // c = a / b c.Exp(a, b, nil) // c = a^b (mod nil means no mod) c.GCD(nil, nil, a, b) // GCD(a, b) // From string n, _ := new(big.Int).SetString("123456789012345678901234567890", 10) // Big rationals (fractions) r := big.NewRat(1, 3) // 1/3 r.FloatString(10) // "0.3333333333" // Big floats (arbitrary precision) f := big.NewFloat(0) f.SetPrec(256) // Precision in bits f.SetString("3.14159265358979323846264338327950288") // Factorial example func factorial(n int64) *big.Int { result := big.NewInt(1) for i := int64(2); i <= n; i++ { result.Mul(result, big.NewInt(i)) } return result } factorial(100) // Works! (158 digits)
Container Packages
container/list (doubly linked list)
The container/list package implements a doubly linked list. Elements can be inserted/removed in O(1). Use when you need frequent insertions/removals in the middle; slices are better for most use cases.
import "container/list" l := list.New() // Insert elements l.PushBack("b") // [b] l.PushFront("a") // [a, b] e := l.PushBack("d") // [a, b, d] l.InsertBefore("c", e) // [a, b, c, d] // Traverse for e := l.Front(); e != nil; e = e.Next() { fmt.Println(e.Value) } // Remove l.Remove(e) // Remove specific element // Move elements l.MoveToFront(e) l.MoveToBack(e) l.MoveBefore(e, mark) l.MoveAfter(e, mark) // List info l.Len() // Number of elements l.Front() // First element l.Back() // Last element // ⚠️ Elements store interface{}, type assert on access val := e.Value.(string)
container/ring (circular list)
The container/ring package implements a circular list where the last element points to the first. Useful for round-robin scheduling, buffers, or any cyclic data structure.
import "container/ring" // Create ring of n elements r := ring.New(5) // Initialize values for i := 0; i < r.Len(); i++ { r.Value = i r = r.Next() } // Traverse (Do visits each element) r.Do(func(v interface{}) { fmt.Println(v) }) // Navigation r.Next() // Next element r.Prev() // Previous element r.Move(3) // Move n elements forward (negative = backward) // Link/Unlink r.Link(s) // Insert ring s after r r.Unlink(3) // Remove 3 elements starting after r // Example: round-robin servers := ring.New(3) servers.Value = "server1" servers = servers.Next() servers.Value = "server2" // ... nextServer := servers.Value.(string) servers = servers.Next() // Rotate
container/heap
The container/heap package provides heap operations for any type implementing heap.Interface. A heap is a tree where parent <= children (min-heap). Use for priority queues.
import "container/heap" // Implement heap.Interface type IntHeap []int func (h IntHeap) Len() int { return len(h) } func (h IntHeap) Less(i, j int) bool { return h[i] < h[j] } // Min heap func (h IntHeap) Swap(i, j int) { h[i], h[j] = h[j], h[i] } func (h *IntHeap) Push(x interface{}) { *h = append(*h, x.(int)) } func (h *IntHeap) Pop() interface{} { old := *h n := len(old) x := old[n-1] *h = old[0 : n-1] return x } // Usage h := &IntHeap{3, 1, 4, 1, 5} heap.Init(h) // Heapify heap.Push(h, 2) // Add element min := heap.Pop(h) // Remove and return min (1) // Peek without removing min = (*h)[0] // Priority queue with custom type type Item struct { value string priority int index int // Maintained by heap }
Flag Parsing
flag package
The flag package provides command-line flag parsing. Define flags, call Parse(), then access values. Flags must come before positional arguments unless you use -- separator.
import "flag" func main() { // flag.Parse() must be called after all flags are defined // and before flags are accessed verbose := flag.Bool("v", false, "verbose output") name := flag.String("name", "World", "name to greet") count := flag.Int("n", 1, "number of greetings") flag.Parse() if *verbose { fmt.Println("Verbose mode enabled") } for i := 0; i < *count; i++ { fmt.Printf("Hello, %s!\n", *name) } // Positional arguments (after flags) args := flag.Args() // []string arg0 := flag.Arg(0) // First positional arg n := flag.NArg() // Number of positional args }
Defining flags
Define flags with typed functions that return pointers, or bind to existing variables with *Var functions. Define before calling Parse().
// Pointer style - returns *T verbose := flag.Bool("verbose", false, "enable verbose mode") name := flag.String("name", "", "your name") count := flag.Int("count", 1, "repeat count") rate := flag.Float64("rate", 1.0, "rate value") timeout := flag.Duration("timeout", 30*time.Second, "timeout duration") // Variable style - binds to existing variable var config string flag.StringVar(&config, "config", "config.yaml", "config file path") var debug bool flag.BoolVar(&debug, "debug", false, "enable debug mode") // Short and long flags flag.BoolVar(&verbose, "v", false, "verbose (shorthand)") flag.BoolVar(&verbose, "verbose", false, "verbose output") flag.Parse() fmt.Println(*name, *count, config)
Flag types
The flag package supports common types directly. For other types, implement flag.Value interface. Duration has special parsing for time strings.
// Built-in types flag.Bool("b", false, "bool flag") // -b or -b=true flag.Int("i", 0, "int flag") // -i=42 flag.Int64("i64", 0, "int64 flag") flag.Uint("u", 0, "uint flag") flag.Uint64("u64", 0, "uint64 flag") flag.Float64("f", 0.0, "float64 flag") flag.String("s", "", "string flag") // -s=value or -s value flag.Duration("d", 0, "duration flag") // -d=1h30m or -d=500ms // Duration parsing examples // "300ms", "1.5h", "2h45m", "1s", "100us", "1m30s" // Bool flag variations // -flag (sets to true) // -flag=true (explicit true) // -flag=false (explicit false) // No -flag (uses default)
Parsing flags
flag.Parse() parses os.Args[1:]. Flags stop at the first non-flag argument or --. Handle errors with custom usage function.
// Custom usage message flag.Usage = func() { fmt.Fprintf(os.Stderr, "Usage: %s [options] <args>\n\n", os.Args[0]) fmt.Fprintf(os.Stderr, "Options:\n") flag.PrintDefaults() fmt.Fprintf(os.Stderr, "\nExamples:\n") fmt.Fprintf(os.Stderr, " %s -v -n 3 file.txt\n", os.Args[0]) } flag.Parse() // After Parse(): flag.NFlag() // Number of flags set flag.NArg() // Number of remaining args flag.Args() // Remaining args as []string flag.Arg(0) // First remaining arg // Flag lookup f := flag.Lookup("name") if f != nil { fmt.Println(f.Value.String()) } // Check if flag was set flag.Visit(func(f *flag.Flag) { fmt.Printf("%s was set to %s\n", f.Name, f.Value) })
Custom flag types
Implement flag.Value interface to parse custom types. The interface has String() and Set(string) error methods.
// Custom type type StringList []string func (s *StringList) String() string { return strings.Join(*s, ",") } func (s *StringList) Set(value string) error { *s = append(*s, value) return nil } // Usage var files StringList flag.Var(&files, "f", "file to process (can be repeated)") flag.Parse() // -f file1.txt -f file2.txt → files = ["file1.txt", "file2.txt"] // Enum type type LogLevel int const ( Debug LogLevel = iota Info Warn Error ) func (l *LogLevel) String() string { ... } func (l *LogLevel) Set(s string) error { switch strings.ToLower(s) { case "debug": *l = Debug case "info": *l = Info // ... default: return fmt.Errorf("invalid log level: %s", s) } return nil }
Subcommands
The flag package doesn't directly support subcommands. Create separate FlagSet for each subcommand and parse based on the first argument.
func main() { // Subcommand flag sets addCmd := flag.NewFlagSet("add", flag.ExitOnError) addName := addCmd.String("name", "", "item name") listCmd := flag.NewFlagSet("list", flag.ExitOnError) listAll := listCmd.Bool("all", false, "list all items") if len(os.Args) < 2 { fmt.Println("Expected 'add' or 'list' subcommand") os.Exit(1) } switch os.Args[1] { case "add": addCmd.Parse(os.Args[2:]) fmt.Println("Adding:", *addName) case "list": listCmd.Parse(os.Args[2:]) fmt.Println("Listing all:", *listAll) default: fmt.Printf("Unknown subcommand: %s\n", os.Args[1]) os.Exit(1) } } // Usage: ./app add -name=item1 // ./app list -all // For complex CLIs, consider: // - github.com/spf13/cobra // - github.com/urfave/cli
Logging
log package
The standard log package provides simple, thread-safe logging to stderr by default. It's minimal but sufficient for basic needs. For structured or leveled logging, use third-party libraries.
import "log" func main() { // Default logger writes to stderr log.Println("Application started") log.Printf("Processing %d items", 100) // Log and exit log.Fatal("Fatal error!") // Logs then os.Exit(1) // Log and panic log.Panic("Panic!") // Logs then panic() // Default output format: // 2024/01/15 10:30:45 Application started }
log.Print, log.Println, log.Printf
These are the basic logging functions, equivalent to fmt.Print* but with timestamp prefix. They write to the logger's output (stderr by default).
log.Print("Hello") // No newline, joins args log.Println("Hello") // Adds newline, spaces between args log.Printf("Value: %d", 42) // Formatted output // Output: // 2024/01/15 10:30:45 Hello // 2024/01/15 10:30:45 Hello // 2024/01/15 10:30:45 Value: 42 // Multiple arguments log.Println("User:", name, "Age:", age) // 2024/01/15 10:30:45 User: Alice Age: 30 // Print vs Println vs Printf log.Print("a", "b", "c") // abc log.Println("a", "b", "c") // a b c (spaces, newline) log.Printf("%s-%s-%s", "a", "b", "c") // a-b-c
log.Fatal, log.Panic
Fatal* logs then calls os.Exit(1), deferred functions don't run. Panic* logs then panics, deferred functions do run. Use Fatal for unrecoverable startup errors, Panic sparingly.
// Fatal - logs and exits immediately func main() { cfg, err := loadConfig() if err != nil { log.Fatal("Failed to load config:", err) // This line never runs } // Deferred functions DON'T run! } // Panic - logs and panics func process() { defer cleanup() // WILL run if err := riskyOperation(); err != nil { log.Panic("Operation failed:", err) } } // Fatalf, Fatalln, Panicf, Panicln also available log.Fatalf("Config error: %v", err) log.Panicf("Unexpected state: %v", state) // When to use: // Fatal: Startup failures, can't continue at all // Panic: Programming errors, can be recovered // Error: Normal errors, handle gracefully
Custom loggers
Create custom loggers with log.New for different outputs, prefixes, or flags. Useful for module-specific logging or separating error and info logs.
import ( "log" "os" ) // Custom logger infoLog := log.New(os.Stdout, "INFO: ", log.Ldate|log.Ltime) errLog := log.New(os.Stderr, "ERROR: ", log.Ldate|log.Ltime|log.Lshortfile) infoLog.Println("Server started") // INFO: 2024/01/15 10:30:45 Server started errLog.Println("Connection failed") // ERROR: 2024/01/15 10:30:45 main.go:42: Connection failed // Log to file f, _ := os.OpenFile("app.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) fileLog := log.New(f, "", log.LstdFlags) defer f.Close() // Discard log (for testing) silentLog := log.New(io.Discard, "", 0)
Logger prefixes
Prefixes identify log sources. Set with log.New or SetPrefix. Include trailing space or colon for readability.
// Set prefix on default logger log.SetPrefix("[MyApp] ") log.Println("Started") // [MyApp] 2024/01/15 10:30:45 Started // Module-specific loggers dbLog := log.New(os.Stdout, "[DB] ", log.LstdFlags) httpLog := log.New(os.Stdout, "[HTTP] ", log.LstdFlags) authLog := log.New(os.Stdout, "[AUTH] ", log.LstdFlags) dbLog.Println("Connected to database") // [DB] 2024/01/15 10:30:45 Connected to database // Dynamic prefix logger := log.New(os.Stdout, "", log.LstdFlags) logger.SetPrefix("[Request-123] ") logger.Println("Processing") // [Request-123] 2024/01/15 10:30:45 Processing
Logger flags
Flags control what metadata appears in log output. Combine with | operator. Set globally with log.SetFlags or per-logger.
const ( Ldate = 1 << iota // 2024/01/15 Ltime // 10:30:45 Lmicroseconds // 10:30:45.123456 Llongfile // /full/path/to/file.go:42 Lshortfile // file.go:42 LUTC // Use UTC time Lmsgprefix // Move prefix before flags LstdFlags = Ldate | Ltime ) // Examples log.SetFlags(log.LstdFlags | log.Lshortfile) log.Println("Test") // 2024/01/15 10:30:45 main.go:10: Test log.SetFlags(log.Ltime | log.Lmicroseconds | log.LUTC) log.Println("Test") // 10:30:45.123456 Test // Minimal logging log.SetFlags(0) log.SetPrefix("") log.Println("Just the message") // Just the message // Lmsgprefix - prefix after timestamp log.SetFlags(log.LstdFlags | log.Lmsgprefix) log.SetPrefix("[INFO] ") log.Println("Test") // 2024/01/15 10:30:45 [INFO] Test
Output destination
Direct logs to any io.Writer: files, network, multiple destinations with io.MultiWriter. Change with SetOutput.
// Log to file f, _ := os.OpenFile("app.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) log.SetOutput(f) defer f.Close() // Log to multiple destinations multi := io.MultiWriter(os.Stdout, f) log.SetOutput(multi) // Log to buffer (for testing) var buf bytes.Buffer log.SetOutput(&buf) log.Println("Test") logged := buf.String() // Rotating log files (use lumberjack) import "gopkg.in/natefinch/lumberjack.v2" log.SetOutput(&lumberjack.Logger{ Filename: "/var/log/app.log", MaxSize: 100, // MB MaxBackups: 3, MaxAge: 28, // days Compress: true, })
Structured logging libraries (zap, zerolog, logrus)
For production applications, use structured logging libraries that support levels, JSON output, and high performance. Zap and zerolog are fastest; logrus is most popular but slower.
// zerolog - fast, zero allocation import "github.com/rs/zerolog/log" log.Info(). Str("user", "alice"). Int("attempt", 3). Msg("Login successful") // {"level":"info","user":"alice","attempt":3,"message":"Login successful","time":"2024-01-15T10:30:45Z"} // zap - uber's structured logger import "go.uber.org/zap" logger, _ := zap.NewProduction() defer logger.Sync() logger.Info("Login successful", zap.String("user", "alice"), zap.Int("attempt", 3), ) // logrus - popular, feature-rich import log "github.com/sirupsen/logrus" log.SetFormatter(&log.JSONFormatter{}) log.WithFields(log.Fields{ "user": "alice", "attempt": 3, }).Info("Login successful") // slog - Go 1.21+ standard library structured logging import "log/slog" slog.Info("Login successful", "user", "alice", "attempt", 3, )