45 min read
GoLang API Architecture: Building REST, GraphQL & gRPC Services
Modern backends require versatility. This guide compares and implements the three pillars of service communication: REST for compatibility, GraphQL for data aggregation, and gRPC for low-latency internal systems.
REST API Development
RESTful Principles
REST (Representational State Transfer) is an architectural style using stateless HTTP requests to access and manipulate resources via standard methods, following constraints like uniform interface, client-server separation, and cacheability.
/* REST Constraints: ┌──────────────────────────────────────────────────────────────┐ │ 1. Client-Server - Separation of concerns │ │ 2. Stateless - No client context stored on server │ │ 3. Cacheable - Responses must define cacheability │ │ 4. Uniform Interface- Consistent resource identification │ │ 5. Layered System - Client can't tell if direct or proxy │ │ 6. Code on Demand - Optional executable code (JavaScript) │ └──────────────────────────────────────────────────────────────┘ */ // RESTful API example r := gin.Default() // Resource: Users r.GET("/users", listUsers) // Get all users r.GET("/users/:id", getUser) // Get specific user r.POST("/users", createUser) // Create user r.PUT("/users/:id", updateUser) // Full update r.PATCH("/users/:id", patchUser) // Partial update r.DELETE("/users/:id", deleteUser) // Delete user // Nested resources r.GET("/users/:id/orders", getUserOrders) r.POST("/users/:id/orders", createOrder) // Each request is stateless - contains all needed info func getUser(c *gin.Context) { // Authentication from header, not session userID := c.Param("id") // ... return user resource }
Resource Design
Resource design identifies the nouns (entities) in your system and structures URLs around them, avoiding verbs in paths, using plural names, and properly nesting related resources to reflect relationships.
/* Good Resource Design: ┌─────────────────────────────────────────────────┐ │ /users - Collection │ │ /users/123 - Single resource │ │ /users/123/posts - Sub-collection │ │ /users/123/posts/456 - Nested resource │ │ /users/123/profile - Singleton │ └─────────────────────────────────────────────────┘ Bad Design (avoid): ┌─────────────────────────────────────────────────┐ │ /getUsers - Verb in URL │ │ /user/123 - Singular (use plural) │ │ /users/123/getPosts - Verb in URL │ │ /createUser - Verb in URL │ └─────────────────────────────────────────────────┘ */ // Resource representations type User struct { ID int `json:"id"` Email string `json:"email"` Name string `json:"name"` CreatedAt time.Time `json:"created_at"` Links Links `json:"_links,omitempty"` // HATEOAS } type Links struct { Self string `json:"self"` Posts string `json:"posts,omitempty"` Orders string `json:"orders,omitempty"` } // API structure func SetupRoutes(r *gin.Engine) { api := r.Group("/api/v1") { // Core resources api.Resource("/users", &UsersController{}) api.Resource("/products", &ProductsController{}) api.Resource("/orders", &OrdersController{}) // Actions (when needed) api.POST("/users/:id/activate", activateUser) ### HTTP Methods Proper Usage HTTP methods define the action performed on resources: GET (read), POST (create), PUT (full replace), PATCH (partial update), DELETE (remove), with proper idempotency guarantees where GET, PUT, DELETE are idempotent but POST is not. ```go /* HTTP Methods Overview: ┌─────────┬───────────┬────────────┬───────────┬─────────────────────┐ │ Method │ Idempotent│ Safe │ Cacheable │ Request Body │ ├─────────┼───────────┼────────────┼───────────┼─────────────────────┤ │ GET │ Yes │ Yes │ Yes │ No │ │ POST │ No │ No │ No* │ Yes │ │ PUT │ Yes │ No │ No │ Yes │ │ PATCH │ No │ No │ No │ Yes │ │ DELETE │ Yes │ No │ No │ Optional │ │ HEAD │ Yes │ Yes │ Yes │ No │ │ OPTIONS │ Yes │ Yes │ No │ No │ └─────────┴───────────┴────────────┴───────────┴─────────────────────┘ */ func SetupRoutes(r *gin.Engine) { // GET - Retrieve resource(s), no side effects r.GET("/users", func(c *gin.Context) { users := db.FindAllUsers() c.JSON(200, users) }) // POST - Create new resource, not idempotent r.POST("/users", func(c *gin.Context) { var user User c.ShouldBindJSON(&user) created := db.CreateUser(user) c.JSON(201, created) // Return created resource with ID }) // PUT - Replace entire resource, idempotent r.PUT("/users/:id", func(c *gin.Context) { var user User c.ShouldBindJSON(&user) user.ID = c.Param("id") db.ReplaceUser(user) // Full replacement c.JSON(200, user) }) // PATCH - Partial update r.PATCH("/users/:id", func(c *gin.Context) { var updates map[string]interface{} c.ShouldBindJSON(&updates) user := db.UpdateUserFields(c.Param("id"), updates) c.JSON(200, user) }) // DELETE - Remove resource r.DELETE("/users/:id", func(c *gin.Context) { db.DeleteUser(c.Param("id")) c.Status(204) // No content }) }
Status Code Selection
HTTP status codes communicate the result of API requests; proper selection is crucial for client understanding, with 2xx for success, 3xx for redirection, 4xx for client errors, and 5xx for server errors.
/* Common Status Codes: ┌─────┬────────────────────────┬───────────────────────────────────┐ │Code │ Name │ When to Use │ ├─────┼────────────────────────┼───────────────────────────────────┤ │ 200 │ OK │ Successful GET, PUT, PATCH │ │ 201 │ Created │ Successful POST (resource created)│ │ 204 │ No Content │ Successful DELETE │ │ 400 │ Bad Request │ Invalid syntax, validation error │ │ 401 │ Unauthorized │ Missing/invalid authentication │ │ 403 │ Forbidden │ Authenticated but not authorized │ │ 404 │ Not Found │ Resource doesn't exist │ │ 405 │ Method Not Allowed │ HTTP method not supported │ │ 409 │ Conflict │ Resource state conflict │ │ 422 │ Unprocessable Entity │ Validation errors (semantic) │ │ 429 │ Too Many Requests │ Rate limit exceeded │ │ 500 │ Internal Server Error │ Unexpected server error │ │ 503 │ Service Unavailable │ Server temporarily unavailable │ └─────┴────────────────────────┴───────────────────────────────────┘ */ func createUser(c *gin.Context) { var req CreateUserRequest // 400 - Bad Request (malformed JSON) if err := c.ShouldBindJSON(&req); err != nil { c.JSON(400, gin.H{"error": "Invalid JSON format"}) return } // 422 - Validation errors if err := validate.Struct(req); err != nil { c.JSON(422, gin.H{"error": "Validation failed", "details": err.Error()}) return } // 409 - Conflict (duplicate email) if db.EmailExists(req.Email) { c.JSON(409, gin.H{"error": "Email already registered"}) return } // 201 - Created user, err := db.CreateUser(req) if err != nil { c.JSON(500, gin.H{"error": "Internal server error"}) return } c.Header("Location", fmt.Sprintf("/api/v1/users/%d", user.ID)) c.JSON(201, user) }
Versioning Strategies
API versioning allows evolving your API while maintaining backward compatibility, with common strategies including URL path versioning, query parameter, custom header, or content negotiation (Accept header).
/* Versioning Strategies: ┌────────────────────┬─────────────────────────────────────────┐ │ Strategy │ Example │ ├────────────────────┼─────────────────────────────────────────┤ │ URL Path │ /api/v1/users, /api/v2/users │ │ Query Parameter │ /api/users?version=1 │ │ Custom Header │ X-API-Version: 1 │ │ Accept Header │ Accept: application/vnd.api.v1+json │ └────────────────────┴─────────────────────────────────────────┘ */ // URL Path Versioning (Most Common) func SetupRoutes(r *gin.Engine) { v1 := r.Group("/api/v1") { v1.GET("/users", getUsersV1) v1.GET("/users/:id", getUserV1) } v2 := r.Group("/api/v2") { v2.GET("/users", getUsersV2) // New response format v2.GET("/users/:id", getUserV2) } } // Header-based Versioning func VersionMiddleware() gin.HandlerFunc { return func(c *gin.Context) { version := c.GetHeader("X-API-Version") if version == "" { version = c.GetHeader("Accept") // Parse: application/vnd.myapi.v2+json } if version == "" { version = "1" // Default version } c.Set("api-version", version) c.Next() } } func getUsers(c *gin.Context) { version := c.GetString("api-version") switch version { case "2": c.JSON(200, getUsersV2Response()) default: c.JSON(200, getUsersV1Response()) } } // Content Negotiation // Accept: application/vnd.mycompany.myapp.user.v2+json
HATEOAS
HATEOAS (Hypermedia as the Engine of Application State) includes links in API responses that guide clients to available actions and related resources, making APIs self-documenting and allowing servers to evolve URLs without breaking clients.
type User struct { ID int `json:"id"` Name string `json:"name"` Email string `json:"email"` Links Links `json:"_links"` } type Links struct { Self Link `json:"self"` Edit *Link `json:"edit,omitempty"` Delete *Link `json:"delete,omitempty"` Posts *Link `json:"posts,omitempty"` Orders *Link `json:"orders,omitempty"` } type Link struct { Href string `json:"href"` Method string `json:"method,omitempty"` Title string `json:"title,omitempty"` } func getUserHandler(c *gin.Context) { userID := c.Param("id") user := db.GetUser(userID) baseURL := "https://api.example.com/v1" response := User{ ID: user.ID, Name: user.Name, Email: user.Email, Links: Links{ Self: Link{Href: fmt.Sprintf("%s/users/%d", baseURL, user.ID)}, Edit: &Link{Href: fmt.Sprintf("%s/users/%d", baseURL, user.ID), Method: "PUT"}, Delete: &Link{Href: fmt.Sprintf("%s/users/%d", baseURL, user.ID), Method: "DELETE"}, Posts: &Link{Href: fmt.Sprintf("%s/users/%d/posts", baseURL, user.ID)}, Orders: &Link{Href: fmt.Sprintf("%s/users/%d/orders", baseURL, user.ID)}, }, } c.JSON(200, response) } /* HATEOAS Response Example: { "id": 123, "name": "John Doe", "email": "john@example.com", "_links": { "self": {"href": "/api/v1/users/123"}, "edit": {"href": "/api/v1/users/123", "method": "PUT"}, "posts": {"href": "/api/v1/users/123/posts"}, "orders": {"href": "/api/v1/users/123/orders"} } } */
API Documentation
API documentation describes endpoints, request/response formats, authentication, and error handling, essential for API consumers; generated from code annotations or separate specification files for accuracy and maintainability.
// Using swaggo/swag for auto-generation // @title My API // @version 1.0 // @description A sample REST API // @host localhost:8080 // @BasePath /api/v1 // @securityDefinitions.apikey BearerAuth // @in header // @name Authorization // @Summary Get user by ID // @Description Retrieves a user by their unique identifier // @Tags users // @Accept json // @Produce json // @Param id path int true "User ID" // @Success 200 {object} User // @Failure 404 {object} ErrorResponse // @Failure 500 {object} ErrorResponse // @Security BearerAuth // @Router /users/{id} [get] func getUser(c *gin.Context) { // Implementation } // @Summary Create a new user // @Description Creates a new user account // @Tags users // @Accept json // @Produce json // @Param user body CreateUserRequest true "User data" // @Success 201 {object} User // @Failure 400 {object} ErrorResponse // @Failure 422 {object} ValidationError // @Router /users [post] func createUser(c *gin.Context) { // Implementation } // Generate docs: swag init // Serve Swagger UI r.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler))
Swagger/OpenAPI Integration
OpenAPI (formerly Swagger) is the industry standard for REST API specifications, enabling auto-generated documentation, client SDKs, and testing tools from a single source of truth definition file.
// Install: go install github.com/swaggo/swag/cmd/swag@latest // Generate: swag init -g main.go import ( "github.com/gin-gonic/gin" swaggerFiles "github.com/swaggo/files" ginSwagger "github.com/swaggo/gin-swagger" _ "myapp/docs" // Generated docs ) // @title User Management API // @version 1.0 // @description RESTful API for managing users // @termsOfService http://example.com/terms/ // @contact.name API Support // @contact.url http://www.example.com/support // @contact.email support@example.com // @license.name Apache 2.0 // @license.url http://www.apache.org/licenses/LICENSE-2.0.html // @host api.example.com // @BasePath /api/v1 // @schemes https // @securityDefinitions.apikey ApiKeyAuth // @in header // @name X-API-Key func main() { r := gin.Default() // Swagger endpoint r.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler, ginSwagger.URL("/swagger/doc.json"), ginSwagger.DefaultModelsExpandDepth(-1), )) // API routes v1 := r.Group("/api/v1") { v1.GET("/users", getUsers) v1.POST("/users", createUser) } r.Run(":8080") } // Access Swagger UI at: http://localhost:8080/swagger/index.html
Request Validation
Request validation ensures incoming data meets business rules before processing, preventing invalid data from reaching the database and providing clear error messages to API consumers.
import "github.com/go-playground/validator/v10" var validate = validator.New() type CreateUserRequest struct { Name string `json:"name" validate:"required,min=2,max=100"` Email string `json:"email" validate:"required,email"` Password string `json:"password" validate:"required,min=8,strongpass"` Age int `json:"age" validate:"omitempty,gte=0,lte=150"` Role string `json:"role" validate:"required,oneof=admin user guest"` Website string `json:"website" validate:"omitempty,url"` Phone string `json:"phone" validate:"omitempty,e164"` } // Custom validation func init() { validate.RegisterValidation("strongpass", func(fl validator.FieldLevel) bool { pass := fl.Field().String() hasUpper := regexp.MustCompile(`[A-Z]`).MatchString(pass) hasLower := regexp.MustCompile(`[a-z]`).MatchString(pass) hasNumber := regexp.MustCompile(`[0-9]`).MatchString(pass) hasSpecial := regexp.MustCompile(`[!@#$%^&*]`).MatchString(pass) return hasUpper && hasLower && hasNumber && hasSpecial }) } func createUser(c *gin.Context) { var req CreateUserRequest if err := c.ShouldBindJSON(&req); err != nil { c.JSON(400, gin.H{"error": "Invalid JSON"}) return } if err := validate.Struct(req); err != nil { errors := make(map[string]string) for _, err := range err.(validator.ValidationErrors) { errors[err.Field()] = getErrorMessage(err) } c.JSON(422, gin.H{"errors": errors}) return } // Process valid request... } func getErrorMessage(err validator.FieldError) string { switch err.Tag() { case "required": return "This field is required" case "email": return "Invalid email format" case "min": return fmt.Sprintf("Minimum length is %s", err.Param()) default: return "Invalid value" } }
Error Responses
Consistent error responses with structured formats, error codes, and helpful messages enable clients to programmatically handle failures and provide meaningful feedback to end users.
// Standard error response structure type ErrorResponse struct { Error ErrorDetail `json:"error"` } type ErrorDetail struct { Code string `json:"code"` Message string `json:"message"` Details []FieldError `json:"details,omitempty"` RequestID string `json:"request_id,omitempty"` Timestamp time.Time `json:"timestamp"` } type FieldError struct { Field string `json:"field"` Message string `json:"message"` Code string `json:"code"` } // Error codes const ( ErrCodeValidation = "VALIDATION_ERROR" ErrCodeNotFound = "NOT_FOUND" ErrCodeUnauthorized = "UNAUTHORIZED" ErrCodeForbidden = "FORBIDDEN" ErrCodeConflict = "CONFLICT" ErrCodeInternal = "INTERNAL_ERROR" ErrCodeRateLimit = "RATE_LIMIT_EXCEEDED" ) func respondWithError(c *gin.Context, status int, code, message string, details []FieldError) { c.JSON(status, ErrorResponse{ Error: ErrorDetail{ Code: code, Message: message, Details: details, RequestID: c.GetString("request_id"), Timestamp: time.Now().UTC(), }, }) } // Usage func getUser(c *gin.Context) { user, err := db.GetUser(c.Param("id")) if err == sql.ErrNoRows { respondWithError(c, 404, ErrCodeNotFound, "User not found", nil) return } if err != nil { respondWithError(c, 500, ErrCodeInternal, "An unexpected error occurred", nil) return } c.JSON(200, user) } /* Error Response Example: { "error": { "code": "VALIDATION_ERROR", "message": "Request validation failed", "details": [ {"field": "email", "message": "Invalid email format", "code": "invalid_format"}, {"field": "password", "message": "Too short", "code": "min_length"} ], "request_id": "abc-123-def", "timestamp": "2024-01-15T10:30:00Z" } } */
Pagination
Pagination breaks large result sets into manageable chunks, preventing performance issues and improving user experience; common strategies include offset-based, cursor-based, and keyset pagination.
// Pagination types type PaginationParams struct { Page int `form:"page" binding:"min=1"` PageSize int `form:"page_size" binding:"min=1,max=100"` } type PaginatedResponse struct { Data interface{} `json:"data"` Pagination Pagination `json:"pagination"` } type Pagination struct { CurrentPage int `json:"current_page"` PageSize int `json:"page_size"` TotalItems int64 `json:"total_items"` TotalPages int `json:"total_pages"` HasNext bool `json:"has_next"` HasPrev bool `json:"has_prev"` NextCursor string `json:"next_cursor,omitempty"` // For cursor-based } // Offset-based pagination func getUsers(c *gin.Context) { var params PaginationParams params.Page = 1 params.PageSize = 20 c.ShouldBindQuery(¶ms) offset := (params.Page - 1) * params.PageSize var users []User var total int64 db.Model(&User{}).Count(&total) db.Offset(offset).Limit(params.PageSize).Find(&users) totalPages := int(math.Ceil(float64(total) / float64(params.PageSize))) c.JSON(200, PaginatedResponse{ Data: users, Pagination: Pagination{ CurrentPage: params.Page, PageSize: params.PageSize, TotalItems: total, TotalPages: totalPages, HasNext: params.Page < totalPages, HasPrev: params.Page > 1, }, }) } // Cursor-based pagination (better for large datasets) func getUsersCursor(c *gin.Context) { cursor := c.Query("cursor") limit := 20 var users []User query := db.Order("id ASC").Limit(limit + 1) if cursor != "" { cursorID, _ := decodeCursor(cursor) query = query.Where("id > ?", cursorID) } query.Find(&users) hasNext := len(users) > limit if hasNext { users = users[:limit] } var nextCursor string if hasNext { nextCursor = encodeCursor(users[len(users)-1].ID) } c.JSON(200, gin.H{ "data": users, "next_cursor": nextCursor, "has_more": hasNext, }) }
Filtering and Sorting
Filtering and sorting allow clients to narrow down and order results through query parameters, with server-side validation to prevent SQL injection and limit complex queries that could impact performance.
type ListParams struct { // Filtering Status string `form:"status"` Role string `form:"role"` CreatedGT string `form:"created_gt"` // created_at > date CreatedLT string `form:"created_lt"` // created_at < date Search string `form:"q"` // Sorting SortBy string `form:"sort_by" binding:"omitempty,oneof=name email created_at"` SortOrder string `form:"sort_order" binding:"omitempty,oneof=asc desc"` // Pagination Page int `form:"page"` PageSize int `form:"page_size"` } func getUsers(c *gin.Context) { var params ListParams params.Page = 1 params.PageSize = 20 params.SortBy = "created_at" params.SortOrder = "desc" c.ShouldBindQuery(¶ms) query := db.Model(&User{}) // Apply filters if params.Status != "" { query = query.Where("status = ?", params.Status) } if params.Role != "" { query = query.Where("role = ?", params.Role) } if params.CreatedGT != "" { query = query.Where("created_at > ?", params.CreatedGT) } if params.Search != "" { search := "%" + params.Search + "%" query = query.Where("name LIKE ? OR email LIKE ?", search, search) } // Count total (before pagination) var total int64 query.Count(&total) // Apply sorting (whitelist to prevent injection) allowedSorts := map[string]bool{"name": true, "email": true, "created_at": true} if allowedSorts[params.SortBy] { query = query.Order(fmt.Sprintf("%s %s", params.SortBy, params.SortOrder)) } // Apply pagination offset := (params.Page - 1) * params.PageSize query = query.Offset(offset).Limit(params.PageSize) var users []User query.Find(&users) c.JSON(200, gin.H{ "data": users, "total": total, "page": params.Page, }) } // Example request: GET /users?status=active&role=admin&sort_by=name&sort_order=asc&page=1
Rate Limiting
Rate limiting protects APIs from abuse by restricting request frequency per client, typically implemented using token bucket, sliding window, or fixed window algorithms with Redis for distributed systems.
import ( "github.com/go-redis/redis_rate/v10" "github.com/redis/go-redis/v9" ) // Redis-based rate limiter for distributed systems func RateLimitMiddleware(rdb *redis.Client) gin.HandlerFunc { limiter := redis_rate.NewLimiter(rdb) return func(c *gin.Context) { // Identify client (API key, user ID, or IP) key := c.GetHeader("X-API-Key") if key == "" { key = c.ClientIP() } // Different limits for different tiers limit := redis_rate.PerMinute(60) // 60 req/min default if tier := getUserTier(key); tier == "premium" { limit = redis_rate.PerMinute(1000) } res, err := limiter.Allow(c, "rate:"+key, limit) if err != nil { c.AbortWithStatusJSON(500, gin.H{"error": "Rate limiter error"}) return } // Set rate limit headers c.Header("X-RateLimit-Limit", strconv.Itoa(res.Limit.Rate)) c.Header("X-RateLimit-Remaining", strconv.Itoa(res.Remaining)) c.Header("X-RateLimit-Reset", strconv.FormatInt(res.ResetAfter.Milliseconds(), 10)) if res.Remaining < 0 { c.Header("Retry-After", strconv.Itoa(int(res.RetryAfter.Seconds()))) c.AbortWithStatusJSON(429, gin.H{ "error": "Rate limit exceeded", "retry_after": res.RetryAfter.Seconds(), }) return } c.Next() } } /* Rate Limit Headers: ┌────────────────────────────────────────┐ │ X-RateLimit-Limit: 60 │ (requests per window) │ X-RateLimit-Remaining: 45 │ (requests left) │ X-RateLimit-Reset: 1705312800 │ (window reset time) │ Retry-After: 30 │ (when limited) └────────────────────────────────────────┘ */
API Authentication
API authentication verifies client identity using methods like API keys, JWT tokens, OAuth 2.0, or mutual TLS; the choice depends on use case, with tokens preferred for user-facing APIs and API keys for server-to-server communication.
// JWT Authentication func JWTAuthMiddleware(secretKey string) gin.HandlerFunc { return func(c *gin.Context) { authHeader := c.GetHeader("Authorization") if authHeader == "" { c.AbortWithStatusJSON(401, gin.H{"error": "Missing authorization header"}) return } parts := strings.Split(authHeader, " ") if len(parts) != 2 || parts[0] != "Bearer" { c.AbortWithStatusJSON(401, gin.H{"error": "Invalid authorization format"}) return } token, err := jwt.Parse(parts[1], func(token *jwt.Token) (interface{}, error) { if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok { return nil, fmt.Errorf("unexpected signing method") } return []byte(secretKey), nil }) if err != nil || !token.Valid { c.AbortWithStatusJSON(401, gin.H{"error": "Invalid token"}) return } claims := token.Claims.(jwt.MapClaims) c.Set("user_id", claims["sub"]) c.Set("role", claims["role"]) c.Next() } } // API Key Authentication func APIKeyAuthMiddleware(db *Database) gin.HandlerFunc { return func(c *gin.Context) { apiKey := c.GetHeader("X-API-Key") if apiKey == "" { apiKey = c.Query("api_key") } if apiKey == "" { c.AbortWithStatusJSON(401, gin.H{"error": "Missing API key"}) return } // Hash and lookup keyHash := sha256.Sum256([]byte(apiKey)) client, err := db.GetClientByKeyHash(hex.EncodeToString(keyHash[:])) if err != nil { c.AbortWithStatusJSON(401, gin.H{"error": "Invalid API key"}) return } c.Set("client_id", client.ID) c.Set("scopes", client.Scopes) c.Next() } } /* Authentication Methods: ┌──────────────────┬─────────────────────────────────────────┐ │ Method │ Use Case │ ├──────────────────┼─────────────────────────────────────────┤ │ API Keys │ Server-to-server, simple integrations │ │ JWT │ User authentication, stateless │ │ OAuth 2.0 │ Third-party access, delegated auth │ │ Basic Auth │ Simple cases (over HTTPS only) │ │ Mutual TLS │ High security, service mesh │ └──────────────────┴─────────────────────────────────────────┘ */
GraphQL
gqlgen
gqlgen is the most popular Go GraphQL library, using a schema-first approach where you define your GraphQL schema and it generates type-safe Go code for resolvers, models, and runtime.
// Install: go install github.com/99designs/gqlgen@latest // Initialize: gqlgen init // schema.graphql /* type Query { user(id: ID!): User users(limit: Int = 10, offset: Int = 0): [User!]! } type Mutation { createUser(input: CreateUserInput!): User! updateUser(id: ID!, input: UpdateUserInput!): User! } type User { id: ID! name: String! email: String! posts: [Post!]! createdAt: Time! } input CreateUserInput { name: String! email: String! } */ // resolver.go (generated, you implement) type Resolver struct { DB *gorm.DB } func (r *queryResolver) User(ctx context.Context, id string) (*model.User, error) { var user model.User if err := r.DB.First(&user, id).Error; err != nil { return nil, err } return &user, nil } func (r *mutationResolver) CreateUser(ctx context.Context, input model.CreateUserInput) (*model.User, error) { user := &model.User{ Name: input.Name, Email: input.Email, } if err := r.DB.Create(user).Error; err != nil { return nil, err } return user, nil } // main.go func main() { srv := handler.NewDefaultServer(generated.NewExecutableSchema(generated.Config{ Resolvers: &resolver.Resolver{DB: db}, })) http.Handle("/graphql", srv) http.Handle("/", playground.Handler("GraphQL", "/graphql")) log.Fatal(http.ListenAndServe(":8080", nil)) }
graphql-go
graphql-go is an alternative GraphQL library taking a code-first approach where you define your schema using Go code, offering more flexibility but less type safety than gqlgen's schema-first approach.
import "github.com/graphql-go/graphql" var userType = graphql.NewObject(graphql.ObjectConfig{ Name: "User", Fields: graphql.Fields{ "id": &graphql.Field{Type: graphql.ID}, "name": &graphql.Field{Type: graphql.String}, "email": &graphql.Field{Type: graphql.String}, }, }) var queryType = graphql.NewObject(graphql.ObjectConfig{ Name: "Query", Fields: graphql.Fields{ "user": &graphql.Field{ Type: userType, Args: graphql.FieldConfigArgument{ "id": &graphql.ArgumentConfig{Type: graphql.NewNonNull(graphql.ID)}, }, Resolve: func(p graphql.ResolveParams) (interface{}, error) { id := p.Args["id"].(string) return db.GetUser(id) }, }, "users": &graphql.Field{ Type: graphql.NewList(userType), Resolve: func(p graphql.ResolveParams) (interface{}, error) { return db.GetAllUsers() }, }, }, }) func main() { schema, _ := graphql.NewSchema(graphql.SchemaConfig{ Query: queryType, }) http.HandleFunc("/graphql", func(w http.ResponseWriter, r *http.Request) { var params struct { Query string `json:"query"` Variables map[string]interface{} `json:"variables"` } json.NewDecoder(r.Body).Decode(¶ms) result := graphql.Do(graphql.Params{ Schema: schema, RequestString: params.Query, VariableValues: params.Variables, }) json.NewEncoder(w).Encode(result) }) }
Schema Definition
GraphQL schema defines the API's type system, including object types, queries, mutations, subscriptions, and input types, serving as the contract between client and server.
# schema.graphql # Custom scalars scalar Time scalar Upload # Enums enum Role { ADMIN USER GUEST } enum OrderStatus { PENDING PROCESSING SHIPPED DELIVERED } # Object types type User { id: ID! name: String! email: String! role: Role! posts(first: Int, after: String): PostConnection! createdAt: Time! } type Post { id: ID! title: String! content: String! author: User! comments: [Comment!]! publishedAt: Time } type Comment { id: ID! text: String! author: User! } # Input types input CreateUserInput { name: String! email: String! password: String! role: Role = USER } input PostFilter { authorId: ID published: Boolean search: String } # Connections (pagination) type PostConnection { edges: [PostEdge!]! pageInfo: PageInfo! totalCount: Int! } type PostEdge { node: Post! cursor: String! } type PageInfo { hasNextPage: Boolean! hasPreviousPage: Boolean! startCursor: String endCursor: String } # Root types type Query { user(id: ID!): User users(role: Role, limit: Int): [User!]! posts(filter: PostFilter, first: Int, after: String): PostConnection! } type Mutation { createUser(input: CreateUserInput!): User! deleteUser(id: ID!): Boolean! createPost(title: String!, content: String!): Post! } type Subscription { postCreated: Post! userOnline(userId: ID!): User! }
Resolvers
Resolvers are functions that fetch data for each field in your schema, connecting GraphQL queries to your data sources (databases, APIs, caches) and handling the business logic.
// resolver.go with gqlgen type Resolver struct { DB *gorm.DB UserLoader *dataloader.Loader PostService *PostService } // Query resolvers type queryResolver struct{ *Resolver } func (r *queryResolver) User(ctx context.Context, id string) (*model.User, error) { var user model.User if err := r.DB.First(&user, id).Error; err != nil { if errors.Is(err, gorm.ErrRecordNotFound) { return nil, nil // Return null, not error } return nil, fmt.Errorf("failed to fetch user: %w", err) } return &user, nil } func (r *queryResolver) Users(ctx context.Context, role *model.Role, limit *int) ([]*model.User, error) { query := r.DB.Model(&model.User{}) if role != nil { query = query.Where("role = ?", *role) } if limit != nil { query = query.Limit(*limit) } var users []*model.User return users, query.Find(&users).Error } // Mutation resolvers type mutationResolver struct{ *Resolver } func (r *mutationResolver) CreateUser(ctx context.Context, input model.CreateUserInput) (*model.User, error) { user := &model.User{ Name: input.Name, Email: input.Email, Password: hashPassword(input.Password), Role: input.Role, } if err := r.DB.Create(user).Error; err != nil { return nil, fmt.Errorf("failed to create user: %w", err) } return user, nil } // Field resolvers (for nested objects) type userResolver struct{ *Resolver } func (r *userResolver) Posts(ctx context.Context, obj *model.User, first *int, after *string) (*model.PostConnection, error) { // Use dataloader to avoid N+1 queries return r.PostService.GetUserPosts(ctx, obj.ID, first, after) }
Queries and Mutations
Queries read data without side effects while mutations modify server-side data; both use the same resolver pattern but mutations conventionally return the modified resource for client cache updates.
// Queries - Read operations /* query GetUser($id: ID!) { user(id: $id) { id name email posts { title } } } query ListUsers($role: Role) { users(role: $role, limit: 10) { id name role } } */ func (r *queryResolver) User(ctx context.Context, id string) (*model.User, error) { return r.UserService.GetByID(ctx, id) } // Mutations - Write operations /* mutation CreateUser($input: CreateUserInput!) { createUser(input: $input) { id name email } } mutation UpdatePost($id: ID!, $input: UpdatePostInput!) { updatePost(id: $id, input: $input) { id title content updatedAt } } mutation DeleteUser($id: ID!) { deleteUser(id: $id) } */ func (r *mutationResolver) CreateUser(ctx context.Context, input model.CreateUserInput) (*model.User, error) { // Validate if err := validate.Struct(input); err != nil { return nil, fmt.Errorf("validation error: %w", err) } // Create user, err := r.UserService.Create(ctx, input) if err != nil { return nil, err } // Publish event for subscriptions r.UserCreated <- user return user, nil } func (r *mutationResolver) DeleteUser(ctx context.Context, id string) (bool, error) { err := r.UserService.Delete(ctx, id) return err == nil, err }
Subscriptions
Subscriptions enable real-time updates via WebSocket connections, pushing data to clients when server-side events occur, ideal for chat messages, notifications, and live updates.
// schema.graphql /* type Subscription { postCreated: Post! messageReceived(roomId: ID!): Message! userStatusChanged: UserStatus! } */ // resolver.go type subscriptionResolver struct{ *Resolver } func (r *subscriptionResolver) PostCreated(ctx context.Context) (<-chan *model.Post, error) { // Get user from context for authorization user := auth.GetUserFromContext(ctx) if user == nil { return nil, errors.New("unauthorized") } // Create channel for this subscriber postChan := make(chan *model.Post, 1) // Generate unique ID for this subscription id := uuid.New().String() // Register subscriber r.PostSubscribers.Store(id, postChan) // Cleanup on disconnect go func() { <-ctx.Done() r.PostSubscribers.Delete(id) close(postChan) }() return postChan, nil } // Publishing events (in mutation or service) func (r *mutationResolver) CreatePost(ctx context.Context, input model.CreatePostInput) (*model.Post, error) { post, err := r.PostService.Create(ctx, input) if err != nil { return nil, err } // Broadcast to all subscribers r.PostSubscribers.Range(func(key, value interface{}) bool { ch := value.(chan *model.Post) select { case ch <- post: default: // Channel full, skip } return true }) return post, nil } // main.go - WebSocket setup srv := handler.New(executableSchema) srv.AddTransport(transport.Websocket{ KeepAlivePingInterval: 10 * time.Second, Upgrader: websocket.Upgrader{ CheckOrigin: func(r *http.Request) bool { return true }, }, }) srv.AddTransport(transport.POST{})
DataLoader Pattern
DataLoader batches and caches data fetching to solve the N+1 query problem in GraphQL, collecting multiple individual requests into a single batch query and caching results within a request.
import "github.com/graph-gophers/dataloader/v7" // Define batch function func newUserLoader(db *gorm.DB) *dataloader.Loader[string, *model.User] { return dataloader.NewBatchedLoader(func(ctx context.Context, keys []string) []*dataloader.Result[*model.User] { // Single query for all users var users []*model.User db.Where("id IN ?", keys).Find(&users) // Map results by ID userMap := make(map[string]*model.User) for _, u := range users { userMap[u.ID] = u } // Return results in same order as keys results := make([]*dataloader.Result[*model.User], len(keys)) for i, key := range keys { if user, ok := userMap[key]; ok { results[i] = &dataloader.Result[*model.User]{Data: user} } else { results[i] = &dataloader.Result[*model.User]{Error: errors.New("not found")} } } return results }) } // Middleware to create per-request loader func DataLoaderMiddleware(db *gorm.DB) func(http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { loaders := &Loaders{ UserLoader: newUserLoader(db), PostLoader: newPostLoader(db), } ctx := context.WithValue(r.Context(), loadersKey, loaders) next.ServeHTTP(w, r.WithContext(ctx)) }) } } // Use in resolver func (r *postResolver) Author(ctx context.Context, obj *model.Post) (*model.User, error) { loaders := ctx.Value(loadersKey).(*Loaders) return loaders.UserLoader.Load(ctx, obj.AuthorID)() } /* Without DataLoader: With DataLoader: ┌─────────────────┐ ┌─────────────────┐ │ Query posts │ │ Query posts │ │ ↓ │ │ ↓ │ │ Query author 1 │ │ Collect IDs │ │ Query author 2 │ ──► │ ↓ │ │ Query author 3 │ │ Single query: │ │ Query author 4 │ │ WHERE id IN │ │ (N+1 queries) │ │ (1,2,3,4) │ └─────────────────┘ └─────────────────┘ */
GraphQL Middleware
GraphQL middleware (called directives or plugins in gqlgen) intercepts resolver execution for cross-cutting concerns like authentication, logging, tracing, and field-level authorization.
// gqlgen handler middleware func main() { srv := handler.NewDefaultServer(generated.NewExecutableSchema(cfg)) // Add extensions srv.Use(extension.Introspection{}) srv.Use(extension.AutomaticPersistedQuery{Cache: lru.New(100)}) // Custom middleware srv.AroundFields(func(ctx context.Context, next graphql.Resolver) (interface{}, error) { fc := graphql.GetFieldContext(ctx) // Logging start := time.Now() result, err := next(ctx) log.Printf("Field %s took %v", fc.Field.Name, time.Since(start)) return result, err }) srv.AroundOperations(func(ctx context.Context, next graphql.OperationHandler) graphql.ResponseHandler { oc := graphql.GetOperationContext(ctx) log.Printf("Operation: %s", oc.OperationName) return next(ctx) }) } // Schema directives /* directive @auth(requires: Role!) on FIELD_DEFINITION directive @deprecated(reason: String) on FIELD_DEFINITION type Query { publicData: String secretData: String @auth(requires: ADMIN) } */ // Directive implementation type Directive struct{} func (d *Directive) Auth(ctx context.Context, obj interface{}, next graphql.Resolver, requires model.Role) (interface{}, error) { user := auth.GetUser(ctx) if user == nil { return nil, errors.New("unauthorized") } if user.Role != requires { return nil, errors.New("forbidden") } return next(ctx) } // Config cfg := generated.Config{ Resolvers: &resolver.Resolver{}, Directives: generated.DirectiveRoot{ Auth: directive.Auth, }, }
Error Handling
GraphQL error handling returns errors alongside partial data in the response, supporting multiple errors with paths pointing to the problematic fields, allowing clients to handle failures gracefully.
import "github.com/vektah/gqlparser/v2/gqlerror" // Custom error types type NotFoundError struct { Resource string ID string } func (e NotFoundError) Error() string { return fmt.Sprintf("%s with id %s not found", e.Resource, e.ID) } // Resolver with error handling func (r *queryResolver) User(ctx context.Context, id string) (*model.User, error) { user, err := r.DB.GetUser(id) if err != nil { if errors.Is(err, sql.ErrNoRows) { // Return custom error with extensions return nil, &gqlerror.Error{ Message: "User not found", Path: graphql.GetPath(ctx), Extensions: map[string]interface{}{ "code": "NOT_FOUND", "resource": "User", "resourceId": id, }, } } // Log internal errors, return generic message log.Printf("Database error: %v", err) return nil, gqlerror.Errorf("internal server error") } return user, nil } // Error presenter for consistent formatting func errorPresenter(ctx context.Context, err error) *gqlerror.Error { var gqlErr *gqlerror.Error if errors.As(err, &gqlErr) { return gqlErr } // Convert known errors var notFound NotFoundError if errors.As(err, ¬Found) { return &gqlerror.Error{ Message: notFound.Error(), Path: graphql.GetPath(ctx), Extensions: map[string]interface{}{ "code": "NOT_FOUND", }, } } // Default error return gqlerror.Errorf("internal error") } srv.SetErrorPresenter(errorPresenter) /* GraphQL Error Response: { "data": { "user": null }, "errors": [ { "message": "User not found", "path": ["user"], "extensions": { "code": "NOT_FOUND", "resource": "User", "resourceId": "123" } } ] } */
gRPC
Protocol Buffers
Protocol Buffers (protobuf) is Google's language-neutral, platform-neutral serialization format used by gRPC, offering smaller payloads and faster parsing than JSON, with strong typing and backward compatibility.
// user.proto syntax = "proto3"; package user.v1; option go_package = "github.com/myapp/gen/user/v1;userv1"; // Import common types import "google/protobuf/timestamp.proto"; import "google/protobuf/empty.proto"; // Enum enum Role { ROLE_UNSPECIFIED = 0; ROLE_USER = 1; ROLE_ADMIN = 2; } // Messages message User { string id = 1; string name = 2; string email = 3; Role role = 4; google.protobuf.Timestamp created_at = 5; // Nested message Address address = 6; // Repeated field (array) repeated string tags = 7; // Map map<string, string> metadata = 8; // Optional field optional string phone = 9; } message Address { string street = 1; string city = 2; string country = 3; } // Service definition service UserService { rpc GetUser(GetUserRequest) returns (User); rpc ListUsers(ListUsersRequest) returns (ListUsersResponse); rpc CreateUser(CreateUserRequest) returns (User); rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty); } message GetUserRequest { string id = 1; } message ListUsersRequest { int32 page_size = 1; string page_token = 2; } message ListUsersResponse { repeated User users = 1; string next_page_token = 2; } message CreateUserRequest { string name = 1; string email = 2; } message DeleteUserRequest { string id = 1; }
.proto Files
.proto files define the service contract including message types and RPC methods, organized with package names, imports, and options for code generation across multiple languages.
// Directory structure: // proto/ // ├── user/v1/user.proto // ├── order/v1/order.proto // └── common/v1/pagination.proto // proto/common/v1/pagination.proto syntax = "proto3"; package common.v1; option go_package = "github.com/myapp/gen/common/v1;commonv1"; message PaginationRequest { int32 page_size = 1; string page_token = 2; } message PaginationResponse { string next_page_token = 1; int32 total_count = 2; } // proto/user/v1/user.proto syntax = "proto3"; package user.v1; option go_package = "github.com/myapp/gen/user/v1;userv1"; import "common/v1/pagination.proto"; import "google/api/annotations.proto"; // For gRPC-Gateway message User { string id = 1; string name = 2; string email = 3; } service UserService { // With HTTP mapping for gRPC-Gateway rpc GetUser(GetUserRequest) returns (User) { option (google.api.http) = { get: "/v1/users/{id}" }; } rpc ListUsers(ListUsersRequest) returns (ListUsersResponse) { option (google.api.http) = { get: "/v1/users" }; } } message ListUsersRequest { common.v1.PaginationRequest pagination = 1; string filter = 2; } message ListUsersResponse { repeated User users = 1; common.v1.PaginationResponse pagination = 2; }
protoc Compiler
The protoc compiler generates language-specific code from .proto files, with plugins for different languages and extensions like gRPC services and validation.
# Install protoc # macOS: brew install protobuf # Linux: apt install protobuf-compiler # Install Go plugins go install google.golang.org/protobuf/cmd/protoc-gen-go@latest go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest # Generate Go code protoc --go_out=. --go_opt=paths=source_relative \ --go-grpc_out=. --go-grpc_opt=paths=source_relative \ proto/user/v1/user.proto # With buf (recommended tool) # buf.yaml version: v1 breaking: use: - FILE lint: use: - DEFAULT # buf.gen.yaml version: v1 plugins: - plugin: go out: gen opt: paths=source_relative - plugin: go-grpc out: gen opt: paths=source_relative - plugin: grpc-gateway out: gen opt: paths=source_relative - plugin: openapiv2 out: gen # Generate buf generate
// Generated code structure /* gen/ └── user/ └── v1/ ├── user.pb.go // Message types └── user_grpc.pb.go // gRPC service interface */ // Generated message (user.pb.go) type User struct { state protoimpl.MessageState Id string `protobuf:"bytes,1,opt,name=id,proto3"` Name string `protobuf:"bytes,2,opt,name=name,proto3"` Email string `protobuf:"bytes,3,opt,name=email,proto3"` // ... } // Generated service interface (user_grpc.pb.go) type UserServiceServer interface { GetUser(context.Context, *GetUserRequest) (*User, error) ListUsers(context.Context, *ListUsersRequest) (*ListUsersResponse, error) CreateUser(context.Context, *CreateUserRequest) (*User, error) mustEmbedUnimplementedUserServiceServer() }
Service Definition
gRPC service definitions specify RPC methods with request/response message types, supporting four patterns: unary, server streaming, client streaming, and bidirectional streaming.
// user_service.proto syntax = "proto3"; package user.v1; service UserService { // Unary RPC - single request, single response rpc GetUser(GetUserRequest) returns (User); // Server streaming - single request, stream of responses rpc ListUsers(ListUsersRequest) returns (stream User); // Client streaming - stream of requests, single response rpc UploadUsers(stream User) returns (UploadUsersResponse); // Bidirectional streaming - stream both ways rpc Chat(stream ChatMessage) returns (stream ChatMessage); }
// Server implementation type userServer struct { userv1.UnimplementedUserServiceServer db *Database } // Unary func (s *userServer) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.User, error) { user, err := s.db.GetUser(req.Id) if err != nil { return nil, status.Errorf(codes.NotFound, "user not found: %v", err) } return user, nil } // Server streaming func (s *userServer) ListUsers(req *userv1.ListUsersRequest, stream userv1.UserService_ListUsersServer) error { users, err := s.db.ListUsers() if err != nil { return status.Errorf(codes.Internal, "failed to list users: %v", err) } for _, user := range users { if err := stream.Send(user); err != nil { return err } } return nil } // Client streaming func (s *userServer) UploadUsers(stream userv1.UserService_UploadUsersServer) error { var count int32 for { user, err := stream.Recv() if err == io.EOF { return stream.SendAndClose(&userv1.UploadUsersResponse{Count: count}) } if err != nil { return err } s.db.CreateUser(user) count++ } } // Bidirectional streaming func (s *userServer) Chat(stream userv1.UserService_ChatServer) error { for { msg, err := stream.Recv() if err == io.EOF { return nil } if err != nil { return err } // Echo back if err := stream.Send(msg); err != nil { return err } } }
Unary RPC
Unary RPC is the simplest pattern where client sends one request and receives one response, similar to a function call, used for most CRUD operations.
// Server type server struct { userv1.UnimplementedUserServiceServer db *gorm.DB } func (s *server) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.User, error) { // Validate request if req.Id == "" { return nil, status.Error(codes.InvalidArgument, "id is required") } // Check context deadline if ctx.Err() == context.DeadlineExceeded { return nil, status.Error(codes.DeadlineExceeded, "request timeout") } // Fetch from database var user User if err := s.db.First(&user, req.Id).Error; err != nil { if errors.Is(err, gorm.ErrRecordNotFound) { return nil, status.Error(codes.NotFound, "user not found") } return nil, status.Error(codes.Internal, "database error") } return &userv1.User{ Id: user.ID, Name: user.Name, Email: user.Email, }, nil } // Start server func main() { lis, _ := net.Listen("tcp", ":50051") s := grpc.NewServer() userv1.RegisterUserServiceServer(s, &server{db: db}) s.Serve(lis) } // Client func main() { conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure()) defer conn.Close() client := userv1.NewUserServiceClient(conn) ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() user, err := client.GetUser(ctx, &userv1.GetUserRequest{Id: "123"}) if err != nil { st, _ := status.FromError(err) log.Fatalf("Error %s: %s", st.Code(), st.Message()) } fmt.Printf("User: %+v\n", user) }
Server Streaming
Server streaming sends multiple responses for a single request, ideal for returning large datasets, real-time updates, or paginated results without multiple round trips.
// Proto /* rpc ListUsers(ListUsersRequest) returns (stream User); rpc WatchOrders(WatchOrdersRequest) returns (stream Order); */ // Server implementation func (s *server) ListUsers(req *userv1.ListUsersRequest, stream userv1.UserService_ListUsersServer) error { // Query database rows, err := s.db.Query("SELECT id, name, email FROM users WHERE role = $1", req.Role) if err != nil { return status.Errorf(codes.Internal, "query failed: %v", err) } defer rows.Close() for rows.Next() { var user userv1.User if err := rows.Scan(&user.Id, &user.Name, &user.Email); err != nil { return status.Errorf(codes.Internal, "scan failed: %v", err) } // Check if client cancelled if stream.Context().Err() != nil { return status.Error(codes.Canceled, "client cancelled") } // Send user if err := stream.Send(&user); err != nil { return err } } return nil } // Real-time updates func (s *server) WatchOrders(req *userv1.WatchOrdersRequest, stream userv1.UserService_WatchOrdersServer) error { ticker := time.NewTicker(time.Second) defer ticker.Stop() for { select { case <-stream.Context().Done(): return nil case <-ticker.C: orders := s.getNewOrders(req.UserId) for _, order := range orders { if err := stream.Send(order); err != nil { return err } } } } } // Client func main() { stream, err := client.ListUsers(ctx, &userv1.ListUsersRequest{Role: "admin"}) if err != nil { log.Fatal(err) } for { user, err := stream.Recv() if err == io.EOF { break } if err != nil { log.Fatal(err) } fmt.Printf("Received user: %+v\n", user) } }
Client Streaming
Client streaming sends multiple requests for a single response, used for file uploads, batch operations, or aggregating data from the client before processing.
// Proto /* rpc UploadFile(stream UploadFileRequest) returns (UploadFileResponse); rpc BatchCreateUsers(stream CreateUserRequest) returns (BatchCreateResponse); */ // Server implementation func (s *server) UploadFile(stream userv1.FileService_UploadFileServer) error { var fileName string var fileData bytes.Buffer for { chunk, err := stream.Recv() if err == io.EOF { // All chunks received, save file if err := os.WriteFile(fileName, fileData.Bytes(), 0644); err != nil { return status.Errorf(codes.Internal, "failed to save: %v", err) } return stream.SendAndClose(&userv1.UploadFileResponse{ FileName: fileName, Size: int64(fileData.Len()), Success: true, }) } if err != nil { return err } // First chunk contains metadata if fileName == "" { fileName = chunk.FileName } fileData.Write(chunk.Data) } } func (s *server) BatchCreateUsers(stream userv1.UserService_BatchCreateUsersServer) error { var created int32 var errors []string for { req, err := stream.Recv() if err == io.EOF { return stream.SendAndClose(&userv1.BatchCreateResponse{ CreatedCount: created, Errors: errors, }) } if err != nil { return err } if err := s.db.CreateUser(req); err != nil { errors = append(errors, fmt.Sprintf("user %s: %v", req.Email, err)) } else { created++ } } } // Client func uploadFile(client userv1.FileServiceClient, filePath string) error { file, _ := os.Open(filePath) defer file.Close() stream, err := client.UploadFile(context.Background()) if err != nil { return err } buf := make([]byte, 1024*1024) // 1MB chunks first := true for { n, err := file.Read(buf) if err == io.EOF { break } req := &userv1.UploadFileRequest{Data: buf[:n]} if first { req.FileName = filepath.Base(filePath) first = false } if err := stream.Send(req); err != nil { return err } } resp, err := stream.CloseAndRecv() fmt.Printf("Uploaded: %s (%d bytes)\n", resp.FileName, resp.Size) return err }
Bidirectional Streaming
Bidirectional streaming allows both client and server to send streams of messages independently, ideal for chat applications, real-time collaboration, or any interactive protocol.
// Proto /* rpc Chat(stream ChatMessage) returns (stream ChatMessage); rpc ProcessData(stream DataChunk) returns (stream ProcessedResult); */ // Server implementation func (s *server) Chat(stream userv1.ChatService_ChatServer) error { clientID := uuid.New().String() // Register client s.clients.Store(clientID, stream) defer s.clients.Delete(clientID) for { msg, err := stream.Recv() if err == io.EOF { return nil } if err != nil { return err } // Broadcast to all clients s.clients.Range(func(key, value interface{}) bool { if key.(string) != clientID { client := value.(userv1.ChatService_ChatServer) client.Send(&userv1.ChatMessage{ User: msg.User, Content: msg.Content, Time: timestamppb.Now(), }) } return true }) } } // Client func chat(client userv1.ChatServiceClient) { stream, err := client.Chat(context.Background()) if err != nil { log.Fatal(err) } // Receive messages in goroutine go func() { for { msg, err := stream.Recv() if err == io.EOF { return } if err != nil { log.Printf("Recv error: %v", err) return } fmt.Printf("[%s]: %s\n", msg.User, msg.Content) } }() // Send messages scanner := bufio.NewScanner(os.Stdin) for scanner.Scan() { if err := stream.Send(&userv1.ChatMessage{ User: "me", Content: scanner.Text(), }); err != nil { log.Printf("Send error: %v", err) return } } } /* Bidirectional Flow: Client Server │ │ │──── ChatMessage ─────────────►│ │ │ (broadcast) │◄──── ChatMessage ─────────────│ │ │ │──── ChatMessage ─────────────►│ │◄──── ChatMessage ─────────────│ │◄──── ChatMessage ─────────────│ │ │ */
gRPC Metadata
gRPC metadata carries request/response headers for authentication tokens, tracing IDs, and custom data, similar to HTTP headers but using a key-value map structure.
import "google.golang.org/grpc/metadata" // Client - sending metadata func callWithMetadata(client userv1.UserServiceClient) { // Create metadata md := metadata.New(map[string]string{ "authorization": "Bearer " + token, "x-request-id": uuid.New().String(), "x-client-version": "1.0.0", }) // Attach to context ctx := metadata.NewOutgoingContext(context.Background(), md) // Make call user, err := client.GetUser(ctx, &userv1.GetUserRequest{Id: "123"}) // Read response headers var header, trailer metadata.MD user, err = client.GetUser(ctx, req, grpc.Header(&header), grpc.Trailer(&trailer)) fmt.Println("Response header:", header.Get("x-response-id")) } // Server - reading and sending metadata func (s *server) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.User, error) { // Read incoming metadata md, ok := metadata.FromIncomingContext(ctx) if !ok { return nil, status.Error(codes.InvalidArgument, "missing metadata") } // Get specific values authTokens := md.Get("authorization") if len(authTokens) == 0 { return nil, status.Error(codes.Unauthenticated, "missing token") } requestID := md.Get("x-request-id") // Send response metadata (header) header := metadata.Pairs( "x-response-id", uuid.New().String(), "x-server-time", time.Now().String(), ) grpc.SendHeader(ctx, header) // Process request... user := &userv1.User{Id: req.Id, Name: "John"} // Send trailer (after response) trailer := metadata.Pairs("x-processing-time", "50ms") grpc.SetTrailer(ctx, trailer) return user, nil }
Error Handling in gRPC
gRPC uses status codes and detailed error messages, with the status package providing rich error types including code, message, and additional details for structured error information.
import ( "google.golang.org/grpc/codes" "google.golang.org/grpc/status" "google.golang.org/genproto/googleapis/rpc/errdetails" ) // Return appropriate status codes func (s *server) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.User, error) { if req.Id == "" { return nil, status.Error(codes.InvalidArgument, "id is required") } user, err := s.db.GetUser(req.Id) if err != nil { if errors.Is(err, sql.ErrNoRows) { return nil, status.Error(codes.NotFound, "user not found") } return nil, status.Error(codes.Internal, "internal error") } return user, nil } // Rich error with details func (s *server) CreateUser(ctx context.Context, req *userv1.CreateUserRequest) (*userv1.User, error) { // Validation errors with details var violations []*errdetails.BadRequest_FieldViolation if req.Name == "" { violations = append(violations, &errdetails.BadRequest_FieldViolation{ Field: "name", Description: "name is required", }) } if !isValidEmail(req.Email) { violations = append(violations, &errdetails.BadRequest_FieldViolation{ Field: "email", Description: "invalid email format", }) } if len(violations) > 0 { st := status.New(codes.InvalidArgument, "validation failed") st, _ = st.WithDetails(&errdetails.BadRequest{FieldViolations: violations}) return nil, st.Err() } // ... create user } // Client error handling func handleError(err error) { st, ok := status.FromError(err) if !ok { log.Printf("Unknown error: %v", err) return } log.Printf("Code: %s, Message: %s", st.Code(), st.Message()) // Extract details for _, detail := range st.Details() { switch t := detail.(type) { case *errdetails.BadRequest: for _, v := range t.GetFieldViolations() { log.Printf(" Field %s: %s", v.Field, v.Description) } case *errdetails.RetryInfo: log.Printf(" Retry after: %v", t.RetryDelay.AsDuration()) } } } /* gRPC Status Codes: ┌──────────────────┬──────┬────────────────────────────────────┐ │ Code │ Num │ When to Use │ ├──────────────────┼──────┼────────────────────────────────────┤ │ OK │ 0 │ Success │ │ CANCELLED │ 1 │ Operation cancelled │ │ UNKNOWN │ 2 │ Unknown error │ │ INVALID_ARGUMENT │ 3 │ Client error in request │ │ DEADLINE_EXCEEDED│ 4 │ Timeout │ │ NOT_FOUND │ 5 │ Resource doesn't exist │ │ ALREADY_EXISTS │ 6 │ Resource already exists │ │ PERMISSION_DENIED│ 7 │ No permission for operation │ │ UNAUTHENTICATED │ 16 │ No valid auth credentials │ │ RESOURCE_EXHAUST │ 8 │ Rate limit, quota exceeded │ │ INTERNAL │ 13 │ Internal server error │ │ UNAVAILABLE │ 14 │ Service unavailable (retry) │ └──────────────────┴──────┴────────────────────────────────────┘ */
Interceptors
Interceptors are gRPC middleware for cross-cutting concerns like logging, authentication, tracing, and metrics, available for both unary and streaming RPCs on client and server sides.
import "google.golang.org/grpc" // Unary server interceptor func loggingInterceptor( ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler, ) (interface{}, error) { start := time.Now() // Call handler resp, err := handler(ctx, req) // Log after log.Printf("Method: %s, Duration: %v, Error: %v", info.FullMetho ### Interceptors (continued) Interceptors are gRPC middleware for cross-cutting concerns like logging, authentication, tracing, and metrics, available for both unary and streaming RPCs on client and server sides. ```go import "google.golang.org/grpc" // Unary server interceptor func loggingInterceptor( ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler, ) (interface{}, error) { start := time.Now() // Call handler resp, err := handler(ctx, req) // Log after log.Printf("Method: %s, Duration: %v, Error: %v", info.FullMethod, time.Since(start), err) return resp, err } // Authentication interceptor func authInterceptor( ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler, ) (interface{}, error) { // Skip auth for certain methods if info.FullMethod == "/user.v1.UserService/Login" { return handler(ctx, req) } md, ok := metadata.FromIncomingContext(ctx) if !ok { return nil, status.Error(codes.Unauthenticated, "missing metadata") } tokens := md.Get("authorization") if len(tokens) == 0 { return nil, status.Error(codes.Unauthenticated, "missing token") } userID, err := validateToken(tokens[0]) if err != nil { return nil, status.Error(codes.Unauthenticated, "invalid token") } // Add user to context ctx = context.WithValue(ctx, "user_id", userID) return handler(ctx, req) } // Stream server interceptor func streamLoggingInterceptor( srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler, ) error { start := time.Now() err := handler(srv, ss) log.Printf("Stream: %s, Duration: %v, Error: %v", info.FullMethod, time.Since(start), err) return err } // Client interceptor func clientLoggingInterceptor( ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption, ) error { start := time.Now() err := invoker(ctx, method, req, reply, cc, opts...) log.Printf("Client call: %s, Duration: %v", method, time.Since(start)) return err } // Recovery interceptor func recoveryInterceptor( ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler, ) (resp interface{}, err error) { defer func() { if r := recover(); r != nil { log.Printf("Panic recovered: %v\n%s", r, debug.Stack()) err = status.Errorf(codes.Internal, "internal error") } }() return handler(ctx, req) } // Chain interceptors func main() { // Server with chained interceptors server := grpc.NewServer( grpc.ChainUnaryInterceptor( recoveryInterceptor, loggingInterceptor, authInterceptor, ), grpc.ChainStreamInterceptor( streamLoggingInterceptor, ), ) // Client with interceptors conn, _ := grpc.Dial("localhost:50051", grpc.WithUnaryInterceptor(clientLoggingInterceptor), grpc.WithInsecure(), ) } /* Interceptor Chain Flow: Request ──► Recovery ──► Logging ──► Auth ──► Handler │ Response ◄── Recovery ◄── Logging ◄── Auth ◄───┘ */
gRPC Gateway
gRPC Gateway generates a reverse proxy server that translates RESTful HTTP/JSON requests into gRPC calls, allowing your service to support both gRPC and REST clients from a single codebase.
// user.proto with HTTP annotations syntax = "proto3"; package user.v1; import "google/api/annotations.proto"; import "protoc-gen-openapiv2/options/annotations.proto"; option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_swagger) = { info: { title: "User API" version: "1.0" } schemes: HTTPS consumes: "application/json" produces: "application/json" }; service UserService { rpc GetUser(GetUserRequest) returns (User) { option (google.api.http) = { get: "/v1/users/{id}" }; } rpc ListUsers(ListUsersRequest) returns (ListUsersResponse) { option (google.api.http) = { get: "/v1/users" }; } rpc CreateUser(CreateUserRequest) returns (User) { option (google.api.http) = { post: "/v1/users" body: "*" }; } rpc UpdateUser(UpdateUserRequest) returns (User) { option (google.api.http) = { put: "/v1/users/{id}" body: "user" additional_bindings { patch: "/v1/users/{id}" body: "user" } }; } rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty) { option (google.api.http) = { delete: "/v1/users/{id}" }; } } message UpdateUserRequest { string id = 1; User user = 2; }
// Generate gateway code // protoc -I . --grpc-gateway_out . --grpc-gateway_opt paths=source_relative user.proto // main.go - Run both gRPC and HTTP servers func main() { // Start gRPC server go func() { lis, _ := net.Listen("tcp", ":50051") grpcServer := grpc.NewServer() userv1.RegisterUserServiceServer(grpcServer, &userServer{}) grpcServer.Serve(lis) }() // Start HTTP gateway ctx := context.Background() mux := runtime.NewServeMux( runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONPb{ MarshalOptions: protojson.MarshalOptions{ UseProtoNames: true, }, }), runtime.WithErrorHandler(customErrorHandler), ) opts := []grpc.DialOption{grpc.WithInsecure()} err := userv1.RegisterUserServiceHandlerFromEndpoint(ctx, mux, "localhost:50051", opts) if err != nil { log.Fatal(err) } // Add middleware handler := cors.Default().Handler(mux) log.Println("HTTP server listening on :8080") http.ListenAndServe(":8080", handler) } // Custom error handler func customErrorHandler(ctx context.Context, mux *runtime.ServeMux, m runtime.Marshaler, w http.ResponseWriter, r *http.Request, err error) { st, _ := status.FromError(err) httpStatus := runtime.HTTPStatusFromCode(st.Code()) w.Header().Set("Content-Type", "application/json") w.WriteHeader(httpStatus) json.NewEncoder(w).Encode(map[string]interface{}{ "error": map[string]interface{}{ "code": st.Code().String(), "message": st.Message(), }, }) } /* gRPC Gateway Architecture: ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ REST Client │────►│ Gateway │────►│ gRPC Server │ │ (HTTP/JSON) │ │ (HTTP→gRPC) │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ ┌─────────────┐ │ │ gRPC Client │───────────┘ │ │ └─────────────┘ */
Health Checking
gRPC health checking provides a standard protocol for checking service health, used by load balancers, Kubernetes, and service meshes to determine if a service can handle requests.
import ( "google.golang.org/grpc/health" "google.golang.org/grpc/health/grpc_health_v1" ) func main() { server := grpc.NewServer() // Register your services userv1.RegisterUserServiceServer(server, &userServer{}) // Register health service healthServer := health.NewServer() grpc_health_v1.RegisterHealthServer(server, healthServer) // Set service health status healthServer.SetServingStatus("user.v1.UserService", grpc_health_v1.HealthCheckResponse_SERVING) healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING) // Overall health // Update health based on dependencies go func() { for { if checkDatabaseConnection() { healthServer.SetServingStatus("user.v1.UserService", grpc_health_v1.HealthCheckResponse_SERVING) } else { healthServer.SetServingStatus("user.v1.UserService", grpc_health_v1.HealthCheckResponse_NOT_SERVING) } time.Sleep(10 * time.Second) } }() lis, _ := net.Listen("tcp", ":50051") server.Serve(lis) } // Client health check func checkHealth(conn *grpc.ClientConn) error { client := grpc_health_v1.NewHealthClient(conn) ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() resp, err := client.Check(ctx, &grpc_health_v1.HealthCheckRequest{ Service: "user.v1.UserService", }) if err != nil { return err } if resp.Status != grpc_health_v1.HealthCheckResponse_SERVING { return fmt.Errorf("service not serving: %v", resp.Status) } return nil } // Kubernetes gRPC health probe (K8s 1.24+) /* livenessProbe: grpc: port: 50051 initialDelaySeconds: 10 readinessProbe: grpc: port: 50051 service: "user.v1.UserService" initialDelaySeconds: 5 */ // Or use grpc-health-probe CLI // grpc-health-probe -addr=localhost:50051 -service=user.v1.UserService /* Health Check States: ┌────────────────┬───────────────────────────────────┐ │ Status │ Meaning │ ├────────────────┼───────────────────────────────────┤ │ UNKNOWN │ Status not known │ │ SERVING │ Service is healthy │ │ NOT_SERVING │ Service is unhealthy │ │ SERVICE_UNKNOWN│ Service name not registered │ └────────────────┴───────────────────────────────────┘ */
Reflection
gRPC reflection allows clients to query the server for available services and methods at runtime, enabling tools like grpcurl and grpcui to work without proto files.
import "google.golang.org/grpc/reflection" func main() { server := grpc.NewServer() // Register services userv1.RegisterUserServiceServer(server, &userServer{}) orderv1.RegisterOrderServiceServer(server, &orderServer{}) // Enable reflection (typically only in development) if os.Getenv("ENV") != "production" { reflection.Register(server) } lis, _ := net.Listen("tcp", ":50051") server.Serve(lis) } // Using grpcurl with reflection /* # List all services grpcurl -plaintext localhost:50051 list # Output: # grpc.health.v1.Health # grpc.reflection.v1alpha.ServerReflection # user.v1.UserService # Describe a service grpcurl -plaintext localhost:50051 describe user.v1.UserService # Describe a message grpcurl -plaintext localhost:50051 describe user.v1.User # Call a method grpcurl -plaintext -d '{"id": "123"}' \ localhost:50051 user.v1.UserService/GetUser # Call with headers grpcurl -plaintext \ -H "Authorization: Bearer token123" \ -d '{"name": "John", "email": "john@example.com"}' \ localhost:50051 user.v1.UserService/CreateUser # Stream call grpcurl -plaintext -d '{"role": "ADMIN"}' \ localhost:50051 user.v1.UserService/ListUsers */ // Programmatic reflection client import ( "google.golang.org/grpc/reflection/grpc_reflection_v1alpha" ) func listServices(conn *grpc.ClientConn) ([]string, error) { client := grpc_reflection_v1alpha.NewServerReflectionClient(conn) stream, err := client.ServerReflectionInfo(context.Background()) if err != nil { return nil, err } if err := stream.Send(&grpc_reflection_v1alpha.ServerReflectionRequest{ MessageRequest: &grpc_reflection_v1alpha.ServerReflectionRequest_ListServices{}, }); err != nil { return nil, err } resp, err := stream.Recv() if err != nil { return nil, err } var services []string for _, svc := range resp.GetListServicesResponse().GetService() { services = append(services, svc.GetName()) } return services, nil } /* Reflection Use Cases: ┌─────────────────────────────────────────────────────────┐ │ • CLI tools (grpcurl, grpcui) │ │ • API explorers and documentation │ │ • Dynamic client generation │ │ • Testing and debugging │ │ • Service discovery │ │ │ │ ⚠️ Disable in production for security! │ └─────────────────────────────────────────────────────────┘ */