Goroutines in Go - Complete Concurrency Guide

📚 Theory: Understanding Goroutines

Goroutines are lightweight threads managed by the Go runtime. They're the foundation of Go's concurrency model, enabling you to write efficient concurrent programs with minimal overhead.

🔍 Key Concepts

G (Goroutine): Lightweight thread with its own stack (initially 2KB)
M (Machine): OS thread that executes goroutines
P (Processor): Scheduling context, holds local run queue
GOMAXPROCS: Number of P's (default = number of CPU cores)
Work Stealing: Idle P's can steal work from other P's queues

🎯 Creating and Managing Goroutines

Basic Goroutine Creation

// Basic goroutine syntax
package main

import (
    "fmt"
    "time"
)

func main() {
    // Starting a goroutine with named function
    go sayHello("World")
    
    // Starting goroutine with anonymous function
    go func() {
        fmt.Println("Anonymous goroutine")
    }()
    
    // Starting goroutine with closure
    message := "Closure variable"
    go func() {
        fmt.Println(message) // Accessing outer variable
    }()
    
    // Wait for goroutines to complete
    time.Sleep(100 * time.Millisecond)
}

func sayHello(name string) {
    fmt.Printf("Hello, %s!\n", name)
}

WaitGroup for Synchronization

package main

import (
    "fmt"
    "sync"
    "time"
)

func worker(id int, wg *sync.WaitGroup) {
    defer wg.Done() // Decrement counter when done
    
    fmt.Printf("Worker %d starting\n", id)
    time.Sleep(time.Second)
    fmt.Printf("Worker %d done\n", id)
}

func main() {
    var wg sync.WaitGroup
    
    for i := 1; i <= 5; i++ {
        wg.Add(1) // Increment counter
        go worker(i, &wg)
    }
    
    wg.Wait() // Block until counter reaches 0
    fmt.Println("All workers completed")
}

⚡ Concurrency Patterns

Worker Pool Pattern

Distribute work among fixed number of goroutines to control resource usage.

Common

Efficient

Fan-In/Fan-Out

Merge multiple channels into one (fan-in) or distribute work to multiple goroutines (fan-out).

Advanced

Complex

Pipeline Pattern

Chain goroutines together where output of one is input to another.

Composable

Clean

Worker Pool Implementation

package main

import (
    "fmt"
    "sync"
    "time"
)

type Job struct {
    ID     int
    Data   string
}

type Result struct {
    JobID  int
    Output string
    Error  error
}

func workerPool(numWorkers int, jobs <-chan Job, results chan<- Result) {
    var wg sync.WaitGroup
    
    // Start workers
    for w := 1; w <= numWorkers; w++ {
        wg.Add(1)
        go func(workerID int) {
            defer wg.Done()
            for job := range jobs {
                fmt.Printf("Worker %d processing job %d\n", workerID, job.ID)
                
                // Simulate work
                time.Sleep(100 * time.Millisecond)
                
                result := Result{
                    JobID:  job.ID,
                    Output: fmt.Sprintf("Processed: %s", job.Data),
                }
                results <- result
            }
        }(w)
    }
    
    // Wait for all workers to finish
    go func() {
        wg.Wait()
        close(results)
    }()
}

func main() {
    numJobs := 10
    numWorkers := 3
    
    jobs := make(chan Job, numJobs)
    results := make(chan Result, numJobs)
    
    // Start worker pool
    workerPool(numWorkers, jobs, results)
    
    // Send jobs
    for j := 1; j <= numJobs; j++ {
        jobs <- Job{ID: j, Data: fmt.Sprintf("data-%d", j)}
    }
    close(jobs)
    
    // Collect results
    for result := range results {
        fmt.Printf("Result: %+v\n", result)
    }
}

🔧 Context for Goroutine Management

Using Context for Cancellation

package main

import (
    "context"
    "fmt"
    "time"
)

func longRunningTask(ctx context.Context, id int) {
    for {
        select {
        case <-ctx.Done():
            fmt.Printf("Task %d cancelled: %v\n", id, ctx.Err())
            return
        default:
            // Simulate work
            fmt.Printf("Task %d working...\n", id)
            time.Sleep(500 * time.Millisecond)
        }
    }
}

func main() {
    // Create context with cancellation
    ctx, cancel := context.WithCancel(context.Background())
    
    // Start goroutines
    for i := 1; i <= 3; i++ {
        go longRunningTask(ctx, i)
    }
    
    // Let them run for 2 seconds
    time.Sleep(2 * time.Second)
    
    // Cancel all goroutines
    fmt.Println("Cancelling all tasks...")
    cancel()
    
    // Give time for cleanup
    time.Sleep(time.Second)
}

// Context with timeout
func timeoutExample() {
    ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
    defer cancel()
    
    select {
    case <-time.After(5 * time.Second):
        fmt.Println("Operation completed")
    case <-ctx.Done():
        fmt.Println("Operation timed out")
    }
}

🚨 Common Pitfalls and Solutions

Problem	Cause	Solution
Goroutine Leaks	Goroutines blocked forever on channels or I/O	Use context for cancellation, timeouts
Race Conditions	Multiple goroutines accessing shared state	Use channels or sync.Mutex for synchronization
Deadlocks	Circular dependencies in channel operations	Careful design, use select with default
Too Many Goroutines	Creating goroutines without limit	Use worker pools, semaphores
Closure Variable Capture	Loop variables captured by reference	Pass as parameters or create local copy

Preventing Goroutine Leaks

// BAD: Goroutine leak
func leakyFunction() {
    ch := make(chan int)
    go func() {
        val := <-ch // Blocks forever if no one sends
        fmt.Println(val)
    }()
    // Function returns, channel never written to
}

// GOOD: With proper cleanup
func properFunction(ctx context.Context) {
    ch := make(chan int)
    go func() {
        select {
        case val := <-ch:
            fmt.Println(val)
        case <-ctx.Done():
            fmt.Println("Goroutine cancelled")
            return
        }
    }()
}

// GOOD: Using buffered channel
func bufferedExample() error {
    errCh := make(chan error, 1) // Buffer prevents blocking
    
    go func() {
        // Do work...
        errCh <- nil // Won't block even if no receiver
    }()
    
    select {
    case err := <-errCh:
        return err
    case <-time.After(time.Second):
        return fmt.Errorf("timeout")
    }
}

⚙️ Performance and Debugging

GOMAXPROCS and CPU Affinity

package main

import (
    "fmt"
    "runtime"
)

func main() {
    // Get current GOMAXPROCS value
    fmt.Printf("GOMAXPROCS: %d\n", runtime.GOMAXPROCS(0))
    fmt.Printf("NumCPU: %d\n", runtime.NumCPU())
    fmt.Printf("NumGoroutine: %d\n", runtime.NumGoroutine())
    
    // Set GOMAXPROCS
    runtime.GOMAXPROCS(2) // Limit to 2 cores
    
    // Force garbage collection
    runtime.GC()
    
    // Get memory stats
    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    fmt.Printf("Alloc: %v MB\n", m.Alloc/1024/1024)
    fmt.Printf("TotalAlloc: %v MB\n", m.TotalAlloc/1024/1024)
    fmt.Printf("Sys: %v MB\n", m.Sys/1024/1024)
    fmt.Printf("NumGC: %v\n", m.NumGC)
}

Race Detection

⚠️ Testing for Race Conditions

Always test concurrent code with the race detector:

go test -race ./...

go run -race main.go

🏆 Best Practices

✅ DO's

✓ Use goroutines for I/O bound operations
✓ Limit goroutine creation with worker pools
✓ Always know when and how goroutines exit
✓ Use context for cancellation and timeouts
✓ Test with -race flag
✓ Profile your concurrent code
✓ Keep goroutines simple and focused

❌ DON'Ts

✗ Don't create goroutines in libraries without need
✗ Don't ignore goroutine leaks
✗ Don't use time.Sleep for synchronization
✗ Don't share memory without synchronization
✗ Don't start goroutines in a loop without limiting
✗ Don't panic in goroutines without recovery

🎯 Practice Exercises

Exercise 1: Parallel Web Scraper

Build a concurrent web scraper that fetches multiple URLs in parallel using a worker pool. Include rate limiting and timeout handling.

Exercise 2: Pipeline Processing

Create a data processing pipeline with stages: read → transform → filter → write. Each stage should run in its own goroutine.

Exercise 3: Graceful Shutdown

Implement a service with multiple goroutines that can be gracefully shut down using context and signal handling.

Exercise 4: Fan-Out/Fan-In Pattern

Build a system that distributes work to multiple workers (fan-out) and collects results into a single channel (fan-in).

Challenge: Concurrent Cache

Design a thread-safe cache with TTL support, concurrent reads/writes, and automatic cleanup goroutine for expired entries.

Next: Channels → ← Back to Overview

🚀 Goroutines in Go