📦 Packages and Modules in Go

Master Go's package system, module management, and project organization for building scalable, maintainable applications

📚 Theory: Understanding Go's Package System

Go's package system is fundamental to code organization, reusability, and maintainability. Every Go program is made up of packages, with main being the entry point for executable programs.

Package Design Philosophy

Go packages follow key principles:

  • Simplicity: One package, one purpose
  • Clarity: Package names describe their purpose
  • Orthogonality: Packages should be independent
  • Composability: Small packages that work together

Package Visibility Rules

Identifier Visibility Example Access
Uppercase Exported (Public) func Calculate() Accessible outside package
Lowercase Unexported (Private) func helper() Package-internal only
Internal Module-internal internal/auth Parent module only

Package Basics

// math/operations.go
package math

import (
    "fmt"
    "errors"
    "sync"
)

// Exported constant - starts with capital letter
const Pi = 3.14159265359

// Exported type
type Calculator struct {
    Precision int      // Exported field
    mutex     sync.Mutex // Unexported field
}

// Exported function
func Add(a, b float64) float64 {
    return a + b
}

// Unexported helper function
func validate(x float64) error {
    if x < 0 {
        return errors.New("negative value not allowed")
    }
    return nil
}

// init() runs automatically when package is imported
func init() {
    fmt.Println("Math package initialized")
    // Setup code, register handlers, etc.
}

🚀 Go Modules

Modules are Go's dependency management system, introduced in Go 1.11. They define a collection of related packages versioned together.

Module Components

  • go.mod: Module definition and dependencies
  • go.sum: Cryptographic checksums for security
  • Module path: Import path prefix
  • Version: Semantic version tag

Version Selection

  • MVS: Minimal Version Selection algorithm
  • Semantic: v1.2.3 format
  • Pseudo-versions: For untagged commits
  • Major versions: /v2 in import path

Module Commands

  • init: Create new module
  • tidy: Clean dependencies
  • download: Download modules
  • vendor: Create vendor directory

Creating and Managing Modules

# Initialize a new module
$ go mod init github.com/username/myproject

# Add a dependency
$ go get github.com/gorilla/mux@v1.8.0

# Update dependencies
$ go get -u ./...

# Remove unused dependencies
$ go mod tidy

# Vendor dependencies
$ go mod vendor

# Verify dependencies
$ go mod verify

# Show module dependency graph
$ go mod graph

# Why is a package needed?
$ go mod why github.com/pkg/errors

go.mod File Structure

// go.mod
module github.com/username/myproject

go 1.21

require (
    github.com/gorilla/mux v1.8.0
    github.com/stretchr/testify v1.8.4
    github.com/redis/go-redis/v9 v9.0.5
)

require (
    // Indirect dependencies
    github.com/davecgh/go-spew v1.1.1 // indirect
    github.com/pmezard/go-difflib v1.0.0 // indirect
)

// Replace directive for local development
replace github.com/mycompany/shared => ../shared

// Exclude specific versions
exclude github.com/broken/package v1.0.0

// Retract versions with issues
retract (
    v1.0.0 // Published accidentally
    v1.0.1 // Contains critical bug
)

📁 Project Organization

Well-organized project structure is crucial for maintainability and collaboration. Go projects follow common patterns and conventions.

Standard Project Layout

myproject/ ├── go.mod ├── go.sum ├── main.go # Application entry point ├── Makefile # Build automation ├── README.md # Project documentation ├── .gitignore ├── cmd/ # Command-line applications │ ├── server/ │ │ └── main.go # Server entry point │ └── cli/ │ └── main.go # CLI tool entry point ├── internal/ # Private application code │ ├── config/ │ │ ├── config.go │ │ └── config_test.go │ ├── database/ │ │ ├── postgres.go │ │ └── migrations/ │ └── service/ │ ├── user.go │ └── auth.go ├── pkg/ # Public libraries │ ├── api/ │ │ ├── handler.go │ │ └── middleware.go │ └── utils/ │ ├── validator.go │ └── crypto.go ├── api/ # API definitions │ ├── openapi.yaml │ └── proto/ ├── web/ # Web assets │ ├── templates/ │ └── static/ ├── scripts/ # Build/install scripts │ ├── build.sh │ └── install.sh ├── test/ # Additional test data │ ├── integration/ │ └── testdata/ └── vendor/ # Vendored dependencies (optional)

Package Organization Patterns

Directory Purpose Visibility
/cmd Application entry points Main packages
/internal Private application code Module-internal only
/pkg Public libraries Importable by others
/api API contracts and schemas Documentation
/vendor Vendored dependencies Local copies

🔒 Internal Packages

The internal directory provides module-level encapsulation, preventing external imports while allowing internal sharing.

// internal/auth/auth.go
// Can only be imported by packages in the same module
package auth

import (
    "crypto/rand"
    "encoding/base64"
    "time"
)

type TokenManager struct {
    secret []byte
    expiry time.Duration
}

func NewTokenManager(secret string) *TokenManager {
    return &TokenManager{
        secret: []byte(secret),
        expiry: 24 * time.Hour,
    }
}

func (tm *TokenManager) GenerateToken() (string, error) {
    b := make([]byte, 32)
    if _, err := rand.Read(b); err != nil {
        return "", err
    }
    return base64.URLEncoding.EncodeToString(b), nil
}

// main.go can import this
import "myproject/internal/auth"

// External packages cannot import internal/
// This will fail: import "github.com/user/project/internal/auth"

🔄 Dependency Management

Go modules provide powerful dependency management with version control, security verification, and reproducible builds.

Dependency Management Strategies

Version Selection

# Specific version
go get pkg@v1.2.3

# Latest version
go get pkg@latest

# Specific commit
go get pkg@commithash

# Branch
go get pkg@main

Private Modules

# Configure private repos
go env -w GOPRIVATE=*.corp.com
go env -w GONOSUMDB=*.corp.com

# Git configuration
git config --global \
  url."ssh://git@github.com/".insteadOf \
  "https://github.com/"

Advanced Module Features

// Working with major versions
import (
    "github.com/user/pkg"       // v0 or v1
    pkgv2 "github.com/user/pkg/v2" // v2+
)

// Build constraints for conditional compilation
//go:build integration
// +build integration

package tests

// Module-aware tools
// Install tools as dependencies
//go:generate go run github.com/tool/cmd@v1.0.0

// tools.go pattern for development tools
//go:build tools
package tools

import (
    _ "github.com/golangci/golangci-lint/cmd/golangci-lint"
    _ "golang.org/x/tools/cmd/goimports"
)

📖 Package Documentation

Well-documented packages are essential for usability. Go provides built-in documentation tools through godoc.

// Package mathutil provides utility functions for mathematical operations
// that are not available in the standard library.
//
// Basic Usage
//
// The package offers various mathematical utilities:
//
//	result := mathutil.Factorial(5)
//	fmt.Println(result) // Output: 120
//
// For more complex operations, use the Calculator type:
//
//	calc := mathutil.NewCalculator()
//	calc.SetPrecision(10)
//	result := calc.SquareRoot(2)
package mathutil

import "errors"

// ErrNegativeInput is returned when a function receives
// a negative input where only non-negative values are valid.
var ErrNegativeInput = errors.New("negative input not allowed")

// Factorial calculates the factorial of n.
// It returns an error if n is negative.
//
// Example:
//
//	fact, err := Factorial(5)
//	if err != nil {
//	    log.Fatal(err)
//	}
//	fmt.Println(fact) // Output: 120
func Factorial(n int) (int, error) {
    if n < 0 {
        return 0, ErrNegativeInput
    }
    if n == 0 || n == 1 {
        return 1, nil
    }
    
    result := 1
    for i := 2; i <= n; i++ {
        result *= i
    }
    return result, nil
}

// Deprecated: Use Factorial instead.
// This function will be removed in v2.0.0.
func OldFactorial(n int) int {
    // Legacy implementation
    return 0
}

Documentation Tools

# View documentation locally
$ go doc fmt.Printf

# Start local documentation server
$ go install golang.org/x/tools/cmd/godoc@latest
$ godoc -http=:6060

# Generate static HTML documentation
$ go doc -all > docs.txt

# Check for documentation issues
$ go vet ./...

# Host on pkg.go.dev automatically
# Just push to a public repository!

⚡ Best Practices and Performance

Package Design Guidelines

Principle Description Example
Small Interfaces Define minimal interfaces io.Reader has one method
No Circular Deps Avoid circular imports Use interfaces to break cycles
Stable APIs Minimize breaking changes Use versioning for major changes
Clear Names Package names describe purpose http, json, time
Single Purpose One package, one responsibility Separate concerns into packages

Module Best Practices

  • Semantic Versioning: Follow v1.2.3 format strictly
  • Minimal Dependencies: Only add necessary dependencies
  • Regular Updates: Keep dependencies current but stable
  • Version Pinning: Pin versions for reproducible builds
  • Module Proxy: Use proxy for faster downloads and availability
  • Checksum Database: Verify integrity with go.sum
  • License Compliance: Check dependency licenses

Common Pitfalls to Avoid

  • Circular dependencies: Redesign package boundaries
  • God packages: Split large packages into smaller ones
  • Mixing concerns: Separate business logic from infrastructure
  • Ignoring go.sum: Always commit go.sum to version control
  • Import side effects: Be explicit about blank imports
  • Version conflicts: Use MVS and go mod tidy regularly

🚢 Module Release Process

Release Checklist

# 1. Run tests
$ go test ./...

# 2. Check formatting
$ go fmt ./...
$ goimports -w .

# 3. Run linters
$ golangci-lint run

# 4. Update documentation
$ go doc -all > API.md

# 5. Update version in code if needed
$ echo "const Version = \"v1.0.0\"" > version.go

# 6. Commit changes
$ git add .
$ git commit -m "Release v1.0.0"

# 7. Tag version
$ git tag v1.0.0
$ git push origin v1.0.0

# 8. Create release notes
$ git log --oneline v0.9.0..v1.0.0 > CHANGELOG.md

# 9. For major version changes (v2+)
# Update module path in go.mod
module github.com/user/project/v2

# 10. Verify on pkg.go.dev
# Visit https://pkg.go.dev/github.com/user/project

🏋️ Practice Exercises

Challenge 1: Create a Package Library

Build a reusable package with the following:

  • Public and private functions
  • Custom types with methods
  • Comprehensive documentation
  • Unit tests with coverage

Challenge 2: Module Management

Create a module that:

  • Uses external dependencies
  • Implements version constraints
  • Includes replace directives
  • Vendors dependencies

Challenge 3: Project Structure

Organize a complete project with:

  • Multiple cmd applications
  • Internal and pkg packages
  • API definitions
  • Integration tests

Challenge 4: Module Publishing

Publish a module by:

  • Creating semantic versions
  • Writing release notes
  • Managing breaking changes
  • Supporting multiple versions