🔌 gRPC and Protobuf Alongside Your HTTP API

Learning objectives

By the end of this lesson you can:

  • Write a proto3 file that defines messages, an enum, and a service with unary and streaming RPCs.
  • Generate Go stubs with protoc, protoc-gen-go, and protoc-gen-go-grpc, and explain what each generated symbol is for.
  • Run a grpc.Server that calls the same business logic your HTTP handlers already call.
  • Translate domain errors into gRPC status codes, and know why that translation belongs at the edge, not in the core.

Quick review

You already have the vertical slice: handler, middleware, repository, real Postgres, integration tests. One habit carries straight into this lesson: context.Context is threaded from the transport down to the query, so cancellation and deadlines reach the database. gRPC gives you that same contract for free, because every generated method signature starts with a context.

Theory

Protobuf is the contract, Go is one binding

A .proto file describes messages and services. Field numbers are the wire format: clients and servers agree on the numbers, not the names. Rename a field freely, never renumber or reuse one. Each proto field has a {{c1::field number}}, and that number is what travels on the wire.

syntax = "proto3";

package orders.v1;

option go_package = "example.com/shop/gen/orders/v1;ordersv1";

message GetOrderRequest {
  string id = 1;
}

message ListOrdersRequest {
  string customer_id = 1;
}

enum Status {
  STATUS_UNSPECIFIED = 0;
  STATUS_OPEN = 1;
  STATUS_SHIPPED = 2;
}

message Order {
  string id = 1;
  string customer_id = 2;
  int64 total_cents = 3;
  Status status = 4;
}

service OrderService {
  rpc GetOrder(GetOrderRequest) returns (Order);
  rpc ListOrders(ListOrdersRequest) returns (stream Order);
}

Two gotchas worth memorizing: proto3 enums must start at {{c2::zero}}, and zero is the default, so name it STATUS_UNSPECIFIED rather than something meaningful. And option go_package decides the import path and package name of the generated Go code; get it wrong once and every import in your repo drifts.

What protoc actually generates

protoc --go_out=. --go_opt=paths=source_relative \
       --go-grpc_out=. --go-grpc_opt=paths=source_relative \
       proto/orders/v1/orders.proto

You get two files. orders.pb.go holds the message structs, getters (req.GetId() is nil-safe), and enum maps. orders_grpc.pb.go holds the transport surface:

  • OrderServiceClient, with one method per RPC.
  • OrderServiceServer, the interface you implement.
  • RegisterOrderServiceServer(s *grpc.Server, impl OrderServiceServer).
  • UnimplementedOrderServiceServer, which you embed so your struct keeps compiling when the schema grows.

Embedding {{c3::UnimplementedOrderServiceServer}} is not optional boilerplate, it is your forward-compatibility insurance.

One core, two adapters

The mistake I see most often is putting business logic inside the gRPC server impl, which forks the service into two divergent copies. Instead, keep orders.Service transport-agnostic: plain structs in, plain structs out, errors from orders package sentinels.

flowchart LR
    HTTP["🌐<br/>HTTP handler"] --> SVC["⚙️<br/>orders.Service"]
    GRPC["🔌<br/>gRPC server"] --> SVC
    SVC --> REPO["🗄️<br/>Postgres repo"]

Both edges are thin: decode, call, map errors, encode.

Errors across two worlds

HTTP speaks status codes as integers; gRPC speaks codes.Code. Your core should know neither. Map at the edge:

func toStatus(err error) error {
    switch {
    case errors.Is(err, orders.ErrNotFound):
        return status.Error(codes.NotFound, "order not found")
    case errors.Is(err, orders.ErrInvalid):
        return status.Error(codes.InvalidArgument, err.Error())
    default:
        return status.Error(codes.Internal, "internal error")
    }
}

A gRPC handler returns a {{c4::status error}} built with status.Error(codes.X, msg), never a raw error. A raw error becomes codes.Unknown on the client and tells them nothing. Also: never echo internal error text, that leaks schema and query details.

Interceptors are gRPC's version of {{c5::middleware}}: grpc.ChainUnaryInterceptor(logging, recovery) gives you the same cross-cutting stack you built for net/http.

One port or two?

Simplest: HTTP on :8080, gRPC on :9090. Clean, easy to route and to containerize. One port is possible with cmux (sniff the connection preface) or grpc-gateway (a generated HTTP/JSON front door that proxies to gRPC). Both add moving parts. For this module, run two ports and keep your attention on correctness.

Worked example: from schema to running server

  1. Write the proto above, then generate. Confirm orders.pb.go and orders_grpc.pb.go landed in gen/orders/v1.
  2. Implement the adapter. Embed the unimplemented server, hold a *orders.Service, convert on the boundary.
type server struct {
    ordersv1.UnimplementedOrderServiceServer
    svc *orders.Service
}

func (s *server) GetOrder(ctx context.Context, req *ordersv1.GetOrderRequest) (*ordersv1.Order, error) {
    o, err := s.svc.Get(ctx, req.GetId())
    if err != nil {
        return nil, toStatus(err)
    }
    return toProto(o), nil
}
  1. Streaming is a loop. For ListOrders, read from the service and call stream.Send per row; check stream.Context().Err() so a client disconnect stops the query.
func (s *server) ListOrders(req *ordersv1.ListOrdersRequest, stream ordersv1.OrderService_ListOrdersServer) error {
    rows, err := s.svc.List(stream.Context(), req.GetCustomerId())
    if err != nil {
        return toStatus(err)
    }
    for _, o := range rows {
        if err := stream.Send(toProto(o)); err != nil {
            return err
        }
    }
    return nil
}
  1. Wire and serve.
lis, err := net.Listen("tcp", ":9090")
if err != nil {
    return err
}
s := grpc.NewServer(grpc.ChainUnaryInterceptor(loggingInterceptor, recoveryInterceptor))
ordersv1.RegisterOrderServiceServer(s, &server{svc: svc})
return s.Serve(lis)

  1. Shut down. grpc.GracefulStop() stops accepting new RPCs and waits for in-flight ones. Wrap it: run it in a goroutine, and if your shutdown deadline passes, call s.Stop() to cut the stragglers.

  2. Verify. Register server reflection and call it from the shell with grpcurl -plaintext localhost:9090 orders.v1.OrderService/GetOrder. Reflection is a dev convenience; usually leave it off in production.

Discussion & reflection prompts

  • Your HTTP handler returns 404 for a missing order. What does the gRPC client see if you forget the mapping, and how long would that take to debug from the client side?
  • Where is the line between "convert" code (toProto, toStatus) and business logic? Which of your current handlers would need refactoring before a second transport could reuse them?
  • Adding a field to a message: what ordering of deploy (server first or client first) is safe, and why does the field number matter more than the name?

Expert corner

  • Validate once, at the core. Protobuf types are not validators; id = "" is still a legal GetOrderRequest. Do the checking in the service so both transports benefit.
  • Do not put proto types in your service package. It feels tidy, then every internal refactor becomes a wire-compatibility decision.
  • Deadlines. gRPC clients set deadlines; propagate them to database/sql with a context timeout or your Postgres queries will outlive the caller.
  • Interceptors need ordering discipline. Recovery first, then logging, then anything auth-related, matching what you did with net/http middleware.
  • Generated code is vendored, versioned, and committed. Regenerate in CI and fail the build if the output differs from what is checked in.

Review deck

[
  { "front": "Why must proto field numbers never be reused or renumbered?", "back": "They are the wire identity of a field; changing them breaks compatibility with deployed clients." },
  { "front": "What must the first value of a proto3 enum be?", "back": "Zero, conventionally named STATUS_UNSPECIFIED, because it is the default." },
  { "front": "What does protoc-gen-go-grpc generate for a service?", "back": "A client, a server interface, RegisterXServer, and UnimplementedXServer." },
  { "front": "Which type must an embedded field be in your gRPC server struct?", "back": "UnimplementedXServer, so the struct still satisfies the interface when the schema adds methods." },
  { "front": "What error do gRPC handlers return?", "back": "A status error built with status.Error(codes.X, msg), never a raw error." },
  { "front": "What happens to a raw Go error returned from a gRPC handler?", "back": "It surfaces to the client as codes.Unknown with no useful classification." },
  { "front": "What is gRPC's equivalent of HTTP middleware?", "back": "Interceptors, unary and stream, chained with grpc.ChainUnaryInterceptor." },
  { "front": "What does grpc.GracefulStop do?", "back": "Stops accepting new RPCs and waits for in-flight ones to complete." },
  { "front": "Two options for serving HTTP and gRPC from one process?", "back": "Separate ports (simple), or one port via cmux or grpc-gateway." },
  { "front": "What does option go_package control?", "back": "The import path and package name of the generated Go code." },
  { "front": "Where should transport-agnostic business logic live?", "back": "In a service package that both the HTTP handler and the gRPC impl call." },
  { "front": "What does req.GetId() do when req is nil?", "back": "Returns the zero value safely, the getters generated by protoc-gen-go are nil-safe." },
  { "front": "How do you call a gRPC server from a shell?", "back": "grpcurl, provided server reflection is registered on the server." }
]

Summary & next steps

The mental model to keep: one core service, two thin edges, one contract file. Protobuf owns the wire format, protoc owns the boilerplate, and your service package stays free of both. Get the error mapping and the adapter thin-ness right now and adding RPCs later becomes mechanical.

Next up: your gRPC path now has a second transport competing for the same database and CPU, and you have no data on which one is slow. The next lesson puts go test -bench and pprof on it, so you find real hotspots instead of guessing.