Go protobuf upgrade — github.com/golang/protobuf to google.golang.org/protobuf

Contents

  1. 1. Round one
  2. 2. Round two
  3. 3. Round three

A recent protobuf dependency bump looked routine—until CI and protoc disagreed.

Dependabot suggested moving github.com/golang/protobuf from v1.3.5 to v1.4.0.

Round one

I expected a straight version bump. CI failed: tests called generated XXX_Size() to measure message size; on v1.4.0 that path panics.

Compare old vs new generated code. Proto source: official helloworld.proto.

Generate v1.3.5 output:

1
2
3
4
curl -O https://raw.githubusercontent.com/grpc/grpc-go/master/examples/helloworld/helloworld/helloworld.proto
brew install protobuf
GO111MODULE=on go get -u github.com/golang/protobuf/protoc-gen-go@v1.3.5
protoc --go_out=plugins=grpc:. helloworld.proto

Swap in protoc-gen-go@v1.4.0 and regenerate. Search for XXX_Size.

Old codegen still wires it through InternalMessageInfo:

1
2
3
4
5
6
7
8
// helloword.pb.go
func (m *HelloRequest) XXX_Size() int {
return xxx_messageInfo_HelloRequest.Size(m)
}
var xxx_messageInfo_HelloRequest proto.InternalMessageInfo

// github.com/golang/protobuf@v1.3.5/proto/table_marshal.go
func (a *InternalMessageInfo) Size(msg Message) int {

New .pb.go no longer defines InternalMessageInfo; the type still exists in the module as deprecated:

1
2
3
4
5
6
// github.com/golang/protobuf@v1.4.0/proto/deprecated.go
// Deprecated: Do not use.
type InternalMessageInfo struct{}

func (*InternalMessageInfo) Size(Message) int { panic("not implemented") }
// Same for DiscardUnknown, Marshal, Merge, Unmarshal

Use proto.Size(m Message) int instead. InternalMessageInfo became the protoiface.MessageV1 interface alias. Release notes: v1.20 generated code.

Swap the call sites—round one done.

Done for now

Round two

Docs for github.com/golang/protobuf v1.4.0 say maintenance moves to google.golang.org/protobuf. Fine—upgrade the gRPC plugin too.

1
GO111MODULE=on go get -u google.golang.org/protobuf/cmd/protoc-gen-go@v1.21.0

Regenerate. The new plugin wants:

option go_package = ".;helloworld";

Then failure:

1
--go_out: protoc-gen-go: plugins are not supported; use 'protoc --go-grpc_out=...' to generate gRPC

Try the new flag:

1
protoc  --go-grpc_out=. helloworld.proto

Still failing:

1
2
3
protoc-gen-go-grpc: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--go-grpc_out: protoc-gen-go-grpc: Plugin failed with status code 1.

Frustration

protoc-gen-go-grpc is a separate binary. gRPC stubs used to come from protoc --go_out=plugins=grpc:.

Install attempt:

1
GO111MODULE=on go get google.golang.org/protobuf/cmd/protoc-gen-go-grpc

Another miss:

1
go get google.golang.org/protobuf/cmd/protoc-gen-go-grpc: module google.golang.org/protobuf@upgrade found (v1.21.0), but does not contain package google.golang.org/protobuf/cmd/protoc-gen-go-grpc

Frustration again

pkg.go.dev lists protoc-gen-go-grpc; the protobuf-go repo did not ship it yet. Issue thread: plugins are not supported: grpc.

google.golang.org/protobuf shipped new protoc-gen-go without the gRPC plugin; protoc-gen-go-grpc was still in review (not stable at the time).

Awkward rollout

Official workaround: keep generating gRPC with the old github.com/golang/protobuf toolchain—or reuse gengogrpc from the in-flight plugin; see comment.

Round three

Runtime on google.golang.org/protobuf@v1.21.0, codegen with github.com/golang/protobuf/protoc-gen-go@v1.4.0. Regenerate—finally green.

Peace

Generated imports looked split:

1
2
3
4
5
6
proto "github.com/golang/protobuf/proto"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"

Still pulling both protobuf module families felt wrong. Unless you need the new APIs now, waiting is reasonable.

The breaking changes were intentional: first-class reflection and new tooling—see v1.21.0 release.

Questions? Leave a comment below or email: newbvirgil@gmail.com