grpc同时提供restful api
概述根据google cloud的API设计指南：
对比rest api和rpc： 2010 年，大约 74% 的公共网络 API 是 HTTP REST，虽然 HTTP/JSON API 在互联网上非常流行，但它们承载的流量比传统的 RPC API 要小，视频内容，数据中心内部 RPC API来承载大多数网络流量
在实际使用中，人们会出于不同目的选择 RPC API 和 HTTP/JSON API，理想情况下，API 平台应该为所有类型的 API 提供最佳支持
而藏宝阁的使用场景，服务端调用的接口如果统一使用rpc， 也可能存在需要提供rest api的可能，比如同时提供给游戏/大神/cc等调用，grpc不一定方便对接
Google API 和 Cloud Endpoints gRPC API 使用 HTTP 映射功能进行 JSON/HTTP 到 Protocol Buffers/RPC 的转码： gRPC Transcoding
gRPC Transcoding的映射定义demo, 留意option(google.api.http)配置:
// Returns a specific bookstore shelf.rpc GetShelf(GetShelfRequest) returns (Shelf) {// Client example - returns the first shelf:// curl http://DOMAIN_NAME/v1/shelves/1option (google.api.http) = { get: "/v1/shelves/{shelf}" };}...// Request message for GetShelf method.message GetShelfRequest {// The ID of the shelf resource to retrieve.int64 shelf = 1;}
也可以使用独立的yaml文件配置映射，不过推荐在proto文件中定义
支持gRPC Transcoding的系统包括：Google APIs, Cloud Endpoints, gRPC Gateway, and Envoy proxy
Google APIs我们可以无视, 下面对其他3种方式做个说明
Cloud Endpoints
Cloud Endpoints是google cloud提供的服务，部署结构如下图所示，
image.png
留意图中红框的部分是部署在google cloud的Endpoints服务，因为依赖google cloud提供的服务，我们也无法使用
gRPC Gateway
使用这个grpc服务来举例: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/endpoints/bookstore-grpc
如下命令启动gprc server， 默认端口8000
python bookstore_server.py
gRPC Gateway 完全是go的实现，以go grpc为基础，需要生成go grpc的stub，在额外生成grpc gateway的stub
所以需要安装go的环境, 安装protobuf
之后 go install:
google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
google.golang.org/protobuf/cmd/protoc-gen-go@latest
在 bookstore-grpc 创建gw目录，生成go gpc/grpc gateway stub及编写go代码
mkdir gwcd gwgo mod init cbg/demorpc/gw
使用 http_bookstore.proto, 生成stub代码
留意需要需要修改下http_bookstore.proto, 加上option go_package = "cbg/demorpc/gw/pb";
执行如下命令
cp ../http_bookstore.proto ./# 留意修改加上option go_package = "cbg/demorpc/gw/pb";# 因为http_bookstore.proto依赖googleapi的proto filegit clone https://github.com/googleapis/googleapis.git# 生成stub代码# mkdir pbprotoc -I . -I ./googleapis/ --grpc-gateway_out ./pb \ --grpc-gateway_opt paths=source_relative \ --grpc-gateway_opt generate_unbound_methods=true \ --go_out ./pb --go_opt paths=source_relative \ --go-grpc_out ./pb --go-grpc_opt paths=source_relative \ http_bookstore.proto
创建main.go文件，内容如下:
package mainimport ("context""flag""net/http""github.com/golang/glog""github.com/grpc-ecosystem/grpc-gateway/v2/runtime""google.golang.org/grpc""google.golang.org/grpc/credentials/insecure"pb "cbg/demorpc/gw/pb")var (// command-line options:// gRPC server endpointgrpcServerEndpoint = flag.String("grpc-server-endpoint","localhost:8000", "gRPC server endpoint"))func run() error {ctx := context.Background()ctx, cancel := context.WithCancel(ctx)defer cancel()// Register gRPC server endpoint// Note: Make sure the gRPC server is running properly and accessiblemux := runtime.NewServeMux()opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}err := pb.RegisterBookstoreHandlerFromEndpoint(ctx, mux,*grpcServerEndpoint, opts)if err != nil { return err}// Start HTTP server (and proxy calls to gRPC server endpoint)return http.ListenAndServe(":8081", mux)}func main() {flag.Parse()defer glog.Flush()if err := run(); err != nil { glog.Fatal(err)}}
执行如下命令，启动gateway服务， 监听8081端口:
go mod tidygo run main.go
查看效果:
$ curl 127.0.0.1:8081/v1/shelves{"shelves":[{"id":"1", "theme":"Fiction"}, {"id":"2", "theme":"Fantasy"}]}Envoy
安装envoy: https://www.envoyproxy.io/docs/envoy/latest/start/install#tools-images
生成descriptor file
mkdir envoycd envoycp ../http_bookstore.proto ./# 因为http_bookstore.proto依赖googleapi的proto filegit clone https://github.com/googleapis/googleapis.git# 生成 descriptor fileprotoc -o api_descriptor.pb -I ./googleapis/ -I ./--include_importshttp_bookstore.proto
创建conig.yaml, 添加如下内容:
static_resources:listeners:- name: listener1 address: socket_address: {address: 0.0.0.0, port_value: 8080} filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: grpc_json codec_type: AUTO route_config: name: local_route virtual_hosts: - name: local_service domains: ["*"] routes: # NOTE: by default, matching happens based on the gRPC route, and not on the incoming request path. # Reference: https://envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter#route-configs-for-transcoded-requests - match: {prefix: "/endpoints.examples.bookstore.Bookstore"} route: {cluster: grpc, timeout: 60s} http_filters: - name: envoy.filters.http.grpc_json_transcoder typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder proto_descriptor: "/home/gpx/demorpc/envoy/api_descriptor.pb" services: ["endpoints.examples.bookstore.Bookstore"] print_options: add_whitespace: true always_print_primitive_fields: true always_print_enums_as_ints: false preserve_proto_field_names: false - name: envoy.filters.http.routerclusters:- name: grpc type: STATIC lb_policy: ROUND_ROBIN connect_timeout: 2s dns_lookup_family: V4_ONLY typed_extension_protocol_options: envoy.extensions.upstreams.http.v3.HttpProtocolOptions: "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions explicit_http_config: http2_protocol_options: {} load_assignment: cluster_name: grpc endpoints: - lb_endpoints: - endpoint: address: socket_address: address: 127.0.0.1 port_value: 8000
启动envoy
envoy -c config.yaml
验证结果:
$ curl 127.0.0.1:8080/v1/shelves{ "shelves": [{ "id": "1", "theme": "Fiction"},{ "id": "2", "theme": "Fantasy"} ]}istio
因为istio使用的sidecar也是envoy，所以也可以配置istio的sidecar做这样的转码
istio并为提供直接设置grpc转码的配置，但istio开了直接patch sidecar的envoy配置的扣子： EnvoyFilter，可以实现上面Envoy的配置
同样使用bookstore的例子
部署如下的deploy + service:
apiVersion: apps/v1kind: Deploymentmetadata:name: bookstorelabels: app: bookstorespec:replicas: 1selector: matchLabels: app: bookstoretemplate: metadata: labels: app: bookstore spec: containers: - name: bookstore image: dockerhub.nie.netease.com/xiebaogong/bookstore:v1 args: - "--port" - "9090" ports: - containerPort: 9090---apiVersion: v1kind: Servicemetadata:name: bookstorelabels: app: bookstore service: bookstorespec:ports:- port: 9091 targetPort: 9090 name: grpc appProtocol: grpcselector: app: bookstore
进入bookstore容器，验证grpc服务部署成功
# python bookstore_client.py--host=bookstore --port=9091ListShelves: shelves {id: 1theme: "Fiction"}shelves {id: 2theme: "Fantasy"}
配置EnvoyFilter
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: bookstore-grpc-jsonspec:workloadSelector: labels: app: bookstoreconfigPatches:- applyTo: HTTP_FILTER match: context: SIDECAR_INBOUND listener: portNumber: 9090 filterChain: filter: name: "envoy.filters.network.http_connection_manager" subFilter: name: "envoy.filters.http.router" patch: operation: INSERT_BEFORE value: name: envoy.filters.http.grpc_json_transcoder typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder proto_descriptor_bin: Ctd4ChVnb29nbGUvYXBpL2h0dH.... print_options: add_whitespace: true always_print_primitive_fields: true always_print_enums_as_ints: false preserve_proto_field_names: false
留意上面的 proto_descriptor_bin配置， 取值内容是上面Envoy demo中的api_descriptor.pb转成base64，copy上去
使用 kubectl apply -f filter.yaml 部署上述配置之后，可在k8s的任意容器中验证结果:
$ curl bookstore:9091/v1/shelves{ "shelves": [{ "id": "1", "theme": "Fiction"},{ "id": "2", "theme": "Fantasy"} ]
因为EnvoyFilter直接暴露了Envoy的配置，在不通版本下不太一样，上面的测试是在istio 1.11上完成的，在istio 1.4下配置不通，如下：
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: bookstore-grpc-jsonnamespace: cbg-xie1spec:workloadSelector: labels: app: bookstoreconfigPatches:- applyTo: HTTP_FILTER match: context: SIDECAR_INBOUND listener: portNumber: 9390 filterChain: filter: name: "envoy.http_connection_manager" subFilter: name: "envoy.router" patch: operation: INSERT_BEFORE value: name: envoy.grpc_json_transcoder config: auto_mapping: true proto_descriptor_bin: Ctd4ChVnb29nbGUvYXB... services: ["endpoints.examples.bookstore.Bookstore"] print_options: add_whitespace: true always_print_primitive_fields: true always_print_enums_as_ints: false preserve_proto_field_names: false
改配置部署成功，并通过istioctl proxy-config listener 查看，配置确实写入了， 但是访问并不生效，怀疑是bug
总结
gRPC Gateway, Envoy, istio配置这3种方式对比下来各有优劣
gRPC Gateway需要引入go，对于非go语言，引入go，流程有些复杂，不太友好
Envoy 再引入这么一个专门的proxy，比较重，也可以考虑和业务容器分开单独部署
istio配置，不同版本的兼容性不太好
页:
[1]