Getting grpc-web Demo Running

Recently researching the technical feasibility of blockchain wallet plugins, which involves gRPC technology. The official repo has a demo, but getting it running normally has some barriers, so I’m summarizing here.

Installation

protobuf

# Recommended installation method, no need for additional compilation and environment variable configuration, https://formulae.brew.sh/formula/protobuf
brew install protobuf

# Verify successful installation
protoc --version

protoc-gen-grpc-web

To generate Web client code for PB-defined services, this tool needs to be installed

protoc-gen-grpc-web needs to be manually installed. Visit https://github.com/grpc/grpc-web/releases

For example, for my Intel Mac, download protoc-gen-grpc-web-1.3.0-darwin-x86https://grpc.io/_64

Execute the following commands

sudo mv ~/Downloads/protoc-gen-grpc-web-1.3.0-darwin-x86_64 \
  /usr/local/bin/protoc-gen-grpc-web

sudo chmod +x /usr/local/bin/protoc-gen-grpc-web

Same as above, if typing protoc-gen-grpc-web in the terminal shows recognition, it means installation was successful.

protoc-gen-grpc-web installation is too troublesome. Setting a flag here to maintain an npm package to make it easier for everyone to download and install.

Brew Usage Supplement

1. brew search package can search for what version sources a package has
2. brew install package@versionNumber can directly install the target version

Testing

Generate corresponding JS code according to the defined PB format file.

$ protoc -I=. helloworld.proto --js_out=import_style=commonjs:. --grpc-web_out=import_style=commonjs,mode=grpcwebtext:.

Build and Startup

After completing the above steps, environment-level issues are resolved. Then just follow the startup steps in the readme step by step.

Note

If the development machine is a Mac platform, the following two modifications are needed:

• envoy.yaml configuration modification

  address: host.docker.internal
  

  Note: only this line needs to be modified

• Docker startup command adjustment - remove network setting

  docker run -d -v "$(pwd)"/envoy.yaml:/etc/envoy/envoy.yaml:ro \
  -p 8080:8080 -p 9901:9901 envoyproxy/envoy:v1.20.0
  

Error

Http response at 400 or 500 level

After startup, if you encounter this error, it means network services are not communicating. Check if services started normally, for example, if the above proxy configuration error caused container communication issues. If an envoy proxy node goes down, it can also cause this error.

Common Issues

Encountered some problems when using grpc-web. The official documentation and community are not very active, so many pitfalls have to be discovered through personal experience. Here I summarize some common issues.

grpc: received message larger than max (1094796627 vs. 4194304)

When encountering this error, there are two possible situations

Size Settings

1. grpc.max_send_message_length setting, unit is bytes, -1 means no limit
2. Server/client settings are independent of each other

Official configuration see https://grpc.github.io/grpc/core/group__grpc__arg__keys.html#gab4defdabac3610ef8a5946848592458c

–grpc-web_out

Determine whether the WEB output mode is grpcwebtext or grpcweb. If it’s grpcweb but ultimately sends binary, it will report received message larger than max, up to 1G.

$ protoc -I=. helloworld.proto --js_out=import_style=commonjs:. --grpc-web_out=import_style=commonjs,mode=grpcweb:.

413 (Request Entity Too Large)

1. Check nginx client_max_body_size configuration
2. Confirm grpc-server side grpc.max_receive_message_length configuration

Custom Header Fields

Sometimes when sending requests, you want custom headers, for example to implement dynamic proxy for gRPC services.

Implementation requires 2 parts of configuration

var client = new GreeterClient('http://' + window.location.hostname + ':9090', null, null);

// simple unary call
var request = new HelloRequest();
request.setName('World');

client.sayHello(request, {
  'X-Grpc-node': 'hello world' // 自定义头部字段
}, (err, response) => {
  if (err) {
    console.log(`Unexpected error for sayHello: code = ${err.code}` + `, message = "${err.message}"`);
  } else {
    console.log(response.getMessage());
  }
});

nginx configuration to read custom fields for dynamic proxy

server {
    listen       9080;
    server_name  localhost;
    underscores_in_headers on; # Controls custom header field reading, needs to be enabled

    location / {
        grpc_set_header     Content-Type application/grpc;
        grpc_pass    grpc://$http_x_grpc_node; # Note http prefix and underscores
        }
        
    }    

Nginx Dynamic Proxy for gRPC Services

    location / {
        grpc_set_header     Content-Type application/grpc;
        grpc_pass    grpc://$http_x_grpc_node;

       ...
    }

Note that http_x_grpc_node is the HTTP custom header field value. If you need to proxy SSL services, it should be grpcs://, with specific configuration as follows

Proxy gRPCs

  location / {
        grpc_set_header     Content-Type application/grpc;
        grpc_pass    grpcs://$http_x_grpc_node;
        
        # Note: file names support variables
			  grpc_ssl_certificate /var/www/ssl/$http_x_grpc_ssl_cert;
        grpc_ssl_certificate_key /var/www/ssl/$http_x_grpc_ssl_cert_key;
        
       ...
    }

Note

• grpc_ssl_certificate, grpc_ssl_certificate_key file values support variables, so different servers can be dynamically proxied.
• grpc_pass/grpc_ssl_certificate these directives do not support being written in if conditional blocks, so dynamic switching between grpc and grpcs proxy cannot be achieved.

Supplement

1. When determining if the RPC server started normally, you can use telnet to test port connectivity, for example telnet 192.168.1.203 16301

Reference Documentation

• https://www.npmjs.com/package/google-protobuf
• https://grpc.io/docs/protoc-installation/
• https://github.com/grpc/grpc-web
• https://stackoverflow.com/questions/9421933/cross-origin-xmlhttprequest-in-chrome-extensions
• https://stevenocean.github.io/2020/06/20/nginx-grpc-web-go.html

Final Thoughts

After initial attempts, I have the following understandings:

• gRPC defaults to using Protocol Buffers to define interfaces. Therefore, using gRPC involves dealing with PB
• gRPC Web is the JavaScript implementation of the gRPC protocol in browsers, but to actually connect to gRPC servers, proxy services like Envoy/Nginx are needed
• gRPC Web enables the web to indirectly communicate with backend gRPC services