CSE 124 Another gRPC Example
2017 October 31: Another gRPC Example

Review: What is gRPC?

gRPC, as the name implies, is a remote procedure call (RPC) framework. We know that RPC frameworks allow applications to invoke operations on remote servers in an ad-hoc fashion. More specifically, an RPC has an explicit request-response framework, where a client makes an explicit request, and receives a tailored response. Contrast this with an HTTP server, which presents a set of resources to a client that have a predefined set of actions they can be used with.

gRPC was originally developed by Google as an internal project to allow easy communication and development of distributed systems and services used within the company. While certainly not the only framework to exist, gRPC is now widely used in many parts of both industry and academia, and is actively developed by users both inside and outside of Google.

RPC frameworks generally work by taking a specification of the RPCs you want to implement, including inputs and outputs, optional arguments, serialization order, and more.

In gRPC, you define a protobuf file that specifies one or more services, which can contain one or more RPCs. RPCs themselves work via message types defined in the protobuf file, which contain one or more base types (such as ints and strings). You then run a protobuf compiler which generates code in the language you specify in order to create a server and/or client for any of the services you defined.

Why do I need an RPC framework?

You can roll your own RPCs just by having a server listen for incoming connections, and scanning the input to determine the function they want to invoke, followed by whatever serialized inputs are required for that function. For example, say we want to invoke an RPC implementing the function int add(int a, int b). The C++ code might look something like this:

Client:
int requestAdd(int a, int b, int server_skt){
    int tmp;
    // Specify to the server we want to perform the add RPC
    tmp = htons(rpc_msg_types.ADD);
    send(server_skt, &tmp, sizeof(int), 0);
    // Send the actual values
    tmp = htonl(a);
    send(server_skt, &tmp, sizeof(int), 0);
    tmp = htonl(b);
    send(server_skt, &tmp, sizeof(int), 0);
    // Get the response
    recv(server_skt, &tmp, sizeof(int), 0);
    return ntohl(tmp);
}

Server:
int handleClient(int client_skt){
    // Figure out what the client wants to do
    int msg_type;
    recv(client_skt, &msg_type, sizeof(int), 0);

    switch(msg_type){
        ...
        case rpc_msg_types.ADD:
            // Call the appropriate handler
            return handleAdd(int client_skt);
            break;
        ...
    }
    ...
}

int handleAdd(int client_skt){
    int tmp, a, b;
    // Get the values from the client
    recv(client_skt, &tmp, sizeof(int), 0);
    a = ntohl(tmp);
    recv(client_skt, &tmp, sizeof(int), 0);
    b = ntohl(tmp);
    // Create and send the response
    tmp = htonl(a + b);
    send(client_skt, &tmp, sizeof(int), 0);
    return 0;
}

At first glance, this doesn’t look too bad. But, consider the following:

  • Duplicated code. We had to define the order of arguments across two separate code blocks in two completely separate files, and if we want to change the RPC in the future we have to edit both sections of code.
  • Difficult to understand the whole system. If a new developer wants to come in and work on this and other RPCs, they’ll have to read through source code containing a ton of system calls just to figure out the RPC arguments, serialization order, and maybe more!
  • Wasted data. Savvy students might notice that we’re wasting bytes here; do we really need four bytes to encode the message type? There’s no way our server is going to support millions of RPCs! Of course, it’s easy to fix this, but ensuring all of your RPCs are optimized is an arduous task.

An RPC framework provides an easy-to-understand interface for writing and implementing any number of RPCs for a system. By defining all of the RPCs in a central, easy-to-read file that works across all programming languages supported by the framework, developers can easily add or remove features or port the system to another language entirely. It also allows for some more advanced software engineering paradigms, such as backwards compatibility.

Converting to gRPC

Let’s look at what the above C++ code would look like in gRPC:

Protobuf:
service MyService {
    ...
    // Add two numbers together
    rpc Add (AddRequest) return (AddReply) {}
    ...
}

message AddRequest {
    int a = 1;
    int b = 2;
}

message AddReply {
    int sum = 1;
}

Client:
int requestAdd(int a, int b, MyService::Stub* stub_){
    AddRequest request;
    AddReply reply;
    ClientContext context;
    // Build the request
    request.set_a(a);
    request.set_b(b);
    // Send the request and get a reply
    Status status = stub_->Add(&context, request, &reply);
    return reply.sum();
}

Server:
class MyServiceImpl final : public MyService::Service {
    ...
    Status Add(ServerContext* context, const AddRequest* request,
               AddReply* reply) override {
        reply->set_sum(request->a() + request->b());
        return Status::OK;
    }
    ...
}

We can see that we now have pre-generated request and reply objects, and all we have to do is call their member functions and the RPC is ready to go!

Let’s quickly step through each part of this code, starting with the protobuf file. The protobuf defines the service we want, and any RPCs it must support. It also defines any message types used by these RPCs.

  • service MyService: This declares we want to make a new service, called MyService. We define what RPCs this service supports inside of this block.
  • rpc Add (AddRequest) return (AddReply) {}: This creates the Add RPC. The request message is AddRequest, and the response message is AddReply.
  • message AddRequest: Here we create a new message type, called AddRequest.
  • int a = 1;: We create a new value in the AddRequest message, called a. We also state that it should be the first value serialized when it is transmitted on the wire.
  • int b = 2;: Now we create the value b. It’s the second message serialized when an AddRequest is sent over the wire.
  • message AddReply: Similar to AddRequest, we create a new message for the reply for the Add RPC.

Now, we look at the client:

  • AddRequest request: We create a new AddRequest message to use for the RPC.
  • AddReply reply: We create a new AddReply message to use for the RPC.
  • ClientContext context: Context objects can be used in RPC to support multi-transaction RPCs by maintaining client information and other necessary variables. We don’t need that here, so we’ll just create a blank Context.
  • request.set_a(a): We put the actual value given to us in the request.
  • Status status = stub_->Add(&context, request, &reply): Tell the stub object to perform an Add RPC. A gRPC stub can be thought of like a socket- a client uses an IP and port to create a Stub object it uses to execute RPCs on a server. Unlike sockets though, it doesn’t actually represent a continuously active connection; gRPC creates a new connection for every RPC. The RPC returns a Status object, which can be used to see if there were any errors while executing the RPC (connection errors, server failures, etc).
  • return reply.sum(): We get the value we want out of the reply and return it to the caller.

Finally, the server:

  • class MyServiceImpl final : public MyService::Service: In gRPC, servers override an interface representing the service they implement. This forces the server to support every RPC defined in the service, and provides nice compile-time checking in case something is wrong.
  • Status Add(ServerContext* context, const AddRequest* request, AddReply* reply) override: Look familiar? These are the same arguments the client gave when calling the Add RPC.
  • reply->set_sum(request->a() + request->b()) : Set the sum variable in the reply to be the sum of the two variables in the request. Note that we need a function call to access the variables in the request (this is specific to C++, though).
  • return Status::OK: This is the status that is returned to the client.

HelloWorld example

We’ve extended the Greeter service provided as an example in the gRPC source code to show off some extra features of gRPC. We provide the example in both Python and Java. Note that the protobuf file is the same for both languages: this means you can use the Python client with the Java server, or vice versa!

The code is available on ieng6 in the public directory. Run this command to copy it to your home directory:

cp -r $PUBLIC/another_grpc_example ~/another_grpc_example

Use your copy when working with the example.

To use the example in Python, run the script compile_protobuf.sh in the py/ folder. Then run python greeter_<client/server>.py. Make sure to run the server first.

To use the Java example, run ant grpc_helloworld to run both the client and server, or use ant grpc_helloworld_<client/server> to run them individually.

The example executes two kinds of RPCs: a single message, and a batched set of messages.

Setup

First, we need to set up gRPC and create the server and client stub.

Python

To set up gRPC in Python, we need to import two generated files. The server creates a class extending the interface generated by the protobuf compiler, and then defines the RPC functions in the service. It creates a worker pool that is used to serve requests.

The client and server create insecure sockets, since we don’t need authentication. Then, the client creates a stub by giving it the channel object, and we can run RPCs.

Java

The server creates a worker pool and registers our Greeter server implementation with a new gRPC server object. We add a shutdown hook to make things clean up nicely when the process closes. The actual Greeter implementation is done by extending an implementation class generated by gRPC and overriding the RPC functions with our own implementation. No other functions need be overridden.

The client uses a ManagedChannelBuilder object to create a channel to the server. It creates a blocking stub, which is unique to the Java implementation of gRPC. Non-blocking stubs are fairly complex and not needed for the course, so we won’t cover them here.

Note that Java uses camelcase for RPC function names.

Single message RPC

We create a basic HelloRequest with both the name and id fields set, and give it to the SayHello RPC method in the stub. The client waits for the server to send its response, and then prints it out. The server simply reads the fields in the request, and creates a response string using them.

Python

As we can see, populating values inside messages in Python is handled via keyword arguments. In Python, the server returns the response message directly. This matches what the client sees; the RPC function returns the response object.

Note the lack of a status or error object: in Python, gRPC leverages the exception framework to signal errors. Servers throw exceptions if something is wrong, and then these are in turn thrown by the stub object at the client.

Java

Message objects in Java require a Builder, which basically represents a to-be-constructed message. The message class defines a newBuilder() method used to create the builder. From there, message fields can be populated. The final request object is created by calling build(). From there, the RPC is easily executed by calling the appropriate function on the synchronous stub object, and then using the getter function on the returned response.

The server isn’t given the response directly: instead, it’s given a StreamObserver object, which essentially represents the flow of messages in one direction: in this case, from the client to the server.

Batched RPC

This uses a new message type for the request, which uses the repeated keyword to specify that the value is an array of the specified type.

Python

The client just sends the array as-is, and the server iterates over it to create the same response object we’ve been seeing.

Java

Here we create a sequence of replies and add them into a message as they’re created. Note how the repeated message field of the HelloRequestMany message acts like a List object- that’s because it is actually a List<HelloRequest>, meaning we can use functions like add, set, and get to interact with the message field.

We see how powerful this can be in the server: it gets the entire list object, constructs a stream, and then can use map and join functions to easily construct the response message.

FAQ

Can I use Protobuf version 2?

No, sorry. Using multiple protobuf versions makes things really complicated on our end, so we can’t support it for this project.

How can I make a value in a message required/optional?

As of protobuf version 3, all values in a message are optional. Services implementing RPCs that require a value to be set in a message must check for themselves if it’s set (by default they’re set to 0/false/etc), and return an error if it’s not.

Why do I have to specify the serialization order of values in a message?

The technical term for these numbers are “tags”, and allow for backwards compatibility. For example, say you wanted to allow the Add RPC to support adding three numbers together:

message AddRequest {
    int a = 1;
    int b = 2;
    int c = 3;
}

Servers/clients that haven’t been recompiled with the new protobuf will still work with clients/servers that have, as the older version will still place a and b in the same location of the message since their tags didn’t change.

How do servers handle concurrent requests?

In every language, gRPC will launch a new thread for each incoming request, up to a limit. This means that services must leverage synchronization primitves for more complex services. The method to change the limit on the number of active concurrent requests depends on the language. In both Python and Java, it depends on the size of the worker pool created during setup.

I’m getting “port in use” errors!

Since the examples use a hardcoded default port, you’ll need to change it in order to avoid conflicts with other users also testing the example on ieng6.