Building a secure API with gRPC

A Google remote procedure call (gRPC) is Google’s open source version of the remote procedure call (RPC) framework. It’s a communication protocol leveraging HTTP/2 and protocol buffer (protobuf) technologies. gRPC enables a remote client or server to communicate with another server by simply calling the receiving server’s function as if it were local. This makes communicating and transferring large data sets between client and server much easier in distributed systems.

Like other RPC systems, gRPC defines a service. It specifies its methods and return types using protobuf — a Google serialization and deserialization protocol — to enable the easy definition of services and auto-generation of client libraries. gRPC uses this protocol, currently on version 3, as its interface definition language and serialization toolset.

For most modern applications, gRPC is an excellent choice for its outstanding support of all data types. It’s best suited for heavy data, like streaming data, and could be overkill for simple applications where large data transfers are of little concern.

This article will demonstrate how to use gRPC via a client and server-like communication between two Node.js applications. We’ll also highlight some safety measures when using gRPC as the communication mechanism in your services.

Tutorial prerequisites

The tutorial requires you to have OpenSSL and Node.js (version 4.0 or later) installed on your PC. Having a basic understanding of Node.js and JavaScript is essential. You will also need to ensure that your working environment has administrative privileges.

Setting up the Node.js project

First, to set the application’s folder structure, create a folder called event-app-node-grpc and initialize a Node.js project using npm by typing the following commands:

1#bash
2$ mkdir event-app-node-grpc
3$ cd event-app-node-grpc
4$ npm init -y

Having initiated your application, build out the following folder structure for the application. You can access the complete working code used in this tutorial on GitHub:

1Event-app-node-grpc
2client
3app.js
4index.js
5server
6index.js
7scripts
8generate-certs.sh
9events.proto
10README.md

Installing packages

From the terminal, navigate to the root directory of your application. Install the following packages using the npm install command, as shown in the following code snippet:

1#bash
2$ npm install express @grpc/grpc-js @grpc/proto-loader

Let’s go over the packages you just installed in the above code snippet:

• Express is your application HTTP server.

• @grpc/grpc-js is a gRPC library for Nodejs. It enables us to create a gRPC service in the Node.js runtime.

• @grpc/proto-loader is a package needed to load protobuf files for use with gRPC. It uses the version 3 package of protobuf.js.

After installing the packages above, open the package.json file and add the following extra configurations to the scripts tags as shown in the code snippet below:

1#package.json
2
3"scripts": {
4   "start": "node server/index.js",
5   "generate:certs": "./scripts/generate-certs.sh"
6 },

These additional configurations shown in the code snippet above are for the application’s runtime configuration and SSL certificate generation. When these configurations have been added, your updated package.json file will look similar to the code snippet below:

1#package.json(updated)
2
3{
4 "name": "event-app-node-grpc",
5 "version": "1.0.0",
6 "description": "A CRUD application to demonstrate the use of gRPC with NodeJS",
7 "main": "server/index.js",
8 "scripts": {
9   "start": "node server/index.js",
10   "generate:certs": "./scripts/generate-certs.sh"
11 },
12 "author": "(author name here)",
13 "license": "ISC",
14 "dependencies": {
15   "@grpc/proto-loader": "^0.6.12",
16   "express": "^4.18.1",
17   "@grpc/grpc-js": "^1.6.12
18 }
19}

The code snippet above shows the updated package.json file after the addition of the application runtime configuration and SSL certificate generation commands to the scripts tag.

Defining the protocol buffer

This tutorial illustrates how to use gRPC in a simple event tracking application. This demo application takes event details and saves them in an in-memory database while allowing the event data to be updated, fetched, and deleted.

In gRPC applications, the service interface and required payloads are in a protobuf file to enable communication between different applications. protobuf files have a .proto extension, as illustrated in our project setup schema.

Now, in the root directory of your application, create an events.proto file and add the following code to it. You can look at the project structure schema we defined earlier for reference.

1#events.proto
2
3syntax = "proto3";
4
5service EventService {
6   rpc GetAllEvents (Empty) returns (EventList) {}
7   rpc GetEvent (EventId) returns (Event) {}
8   rpc CreateEvent (Event) returns (Event) {}
9   rpc UpdateEvent (Event) returns (Event) {}
10   rpc DeleteEvent (EventId) returns (Empty) {}
11}
12
13message Empty {}
14
15message Event {
16   string id = 1;
17   string name = 2;
18   string description = 3;
19   string location = 4;
20   string duration = 5;
21   int32 lucky_number = 6;
22   string status = 7;
23}
24
25message EventList {
26   repeated Event events = 1;
27}
28
29message EventId {
30   string id = 1;
31}

In the above proto definition code snippets, the following happened, we first specified the protocol buffer version using the syntax = "proto3" definition followed by the protocol service definition.

Next, in the protocol event service description, we created a service named EventService. We then created rpc functions within this service alongside their required parameters and expected return values. You can define as many services as your application needs, but, for simplicity, we only define one.

We also defined the data types for the rpc function in the EventService definition and the return values from the gRPC unique field numbering system. This describes the number of bytes used during encoding. You can see more details in protobuf’s official documentation.

Creating the gRPC server

Following our folder structure above, create a server folder in the root directory of your application, then create an index.js file in this server folder. Paste the following code snippet into your newly created server/index.js file:

1#server/index.js
2
3const PROTO_PATH = "./events.proto";
4
5let grpc = require("@grpc/grpc-js");
6let protoLoader = require("@grpc/proto-loader");
7
8let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
9   keepCase: true,
10   longs: String,
11   enums: String,
12   arrays: true
13});
14
15let eventsProto = grpc.loadPackageDefinition(packageDefinition);

In the above code snippet, we imported the events.proto file we previously defined as the variable PROTO_PATH, and loaded it using the protoLoader library loadSync method. We then saved the proto definitions in the eventsProto variable, which stores all the proto definitions.

Next, add the following code snippet right after the eventsProto variable in the server/index.js file defined earlier.

1const { randomUUID } = require("node:crypto");
2
3const events = [
4   {
5       id: "34415c7c-f82d-4e44-88ca-ae2a1aaa92b7",
6       name: "Birthday Party",
7       description: "27th Birthday in Paris",
8       location: "Paris France",
9       duration: "All Day",
10       lucky_number: 27,
11       status: "Pending"
12   },
13];
14
15const server = new grpc.Server();

In the above code snippet, we required the node:crypto package and its function randomUUID, which is used for generating random unique strings for our event IDs. Since we are using an in-memory database for this tutorial, we’ll define it as an array to store our list of events, and then set up our server instance by calling a new grpc.Server method.

Next, we will register the application services. To do this, add the following code snippet. Place it right after the server variable in the code snippet above:

1server.addService(eventsProto.EventService.service, {
2
3   getAllEvents: (_, callback) => {
4       callback(null, { events });
5   },
6
7   getEvent: (call, callback) => {
8       let event = events.find(n => n.id == call.request.id);
9
10       if (event) {
11           callback(null, event);
12       } else {
13           callback({
14               code: grpc.status.NOT_FOUND,
15               details: "Event Not found"
16           });
17       }
18   },
19
20   createEvent: (call, callback) => {
21       let event = call.request;
22
23       event.id = randomUUID();
24       events.push(event);
25       callback(null, event);
26   },
27
28   updateEvent: (call, callback) => {
29       let existingEvent = events.find(n => n.id == call.request.id);
30
31       if (existingEvent) {
32           existingEvent.name = call.request.name;
33           existingEvent.description = call.request.description;
34           existingEvent.location = call.request.location;
35           existingEvent.duration = call.request.duration;
36           existingEvent.lucky_number = call.request.lucky_number;
37           existingEvent.status = call.request.status;
38           callback(null, existingEvent);
39       } else {
40           callback({
41               code: grpc.status.NOT_FOUND,
42               details: "Event Not found"
43           });
44       }
45   },
46
47   deleteEvent: (call, callback) => {
48       let existingEventIndex = events.findIndex(
49           n => n.id == call.request.id
50       );
51
52       if (existingEventIndex != -1) {
53           events.splice(existingEventIndex, 1);
54           callback(null, {});
55       } else {
56           callback({
57               code: grpc.status.NOT_FOUND,
58               details: "Event Not found"
59           });
60       }
61   }
62});

In the code snippet above, we called the addService method on the gRPC server instance to register the application services, which was basically a create, read, and update operation on the events.

To allow the application server to start, paste the following code snippet right after the addService method from the above code snippet.

1server.bindAsync("127.0.0.1:50051", grpc.ServerCredentials.createInsecure(), (error, port) => {
2console.log(`Server listening at http://127.0.0.1:${port}`);
3server.start();
4});

Creating the gRPC client

Following our folder structure above, create a client folder in the root directory of your application, then create two files in the client folder you just created: an index.js file and an app.js file. Paste the following code snippet into the client/app.js file.

1#client/app.js
2
3const PROTO_PATH = "../events.proto";
4const grpc = require("@grpc/grpc-js");
5const protoLoader = require("@grpc/proto-loader");
6
7let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
8   keepCase: true,
9   longs: String,
10   enums: String,
11   arrays: true
12});
13
14const EventService = grpc.loadPackageDefinition(packageDefinition).EventService;
15const client = new EventService("127.0.0.1:50051", grpc.credentials.createInsecure());
16module.exports = client;

In the code snippet above, we imported the proto definitions we made earlier, loaded it with protoLoader, hooked up the grpc client to the server application’s IP address, and exported the event service using the variable name client. We also attached an SSL certificate to the client for authorization and encryption of client-server communications.

Then, in the client/index.js file, paste the following code snippet:

1#client/index.js
2
3const client = require("./app");
4
5const express = require("express");
6const app = express();
7app.disable('x-powered-by');
8app.use(express.json());
9app.use(express.urlencoded());
10
11app.get("/", (req, res) => {
12   client.getAllEvents(null, (err, data) => {
13       if (!err) {
14           res.status(200).send({
15               data
16           });
17       }
18   });
19});
20
21app.post("/createEvent", (req, res) => {
22
23   let newEvent = {
24       name: req.body.name,
25       description: req.body.age,
26       location: req.body.address,
27       duration: req.body.address,
28       lucky_number: req.body.address,
29       status: req.body.status
30   };
31
32   client.insert(newEvent, (err, data) => {
33       if (err) throw err;
34       res.status(200).send({
35           data,
36           message: 'Event created successfully'
37       });
38   });
39});
40
41app.post("/updateEvent", (req, res) => {
42   let updateEvent = {
43       name: req.body.name,
44       description: req.body.age,
45       location: req.body.address,
46       duration: req.body.address,
47       lucky_number: req.body.address,
48       status: req.body.status
49   };
50
51   client.update(updateEvent, (err, data) => {
52       if (err) throw err;
53
54       res.status(200).send({
55           data,
56           message: 'Event updated successfully'
57       });
58   });
59});
60
61app.delete("/deleteEvent", (req, res) => {
62   client.remove({ id: req.body.eventId }, (err, _) => {
63       if (err) throw err;
64
65       res.status(200).send({
66           message: 'Event deleted successfully'
67       });
68   });
69});
70
71const PORT = process.env.PORT || 50050;
72app.listen(PORT, () => {
73   console.log("Client Server listening to port %d", PORT);
74});

Taking a look at the code snippet above, we imported event-service from the client/app.js file. Then we configured an express server with simple endpoints to manage the creation, update, fetch, and delete events by remotely calling the server application using gRPC techniques.

Test the server and client applications

At this point, we can test our work to make sure we’re on track.

The server

Navigate to the root directory of the project using the terminal and then run the following commands:

1$bash
2$ npm run start

The server application should be live on http://localhost:50051:

1% npm run start
2> event-app-node-grpc@1.0.0 start
3> node server/index.js
4
5Server listening at https://127.0.0.1:50051

The client

Open a new terminal window, navigate to the client folder from the root directory of your application, and then run the following commands:

1$bash
2$ node index

The application should be live on http://localhost:50050:

1$ event-app-node-grpc  cd client
2$ event-app-node-grpc/client node index.js
3
4Client Server listening to port 50050

To test, navigate to localhost:50050 on your browser or use an API testing tool like Postman. You should see the default event we initially added to our events array. Your response should be the same as the screenshot below:

Authenticating and securing the gRPC API

The gRPC protocol supports various authentication mechanisms, making it easy to adapt to new and existing systems. We can implement authentication in gRPC client-server communications using recommended mechanisms like SSL and TLS with or without Google token-based authentication. We can also build custom authentication by merely extending the built-in authentication function in gRPC.

By default, gRPC comes bundled with the following authentication mechanisms:

• SSL and TLS to authenticate the server and encrypt the data exchanged between the client and the server

• ALTS (a mutual transport and authentication protocol engineered by Google) to secure RPC communications for applications running on the Google Cloud Platform (GCP)

• Generic token-based authentication mechanism to attach metadata-based credentials to requests and responses

We’ll implement authentication using SSL for this tutorial, as earlier highlighted in the tutorial introduction, then we will modify our code in the client/app.js and server/index.js files to accommodate this new development.

Generate an SSL certificate with OpenSSL

First, let’s generate an SSL certificate using OpenSSL. This process will require that you have OpenSSL. It also requires that you have permission to execute bash scripts. These are essential inorder to avoid permission errors.

In our folder structure, create a scripts folder and then create a file called generate-certs.sh in it. Paste the following code snippet into that file:

1#scripts/generate-certs.sh
2
3echo "Creating certs folder ..."
4mkdir certs && cd certs
5
6echo "Generating certificates ..."
7
8openssl genrsa -passout pass:1111 -des3 -out ca.key 4096
9
10openssl req -passin pass:1111 -new -x509 -days 365 -key ca.key -out ca.crt -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Test/CN=localhost"
11
12openssl genrsa -passout pass:1111 -des3 -out server.key 4096
13
14openssl req -passin pass:1111 -new -key server.key -out server.csr -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Server/CN=localhost"
15
16openssl x509 -req -passin pass:1111 -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
17
18openssl rsa -passin pass:1111 -in server.key -out server.key
19
20openssl genrsa -passout pass:1111 -des3 -out client.key 4096
21
22openssl req -passin pass:1111 -new -key client.key -out client.csr -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Client/CN=localhost"
23
24openssl x509 -passin pass:1111 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt
25
26openssl rsa -passin pass:1111 -in client.key -out client.key

The above code generates the SSL certificates needed to establish a secured and encrypted connection between the server and client application. When it executes, it creates a certs folder, generates the SSL certificates for the server and client using OpenSSL, and then saves them in the certs folder. To learn more about these configurations and what they do, check out the OpenSSL website.

Generate an application SSL certificate with npm

Now use the script to generate an SSL certificate for your application by running the following commands in the terminal in the application’s root directory.

1$bash
2$ npm run generate:certs

This creates a certs folder that contains the generated SSL certificates.

Note that some admin privilege is required. If you encounter a permissions error when trying to run the script, use the following commands to give the script the execute privilege, and then try again.

1$bash
2$ cd scripts
3$ chmod u+r+x generate-certs.sh
4$ ./generate-certs.sh

You should have the following output on your terminal:

1> event-app-node-grpc@1.0.0 generate:certs
2> ./scripts/generate-certs.sh
3
4Creating certs folder…
5Generating certificates…
6Generating RSA private key, 4096 bit long modulus
7………………………………………………..++
8……………..++
9
10e is 65537 (0x10001)
11Generating RSA private key, 4096 bit long modulus
12…………………++
13………………………………………….….….….….….………..…………..++
14e is 65537 (0x10001)
15Signature ok
16subject=/C=CL/ST=RM/L=Santiago/0=Test/OU=Server/CN=localhost
17Getting CA Private Key
18writing RSA key
19Generating RSA private key, 4096 bit long modulus
20………………………………………………..….….….…………..++
21.…………..++
22e is 65537 (0x10001)
23Signature ok
24subject=/C=CL/ST=RM/L=Santiago/0=Test/OU=Client/ON=localhost
25Getting CA Private Key
26writing RSA key

Updating the client/app.js and server/index.js files

So far we have generated the SSL certificates needed to authenticate our gRPC APIs. Now we’ll proceed to modify the client/index.js and server/index.js files in order to work with these generated certificates.

In the updated client/app.js file shown below, we introduced the fs module to help us read the generated certificates, and we then used those certificates to create gRPC SSL credentials. Finally, we applied those credentials to the gRPC service.

1#client/app.js(updated)
2
3const PROTO_PATH = "../events.proto";
4const fs = require('fs');
5const grpc = require("@grpc/grpc-js");
6const protoLoader = require("@grpc/proto-loader");
7
8let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
9   keepCase: true,
10   longs: String,
11   enums: String,
12   arrays: true
13});
14
15const credentials = grpc.credentials.createSsl(
16   fs.readFileSync('../certs/ca.crt'),
17   fs.readFileSync('../certs/client.key'),
18   fs.readFileSync('../certs/client.crt')
19);
20
21const EventService = grpc.loadPackageDefinition(packageDefinition).EventService;
22const client = new EventService("localhost:50051",credentials);
23module.exports = client;

We also introduced the fs module In the updated server/index.js file shown below to help us read the generated certificates. We then used those certificates to create gRPC SSL credentials and applied those credentials to the server.

1#server/index.js(updated)
2
3const PROTO_PATH = "./events.proto";
4const fs = require('fs');
5
6let grpc = require("@grpc/grpc-js");
7let protoLoader = require("@grpc/proto-loader");
8
9let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
10   keepCase: true,
11   longs: String,
12   enums: String,
13   arrays: true
14});
15
16let eventsProto = grpc.loadPackageDefinition(packageDefinition);
17
18const server = new grpc.Server();
19
20let credentials = grpc.ServerCredentials.createSsl(
21   fs.readFileSync('./certs/ca.crt'), [{
22   cert_chain: fs.readFileSync('./certs/server.crt'),
23   private_key: fs.readFileSync('./certs/server.key')
24}], true);
25
26----------
27
28----------
29
30server.bindAsync("0.0.0.0:50051", credentials, (error, port) => {
31console.log(`Server listening at http://0.0.0.0:${port}`);
32server.start();
33});

Running the server and client applications

We’ve successfully implemented an event management solution using gRPC specifications. To test the endpoints, start the application from the terminal by working through the steps below.

The server

Navigate to the root directory of the project using the terminal and run the following commands:

1$bash
2$ npm run start

After running that, the server application should be live on http://0.0.0.0:50051:

1event-app-node-grpc % npm run start
2
3> event-app-node-grpc@1.0.0 start
4> node server/index.js
5
6Server listening at http://0.0.0.0:50051

The client

Open a new terminal window, navigate to the client folder from the root directory of your application, and then run the following commands:

1$bash
2$ node index

After running that, the client application should be live on http://localhost:50050:

1> event-app-node-grpc % cd client
2> client % node index
3Client Server listening to port 50050

To test the application, navigate to localhost:50050 on your browser or use an API testing tool like Postman. You should see the default event we initially added to our events array. Your response should be the same as the screenshot below:

You can proceed to test other endpoints added to the application to ensure everything works as expected.

You built a secure API with gRPC!

In this tutorial, we built a simple API in gRPC using Node.js, noting its operating concepts alongside its numerous advantages, like HTTP/2 and SSL/TLS for end-to-end authentication and encryption to improve API security.

Despite these advantages, gRPC also has weaknesses, ranging from limited browser support, its non-human readable data format, steep learning curve, and poor edge caching support. But regardless of these limitations, gRPC is the best option for communication between internal microservices, thanks to its unmatched performance and multilingual nature. The gRPC protocol is impressive, and has gained a significant level of adoption in the industry since its initial release in August 2016. It’s bound to continue to grow.

There are many other things you can do with gRPC. The example in this tutorial is just the tip of the iceberg of what gRPC offers. You can review the documentation to expand your domain knowledge on gRPC to enhance your application communication processes and strategies for maintaining gRPC security.

More API Resources:

• Building a secure GraphQL API with Node.js

• Containerizing Go applications with Docker