Giving your Raspberry Pi a REST - 2

Part2: The core code behind many of my Pi Projects.

In our last episode I talked about the overall client/server architecture and how I use a RESTful API to communicate between the two. In this installation I will talk about the framework I use for all my server code.

I found myself needing to address the same issues each time I wanted to tackle a different home automation task with the Raspberry Pi. Since I was using a REST architecture over http/s, my code was required to:

1. Register a TCP listener on specific port/s

2. Listen for and accept incoming connections

3. Read data and decode from TCP

4. Process the protocol and and optionally check fir any authentication methods

5. Dispatch the request to the appropriate handler.

6. Process any replies and status from the handler

7. Close and recycle the connection

Since the server had a REST layer on top of TCP, there was a bit more processing that could be handled by the framework.

1. Dispatch to a handler for that specific URL path

2. Decode any optional parameters and http method

3. Decode option JSON in the request body.

For example if the REST url was http://<server:port>/devices - it would be handle if we could register a handler for the “devices” and dispatch automatically.

FooServer

FooServer name of the collection of code that use to perform most of the TCP REST processing, as well as some optional Telnet protocol, if that is useful. FooServer is not a shared library but rather, I use it as a git submodule in my designs. It’s primary interface is through the include files: “ServerCmdQueue.hpp” and “RESTServerConnection.hpp”

For example, if we wanted our server to handle the url path for “test” we would do the following :

#include "ServerCmdQueue.hpp"
#include "REST/RESTServerConnection.hpp"

void Test_Handler(ServerCmdQueue* cmdQueue, 
		REST_URL url, 
		TCPClientInfo cInfo, 
		ServerCmdQueue::cmdCallback_t completion) {

  json reply;
  
  auto path = url.path();
  auto queries = url.queries();
  auto headers = url.headers();

   makeStatusJSON(reply,STATUS_OK);
  (completion) (reply, STATUS_OK);
}

main() {

// create the server command processor 
auto cmdQueue = new ServerCmdQueue(NULL);

cmdQueue->registerNoun("test", Test_Handler);

//  Create and start the server
TCPServer rest_server(cmdQueue);
rest_server.begin(9000, true, [=](){
	return new RESTServerConnection();
  });

// run the main loop.
while(true) {
	sleep(2);
    }

return 0;
}

This code snippet is pretty much all you need to do.

• The command queue does all the decoding and dispatching work.

• the cmdQueue->registerNoun function associates the “test” url path with the Test_Handler function

• The TCPServer class handles all the server connection work.

• The TCPServer::begin method will call the closure whenever a new connection is established to (in this case) port 9000. The RESTServerConnection() class then handles all the REST specifics.

• Every time a connection comes in with the url path “test” the Test_Handler will get called. If we wish to add more paths, we just register it with another registerNoun call.

What is this CRUD?

Most well-designed REST APIs use a system called CRUD. CRUD is short for Create, Read, Update, and Delete. In effect these verbs mapped into the standard HTTP methods.

• Create - POST

• Read - GET

• Update - PUT/PATCH

• Delete - DELETE

When describing a REST API the path part of the HTTP URL is called the noun. For example when talking to my chicken coop server:

• devices

• /devices/door

• /devices/light

This is why the ServerCmdQueue has a registerNoun() method. It maps the noun to the piece of code that handles it.

DemoServer

I have assembled a piece of sample code on GitHub we can use to demonstrate how all this work.

Demoserver will create a “test” endpoint at port 9000 and will handle the GET and PUT methods. It is written in C++ and has been tested using clang. So assuming you have previously setup your Pi with git-core, cmake, and clang or are running on a similar Linux system. You can build the demo code and follows:

git clone https://github.com/vinthewrench/demoserver.git
cd  demoserver
git submodule update --init --recursive
cmake .
make
# this will create the target bin/demoserver

To get it running in the background, simply

nohup bin/demoserver &

To demonstrate REST processing, demoserver has three different objects you can set and get:

• val.bool - a boolean

• val.int - an integer

• val.str - string

We can use the cURL and json_pp utilities from the command line to test getting and setting these objects.

Get value of objects.

$ curl -s localhost:9000/test |  json_pp -json_opt pretty,canonical

{
   "success" : true,
   "val.bool" : false,
   "val.int" : 123,
   "val.str" : "a string"
}

Set all three objects.

$ curl -s -X PATCH localhost:9000/test \
  -H 'Content-Type: application/json' \
  -d '{"val.int":11111, \
       "val.str":"VCovNwCIdo4Le2Iq",\
       "val.bool":true}' \
|  json_pp -json_opt pretty,canonical   

{
 "success" : true
}

Digging deeper

Let’s dig a little deeper into the sample code that handles the noun “test”. You will notice thats the Test_NounHandler function does a switch on the value of url.method() and dispatches to the appropriate handler for HTTP_GET or HTTP_PATCH.

In the Test_NounHandler_GET function it determines if the path has any subdirectories such as “/test/val.str” or is just “/test” by looking at the value of path.size(). It then constructs a json reply block, and performs a completion callback.

json reply;

// construct a json reply
reply[string(JSON_ARG_VAL_STR)] = val_str;
makeStatusJSON(reply,STATUS_OK); 

// invoke completion with status
(completion) (reply, STATUS_OK);

Let’s dissect what is happening here a bit.

The json data type is derived from an amazing library called JSON for Modern C++. This library allows us to treat JSON as a first class datatype. It handles all the hard work involved in coding and decoding JSON data. If you plan to do any work with JSON in C++ I highly recommend you consider Niels Lohmann work.

The function constructs a json reply by assign the value of val_str to the key JSON_ARG_VAL_STR which is "val.str".

It then calls the FooServer utility function makeStatusJSON() which attaches a value of true to the key "success". This just something I do to distinguish the difference between an HTTP status and the status of the REST request.

The handler function return of true or false just indicates if the URL passed to it was valid.

But one of the arguments that FooServer passes to the handler is a pointer to a ServerCmdQueue::cmdCallback_t callback function. If the URL was valid, FooServer expects the handler to call the completion function passing the reply block and the HTTP status.

This callback system allows our handler to return asynchronously and not stall the network processing of other requests through the server.

Walking though the code

The example code also includes a demoserver.xcodeproj file. If you clone the code from GitHub on an macOS system, you can build and debug it using Xcode.

git clone https://github.com/vinthewrench/demoserver.git
cd  demoserver
git submodule update --init --recursive
open demoserver.xcodeproj

This will allow you to place breakpoints and watch the code respond to the same curl example I posted above. I suggest you trace through some of the code at ServerNouns.cpp to start with.

Who are you?

In the next installment I will describe how you can prevent unauthorized access to your REST API. Until then, Keep watching the Skys.