Image recognition using TensorFlow.js and Node-RED

In this post, I will introduce how to use image recognition using TensorFlow.js and Node-RED. Node-RED flow can answer what image contains (For example, person, dog, car and bottle) using TensorFlow.js.

Settings in Node-RED
To use Tensorflow.js easily, I will use the TensorFlow.js module which contains the learned model from the function node in Node-RED. When we use 3rd party npm modules in function nodes, the document, “Loading additional modules” section in the official Node-RED website will be useful.

(1) Install npm module
After opening the command prompt, I will install the external npm module, “max-image-segmenter” on the home directory of Node-RED (“C:\Users\< User name >\.node-red” on Windows).

— — — — — — — — — — — — — — — — — — — — —
cd
cd .node-red
npm install @codait/max-image-segmenter@0.1.4
— — — — — — — — — — — — — — — — — — — — —

Because the specification of the module in the latest version, v0.1.12 has been changed, the next step would not work correctly. Therefore, I use the old version, v0.1.4.

(2) Edit settings.js file
After opening the settings.js file in the home directory of Node-RED (“C:\Users\< User name >\.node-red” on Windows), I will add the following definition into the functionGlobalContext section around line 216.

— — — — — — — — — — — — — — — — — — — — — — — — —
fnctionGlobalContext: {
imageSegmenter: require(‘@codait/max-image-segmenter’)
},
— — — — — — — — — — — — — — — — — — — — — — — — —

Thanks to this setting, Node-RED can call the external npm module via the global context from function nodes.

(3) Start Node-RED
Using the node-red command, I will start the Node-RED process. If the Node-RED process is running, execute the Node-RED command after killing the process using the Ctrl+c keys.

— — — —
node-red
— — — —

On the Node-RED flow editor ( http://<IP address>:<port number> ), I can use TensorFlow.js module in the function nodes.

Creating a flow to recognize images

(1) Install the necessary nodes
For uploading image files from the Node-RED flow editor or taking pictures using a camera connected to the PC, I will install the “node-red-contrib-browser-utils module”. To install the node module on the Node-RED flow editor, I will select “Manage palette” in the menu at the top right corner, and then select “Palette” and “Install” items. On the search box, I will type “node-red-contrib-browser-utils” to search and install the node module.

(2) Create a flow
To create the flow for the image recognition, I will place the inject node, function node and debug node in sequence on the workspace of the Node-RED flow editor. After that, I will connect them using wires in the processing order.

(3) Write JavaScript code on the function node
As the next step, I will write the following JavaScript code in the function node.

• JavaScript code:
  — — — — — — — — — — — — — — — — — — — — — — — — — — —
  var imageSegmenter = global.get(‘imageSegmenter’);
  imageSegmenter.predict(msg.payload).then(function (response) {
  msg.payload = response.objectsDetected;
  node.send(msg);
  });
  — — — — — — — — — — — — — — — — — — — — — — — — — — —
• Name: image segmenter

This JavaScript code receives binary data of the image from the previous node, and then passes the received data to the external npm module. After the image processing in the TensorFlow.js inside the npm module, this code receives the result of the image recognition and then passes the result to the next node.
The name of the function node should be text which consists of spaces, alphabet characters or numbers to avoid the problems when converting text.

After hitting the red deploy button on the top right corner of the Node-RED flow editor, I will click the button which is located at the left side of the file inject node to upload an image file from my PC. After clicking the button of the file inject node, I can see the new dialog to select a file using explorer. As the example procedures, I will upload the image file which contains a human face. After the image processing using TensorFlow.js, I can see the recognition result, “person” in the debug tab correctly.

This Node-RED flow can detect the various type of objects from an image. For example, after I upload dog and cat image files to the Node-RED, Node-RED shows the “dog” and “cat” text on the debug tab respectively.
Using the camera node instead of the file inject node file, a captured image from the camera can also be used.

Summary
As I tried it, I developed a simple image recognition flow using Node-RED. Utilizing TensorFlow.js module, edge devices can execute image recognition inside small devices like Raspberry Pi. In the TensorFlow community, there have been a lot of models on the Internet. Using the models, you can create original flows to execute original image recognition.