Introduction to Git integration with Node-RED
In this article, I will introduce Git integration functionality in Node-RED. Using Raspberry Pi devices or cloud services, many hobby users have quickly been developing their flow by just connecting nodes to realize their idea of IoT systems like home automation or RPA. To utilize these great experiences as flow developers in their job, they may consider using Node-RED for mission-critical systems. For example, these systems are factory automation and visualization dashboard in their factories, or financial backend APIs for mobile applications. However, in this case, flow developers tend to encounter the following problems with their production systems.
- Problem 1: All functionalities exist in a single flow on a single Node-RED environment
If only one Node-RED environment is available in the mission-critical system, the huge flow which contains many nodes will exist in this single environment. Because the single flow will use huge memory and CPU resources, it leads to performance problems. In other respects, context variables or HTTP endpoints which have the same names are sometimes conflicted due to a single environment. In terms of the Node-RED dashboard, all components according to the different applications have existed in the same dashboard UI. - Problem 2: Who modified the Node-RED flow? When flow was modified?
In the default usage, a lot of flow developers will use a single environment simultaneously. Flow editor has user authentication for login but doesn’t manage who edits the flow and when a flow developer changes it. Due to this mechanism, flow developers cannot revert the flow when the problem occurs in mission-critical systems. Furthermore, the responsibility for the serious situation will be vague. - Problem 3: Manual flow backup and uncommon flow sharing with other developers
To avoid loss of the flow on the flow editor, the flow developer manually backup the flow using the “exporting flow” functionality and then save it as a JSON file to their local PC in a conventional way. They need to set the file name to identify the flows but it is a time-consuming task. Additionally, it will also be difficult for other flow developers to understand the details of the flow and required modules in the case of sharing flow files because the pieces of information are written to the other files like word files by flow developers in their way.
As I described above, there are a lot of concerns in flow development for production systems. If the Node-RED environment is prepared as same as the personal environment, the progress of the flow development may be a slog.
What is Git integration?
This functionality is known as a project feature on the Node-RED official document. Using the Git integration, flow developers can manage their flows on the flow editor as same as general code development by Git command or IDE like Visual Studio Code. In the systems development, three issues described in the previous section can be solved as follows.
- Solution for problem 1: Switching projects
After enabling Git integration, the flow developer can select one development project from multiple projects on the flow editor. Because each project has a minimal flow only to realize one application, flow developers can solve the problems in terms of computer resources and artifact conflicts inside the flow. In my experience, I have seen a nightmare situation in that unnecessary configuration nodes remain in the flow due to sharing the flow development environment among multiple projects. Using the switching projects functionality, flow developers can avoid contamination due to the other project and proceed with a couple of flow development in different projects on the single Node-RED environment in parallel. - Solution for problem 2: Version control of flow
In the project created by Git integration functionality, the flow editor has the “History” tab on the sidebar area. On the tab, flow developers can manually input the commit message when the flow should be in checkpoint to record the developed flow. If the automatic mode is selected in the user setting instead of manual mode, it is also possible to add commits with the default message automatically when the flow developer clicks the deploy button. In both modes, the version-managed flow can be recovered to the previous commit which has executable flow by CLI Git operation when the latest flow doesn’t work due to an error. On the single Node-RED environment, all commits are by a single user. However, if flow developers share their flow on the GitHub repository, each commit has information about the developer who modified the flow. This information will be useful to identify when and who made the mistake when solving problems in the flow. - Solution for problem 3: Flow sharing on GitHub
In addition to the version control in the local repository, Node-RED has the ability to upload to the remote repository on GitHub. Using the ability, it is beneficial for the flow developer to back up the flow in preparation for the loss of the environment like a PC broken or a server down. Flow developers can also input information about how to use the flow and dependency modules to execute flows on the flow editor. The inside flow editor outputs this information to general files like README.md, and package.json files. When other flow developers download the shared flow, they can easily check the details of the application from the documentation files. And they can also install required modules by just clicking the module installation button on the flow editor. Furthermore, this functionality can be used in the automated deployment of the flow for cloud and edge devices using CI/CD pipelines like GitHub Actions.
How to use Git integration
Here, I will explain how to enable the Git integration feature on your Node-RED environment. If there’s no Node-RED environment on your local PC or server on the cloud, you need to install the Node-RED based on the official document in advance.
Prerequisite
To utilize Git integration on Node-RED, the Git command should have been installed on your environment. In general, macOS and Linux have the Git command as the default but the Windows environment has no Git command. Therefore, Windows users need to install the Git command by the installer download from the following website.
https://git-scm.com/downloads
Enabling Git integration
There are versatile ways to turn on the Git integration like environment variables and manual definition on the settings.js file. In this article, I utilize the admin command provided by the Node-RED command. To execute the admin command, please type the following command on your command prompt or terminal.
— — — — — — — — — — —
node-red admin init
— — — — — — — — — — —
This admin command is the command-line wizard to ask questions to generate custom settings of Node-RED. Two of the question in the Projects section are related to Git integration.
In the first question to enable the project feature, select “Yes”. After that, the wizard asks to select workflow mode. In this article, select the “manual” mode because this mode is the fundamental way to understand Git integration. (On the other hand, in the flow development of the production system, “auto” may be recommended to record the flow change when the flow developer clicks the deploy button because it is easy to restore the flow.) Once all questions are finished to answer, the command saves the custom configuration to the settings.js file. To start the Node-RED process using the settings.js file, type the node-red command as usual.
— — — — —
node-red
— — — — —
Creating a new project
After opening the flow editor via the address, http://localhost:1880, you can see the wizard UI for Git integration.
The wizard is easy to understand because it provides step-by-step procedures to configure the Git integration settings with the details of the explanations. But select the “Open existing project” text next to the “Not right now” for simple explanations in this tutorial. The next dialog is the user settings of the Git client. This information is used on both local and remote repositories on GitHub to identify who changes the flow. Therefore, I strongly recommend that the username and e-mail address are the same as your GitHub account to avoid emerging different users on GitHub.
The next dialog has three options, “Open project”, “Create project” and “Clone Repository”. Because we need to create a new project in the first step, select “Create project” from the options. (The “Open project” options will be used to switch the project when a single Node-RED environment has multiple projects in the future. And the “Clone Repository” option is for downloading the remote project which has been uploaded on GitHub by another flow developer.)
At the bottom of the “Create project” button, there are four settings as follows.
- Project name
This is the project name for the local repository. To consider the situation that it will be used for the part of the URL on GitHub after uploading flow to the remote repository in the future, the length of the project name should be as short as possible. In general, the characters in the project name should also be lower cases or numbers with hyphens as delimiters. In this article, I inputted “test” as the project name. - Description
It is an optional setting to input the description of the created project. You can write the text in the natural language. - Flow file
This is also an optional setting to specify the flow file name. But there is no reason why the flow developer changes the name in the single flow file. Remain the default name, “flow.json”. - Credential
The final setting is to select whether the credential of the flow should be encrypted by an encryption key or not. Because this tutorial uses a local repository on your local PC or a remote repository created as a private mode on GitHub, encryption is not needed. Therefore, select the bottom option, “Disable encryption”. (When publishing the flow with credentials to a public repository on GitHub, it should be “Enable encryption” to protect confidential information.)
Developing flow and recording history
Once creating a new repository on the flow editor, you can start to develop a new flow from scratch. For instance, create the simplest flow which consists of the inject and debug nodes to output the timestamp data to the debug tab. After checking the intentional behaviors of the flow, you may want to create a checkpoint to record the flow on the local repository. In this case, click the “+ all” button in the “Local files” section on the history tab to make all changed files ready to record.
Once clicking the “commit” button next to the “Change to commit”, the text area to enter the commit message is emerged. In this tutorial, “The first flow” is set to the text area. After clicking the “Commit” button located at the bottom of the text area, your flow has been completely recorded with the commit message to the version history. You can see the version history in the “Commit History” section.
That is the end of this tutorial. When you have done additional changes to the flow editor, you can repeat the same procedures on the history tab.
Checking Git log
If you’re familiar with Git CLI operation, you may be interested in the status of the local repository. To show the Git log on the command prompt or terminal, firstly change the working directory to the path of the local repository, “~/.node-red/projects/test” on macOS and Linux, “C:¥Users¥<User name>¥.node-red¥projects¥test” on Windows. After this operation, type the “git log” command to show the version history to the command prompt or terminal.
As you can see in the above screenshot, the git history by CLI is shown as the same as the history tab on the flow editor.
Conclusion
In this article, I briefly explained the Git integration on Node-RED which is necessary functionality in the flow development of the mission-critical system. Leading companies have already used Git integrations in their mission-critical systems. For instance, Siemens has used this functionality in their product, Industrial Edge Flow Creator. If you’re interested in the details of the procedures, you can read the documentation of this product. As another example, Red Hat has also adopted this Git integration in their Node-RED operator for Red Hat OpenShift as the default. With the expansion of usage in production, Git integration will become crucial and standard development among Node-RED users.
