Node-Red: Bullseye and serial ports

Sometimes you hit a bad bug at a bad time, and it takes awhile to figure out what to even search for to find an answer. Well, I hit one of those bugs this weekend and man was it a pain.

Long story short, if you are running Node-Red 2.0ish and on Debian Buster using Docker, and you upgrade to Node-Red 3.0ish and Debian Bullseye using a Raspberry Pi 4 you're probably going to run into the following if you use any serial ports in your flows.

24 Oct 11:54:29 - [warn] Encrypted credentials not found
24 Oct 11:54:29 - [info] Server now running at http://127.0.0.1:1880/
24 Oct 11:54:29 - [info] Starting flows
Segmentation fault (core dumped)

There are a number of github issues running around various places that talk about this problem. The issue is a little complicated to explain, but has been busted for awhile now and isn't likely to get better soon because it is a combination of issues that is causing the problem.

The one thing that nearly all of the descriptions of the problem leave out is that the problem only shows up when you are running node-red in a docker container, and that's the only place you can fix it. I wasted so many hours simply because so many people left this one key point out in their discussion of the issue, or alluded to it in a way makes it hard to detect what's going on unless you know about cross-compiling code.

Here's the work-around to the situation:

• Remove the /usr volume from your docker-compose.yml file. If you don't you'll keep getting an error that the entrypoint.sh file can't be found for node-red.
• Rebuild your docker container and start node-red 3.0 up.
• In the node-red editor, add node-red-node-serialport 1.0.1 under the manage palette screen, but do not add a serial in or out node to your flow yet. If you do, there's a good chance node-red will start segfaulting on startup.
• Stop your current node-red container.
• Delete the current node -red container:

cd ~/nodered
mv data data-backup
sudo docker-compose down [your nodered container name]
sudo docker rm [your nodered container name]

sudo docker run -d --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -v /path/to/your/data/volume:/data -p 1880:1880 -e NODE_RED_ENABLE_SAFE_MODE=true nodered/node-red

sudo docker ps

sudo docker exec -it [safe mode nodered container] /bin/sh

What the above commands do is move you to your node-red directory, move your data directory off to the side so you can create a fresh one, remove container you've been using, create a new one in safe mode, and then open a command line in the running container. This last bit is the key thing to know. You have to have node-red up and running, node-red-node-serialport installed, and no serial nodes in your flow to fix the problem. Doing this will set the data directory up with the right files you need. Next, we are going to re-compile node-serialnode and serialnode packages from within the running node-red container. This causes cross-compilation between the Alpine Linux distribution in the container and the host's Debian Bullseye OS.

# From the command line in the container started with the docker exec command.
cd /data
npm rebuild --build-from-source
exit

If everything went well, you should see "rebuilt dependcies successfully" and your problem is now fixed. Now we need to clean things up a bit.

sudo docker stop [safe mode nodered container]

sudo docker ps

sudo docker rm [container id of safe mode nodered container] -f

sudo docker-compose up -d

You can leave off the -d on the docker-compose up command if you want to watch the logs easily. Now, go to the node-red editor to place and configure a serial port node then deploy. You shouldn't see the segmentation fault reproduce and you're back in business.

One final word. The work-around will vanish if you move your data directory back to /data or overwrite the @serialport or serialport directories, so copy your flows back over without copying the node_modules directory.

Github issue with discussion