Idea
This How-To will walk you through the process of connecting a USB webcam to the ctrlX CORE for simple vision applications, without the need for an industry-standard camera. You will have to build your own Python Snap to capture images, encode them as Base64-JSON strings, save them to a Datalayer node, and finally allow Node-RED to display the images on a dashboard. While there is no guarantee that this will work with your specific camera, it was successful with the one I had on my desk.
What you need
ctrlX CORE (I used ctrlX OS is 3.6)
USB Webcam
ctrlX AUTOMATION Software Development Kit (SDK)
(I used version is 3.6.0 running on an ARM Raspberry Pi with Ubuntu Server 22, to not get ARM/AMD cross-compile problems)
Overview
A Datalayer Provider handles the Datalayer node for the webcam image data.
A Datalayer Client App
... finds the USB webcam within /dev/video*
... captures images with the Python library OpenCV
... encodes them as Base64-JSON-strings
... writes them to the specified Datalayer Node
Node-RED can subscribe to the Datalayer Node and display the image stream on a dashboard
Step by Step
Datalayer Provider
First you will need a Datalayer provider snap, that creates a node for the webcam images. I used the python sample datalayer.provider from the Software Development Kit (SDK) and adjusted it a bit to get the Datalayer node seen below.
Datalayer Client with Webcam handling
Next, take the sample datalayer.client.simple and add all the needed webcam functionality. Therefore, please download webcam.py at the end of this article and upload it into your SDK build environment.
The class Webcam is implemented with the python vision library OpenCV. The code won't be discussed here as it should be self explanatory if you have a look at it.
Inside your main.py add the following lines of code at the designated locations:
from ctrlxdatalayer.variant import Variant
from helper.webcam import Webcam
def main():
...
# get webcam access
cam = Webcam()
...
while datalayer_client.is_connected() and not __close_app:
# get image from webcam
imageJsonString = cam.get_frame_json()
# write to datalayer
data = Variant()
data.set_string(imageJsonString)
addr = "sdk/webcam/image/"
result, var = datalayer_client.write_sync(addr, data)
time.sleep(0.1)If you want to adjust the frames per second, that you will stream to the Datalayer, adjust the time.sleep(...) value to your needs.
Next add the required plugs and packages to the snapcraft.yaml file:
apps:
client:
command: bin/main.py
plugs:
- network
- datalayer
- camera
- hardware-observe
daemon: simple
restart-condition: always
passthrough:
restart-delay: 10s
parts:
client:
plugin: python
source: .
stage-packages:
- ctrlx-datalayer
python-packages:
- opencv-python-headlessFinally edit setup.py according to your snap name and the required packages.
After you build the snap successfully and install it on your ctrlX CORE, you should see the Base64-JSON-string output of your snap inside the specified Datalayer node similar to the image seen earlier in the article.
Node-RED Dashboard
To display the webcam stream in a Node-RED dashboard you simply have to add three nodes:
Data Layer Subscribe
json (converts the JSON string to a JSON object)
template (from Dashboard, not Dashboard 2.0)
Inside the template node insert:
<div>
<img
id="cam"
ng-src="data:image/{{msg.payload.format}};base64,{{msg.payload.image}}"
style="max-width: 100%; height: auto;"
onload="
const img = document.getElementById('cam');
document.getElementById('res').innerText = img.naturalWidth + '×' + img.naturalHeight;
"
/>
<div id="res" style="text-align: center; margin-top: 8px;">–</div>
</div>To be able to see your webcam stream at a faster refresh rate than 1 frame per second, go to the configuration nodes via the dropdown menu in the upper right corner (1), open the ctrlx-config-subscription node (2), set the Publish and Sampling Interval (3) (the default value is 1 second) and finally hit Update (4).
After you hit deploy, you should be able to see the webcam stream in the Node-RED Dashboard.
