embedded linux

Together with Giampiero Baggiani of Sfera Labs, we have published Industrial IoT Security: Designing OTA Updates That Recover From Failure in Automation World. The article explains how A/B updates, independent storage, and hardware watchdogs help industrial Linux devices recover automatically from a failed update. It also covers signed update packages, reproducible builds, and other production considerations. Strato Pi Max and Rugix provide the practical example.

Today, we are releasing Rugix Admin 0.5, an open-source, browser-based interface for operating individual devices running Rugix Ctrl.

When developing, demonstrating, diagnosing, or servicing a Rugix-powered device, you often need quick answers to practical questions: Which system is running? Has it been committed? Are the application workloads healthy? Why is an update incompatible? Rugix Ctrl provides the commands and automation interfaces for these tasks. Rugix Admin brings them together in a visual workflow that is easier to use when working directly with a device.

Rugix Admin 0.5 is the first stable release, with a complete redesign of the interface and underlying architecture compared with the initial experimental versions. It gives developers and operators a clear view of one device and lets them act without switching between shell commands, logs, and configuration files.

We are excited to announce the release of Rugix Ctrl 1.3. 🎉

Rugix Ctrl 1.3 focuses on three areas: preventing incompatible deployments, providing a clear privilege boundary for local management, and improving recovery when early boot does not go as planned. To this end, the release introduces component compatibility checks, adds a privileged operation daemon, and provides new options for handling boot failures.

A Raspberry Pi on your desk is easy to recover: replace the SD card, connect a display, or reinstall the operating system. Once that Pi ships inside a product, recovery becomes much more expensive. An interrupted update or an unbootable kernel may mean a site visit or an RMA.

Package-by-package updates with apt or a similar tool are therefore a poor fit for updating the complete system of an unattended device. They modify the running root filesystem in place, can leave it partially updated, and make it difficult to know, let alone ensure, that every device in a fleet is running exactly the same software. The result is a fleet of devices running slightly different software stacks and configurations, which leads to inconsistent behavior and failures that are difficult to reproduce even at small scale.

Production devices commonly avoid these problems with A/B system updates. Raspberry Pi 4 and newer devices provide a firmware feature called tryboot that makes this approach particularly effective.

This article explains how the mechanism works: how the firmware selects between two systems, how a new version is tried without becoming the default, and why a failed update automatically falls back to the previous version.

Shipping a production embedded Linux product on an NXP i.MX board eventually leads to the same question: how do we update these devices in the field without bricking them? If the product is going to live for five, ten, or fifteen years, the answer needs to be more robust than “SSH in and run apt upgrade.” For devices that cannot afford to brick under any circumstances, the established answer is A/B system updates: two copies of the system on separate partitions, a bootloader that picks between them, atomic switchover, and automatic rollback if the new copy does not come up cleanly.

This article is a practical, technically detailed walk-through of how to build A/B system updates for NXP i.MX boards. The approach is generic across the i.MX 8, 8M, and 9 families; we have validated it end-to-end on the NXP FRDM-IMX91. Along the way, we cover partition layout, U-Boot configuration, A/B version selection from the bootloader, signed FIT images for verified boot, and dm-verity for runtime rootfs integrity, all the way up to how these pieces plug into a meta-imx build. By the end, you'll have a reference design that updates atomically, rolls back on failure, and cryptographically verifies boot artifacts and rootfs blocks, ready to drive with Rugix Ctrl or any other update engine.

Today, with the release of Rugix Ctrl 1.1, we are introducing Rugix Apps, a new mechanism for deploying and managing application workloads on embedded Linux devices.

Embedded devices typically run diverse application workloads on top of their base system: containerized services, local HMIs, data pipelines, or inference models. These workloads change more frequently than the OS, and different devices in a fleet often need different combinations of them. Managing their lifecycle, versioning, rollback, crash recovery, and persistent state, has traditionally been left as an exercise for the integrator. Rugix Apps makes it a first-class capability of Rugix Ctrl, building upon Rugix's best-in-class delta update capabilities, security properties, and reliability.

Rugix Apps works standalone: you do not need Rugix Bakery, Rugix's state management, or even Rugix system updates to use it. Any Linux device with Rugix Ctrl installed can deploy and manage application workloads.

In this article, we discuss the challenges of deploying and managing application workloads on embedded Linux devices, show how Rugix Apps addresses those challenges, and walk through a concrete example using Docker Compose.

If you are building an embedded Linux product, one of the engineering decisions you face is which on-device update engine to use. This decision has lasting implications: embedded devices routinely have lifecycles of 5 to 15 years, and whichever update mechanism you choose today will determine how you ship software to devices already in the field for the entire duration of that lifecycle.

Fortunately, the landscape of open-source OTA update engines for embedded Linux has matured significantly over the past decade. Where teams once wrote bespoke shell scripts to flash partitions, there are now several ready-made open-source tools available to choose from. These tools differ in philosophy, architecture, features, and the trade-offs they make. In this article, we survey and compare the major open-source OTA update engines for embedded Linux, examine the fundamental techniques and strategies they employ, and provide a technically grounded comparison.

There is a growing trend in the embedded Linux space: vendors are coupling their cloud platforms with their own on-device OTA update mechanisms. On the surface, this makes sense. You get a single vendor, a single integration, a single bill. But if you look a few years ahead, this coupling comes at a cost that is easy to underestimate.