rsc's Diary: ELC-E 2022 - Day 4

Friday, the last day of ELC-E 2022, is traditionally the day of the Embedded Linux Closing Game, with Tim Bird reporting about the embedded Linux world (universe?) domination progress and the overall state of the union. Of course, there were again several interesting talks.

Oniro Project: How to Push IoT Development to the Next Level

Ettore Chimenti (SECO SpA) outlined in my first talk of the last day of ELC-E the ideas of the Oniro project - an eclipse foundation project. The main idea is to have a common and open base for big (Linux based) and small (Zephyr based) IoT devices; they don't want to reinvent the wheel, but re-use as much as possible. Linux systems are built with Yocto, using the clang toolchain. Besides the usual Yocto layers, a system services layer provides device management functionality, and finally an app layer provides the customer application. Good to know: Oniro uses RAUC for field upgrade, but at least generally have the option to support other systems. Besides the runtime platform, the Oniro project also provides infrastructure for development, with GitLab, LAVA, OpenChain software supplychain and other tools. All in all, the project seems to be a good "best practice" collection.

BoF: Automated Testing and Board Farming

Chris Fiege then led a BoF about automated testing and board farming. For us, board farms usually need interactive access from developers (in the office or at home) and CI test access to the embedded devices. At Pengutronix, we have 16 devices in one 19" rack, with a test server, ethernet switch and serial console. Serial ports make a lot of trouble, both the old ethernet/rack based and the newer USB/serial devices. Until 2020, we had GPIOs on onewire, nowadays moved to CAN based IO boards. A lot of automation is done by CAN and multiplexing devices.

The group then discussed how their test labs look like. Tim Bird explained that he is mainly interested in power control; it turned out everyone seems to be hit by USB being flaky and tries to avoid it, but in reality it's not really possible. So people use things like Sonos WiFi plugs, Gude IT power switches etc. A TI guy operates 40 racks in one facility, 300 ports in another one, with OpenTest and labgrid. For them, camera and display testing is interesting. One participant has a board that is able to framegrab, emulate keyboard+mouse and even remote control the BIOS of x86 PCs; he explained that PiKVM turned out to work best. Another one suggested PicoKVM (on GitHub). Oleksij explained that we are currently working on network and switch test support in labgrid. A neat tool is ykush: it can plug/unplug crappy USB devices physically. Rouven explained that we got a patch into the kernel recently that makes it possible to switch off single USB hub ports.

The discussion moved on to which specific testsuites are used; at Pengutronix, in most cases we run device specific tests. The TI folks use a forked variant of LTP. Oleksij explained that his experience is that most testsuites are difficult to integrate into production environments.

Does anyone test hardware things from the outside? For WiFi and Bluetooth, the Pengutronix labs have one Access Point and a Bluetooth device per rack.

One guy asked if it was possible to control multiple places from labgrid; Jan explained that this is already possible.

For further discussion, there is the automated-testing mailing list.

Linux for Space - Mission Begins

Lenka Kosková Třísková & Lukáš Mázl (Technical University of Liberec) then talked about an open Linux for space applications. They found that many people in the Space business are doing open source, but there are not so many ones using Linux. They work on the Linux4Space project and work on making it compliant with several ESA standards.

They found that, right now, hardware becomes ready for example for cubesat style applications. The satellites are 10x10x10 cm³, have a controller and some application. Building one is easy, but launching it is still a little bit expensive. As the hardware becomes more accessible, more and more companies build it, so there definitely is an opportunity to collaborate.

Operating a Linux computer in space means that certain requirements have to be fulfilled, but luckily the space agencies already defined standards for that. The biggest challenge is that there is a lot of radiation; the power budget is low, and generally it is difficult to control the system remotely. The group collected lots of requirements and wrote the ECSS-E-ST-10-06 requirements specification.

Most systems on the satellite are doubled, as they can be shutdown quickly when there is a radiation impact. There are several well known interfaces such as SpaceWire or SpaceFibre, and generally filesystems have to be operated read-only.

The project is still in an early state, but there are already some meeting minutes available on They are currently working on a stratosphere balloon demo setup, working on a raspberry pi.

The team is working on a reference board based on some unnamed ARM processor. For communication, there is a special cubesat protocol.

Podman 101 and Beyond!

In my next talk, Pierre Blanc (Red Hat) talked about the podman container (or better: pod) manager. It is rootless, daemonless and thus aims to be secure by design. Pierre started with the history of containers, then started to show how to search for container images. Once one has found an appropriate container image, one can start it, connect to it and expose network ports. To get access to data, volumes can be mounted inside the container. A good thing with user containers is that it is not necessary to run containers as root.

Pods are groups of containers used for a common usecase. Stopping a pod means stopping all containers it consists of. So applications composed of several containers can be managed with podman as well.

Pierre showed that there are quite a lot other security features in podman; there are also several other helpful tools in the ecosystem like buildah (container creation) or skopeo.

While the normal usecase is to run the containers on a Kubernetes cluster, it is also possible to run them in a VM (podman machine).

Open Source Enabled Power Grid Management Architecture

After lunch, Lucian Balea & Benoît Jeanson (RTE) highlighted their open source strategy together with Linux Foundation: the idea is to build a new power grid software control architecture. RTE operates the French power distribution network. They found that it is now necessary to migrate towards a more distributed architecture with a growing number of actors across the power grid and thus growing complexity. An analysis has shown that the pain points are interoperability, modularity, upgradability, cost effectiveness and speed of innovation. They looked at how other industries solved similar problems, and - surprise! - found out that open source could help, thus they joined the LF Energy project.

The software that was then put on the table was OperatorFabric GridExchangeFabric and PowSyBl, and they started creating an architecture around that; nowadays, they have 18 open source projects, still with several holes. But more than just software, they see the main value in the community collaboration that happens in the projects, so the project is searching for more contributions.

From Zero to A/B: Swimming Upstream with Yocto, Barebox and RAUC

Roland Hieber & Ahmad Fatoum (Pengutronix) then tried to encourage the audience to use upstream components instead of vendor made downstream code.

RISC-V and Open Source Hardware Projects BoF

Drew Fustini (BayLibre) led a BoF session about open source RISC-V. While RISC-V inherently is not open source but open spec, it of course makes it possible to make open source processors. Recently, several chip companies seem to open up their semiconductor processes for open source projects, i.e. In the industry, several open source processors are being developed, like SweRV, Core-V and LowRISC Ibex.

Closing Game

Further Readings

rsc's Diary: ELC-E 2022 - Day 1

After two years with online conferences, the embedded Linux community once again meets in person for their annual Embedded Linux Conference Europe in Dublin, Ireland this year. Since many years, ELC-E is part of the Linux Foundation's Open Source Summit, and the best opportunity of the year to meet other Linux kernel and wider core ecosystem developers and discuss recent and future technological topics.

rsc's Diary: ELC-E 2022 - Day 2

The Dublin Convention Centre is huge - there is more than enough space for all the developers participating in the Open Source Summit. Fortunately, the talks will be on YouTube after the conference, so it's no problem that one can only hear a small selection of talks. However, here is my report of the talks I heard on the 2nd day of the conference.

rsc's Diary: ELC-E 2022 - Day 3

The Convention Centre is directly at the water front of the Liffey river, just a few walking minutes away from O'Connell Bridge, Temple Bar and Trinity College. Visiting ELC-E is always a good opportunity to visit interesting cities in Europe. However, here is my story of the talks I heard on day 3.

CLT-2022: Voll verteilt!

Unter dem Motto "Voll verteilt" finden die Chemnitzer Linux Tage auch 2022 im virtuellen Raum statt. Wie auch im letzten Jahr, könnt ihr uns in der bunten Pixelwelt des Workadventures treffen und auf einen Schnack über Linux, Open Source, oder neue Entwicklungen vorbei kommen.

Pengutronix at FOSDEM 2022

"FOSDEM is a free event for software developers to meet, share ideas and collaborate. Every year, thousands of developers of free and open source software from all over the world gather at the event in Brussels." -- FOSDEM

Conferences 2021: A Retrospective and Outlook

Besides the improvements around embedded Linux software and ongoing development of Linux kernel components, the Pengutronix team took the corona pandemic as an opportunity: Last year, many conferences were accessible via internet, so we joined from the comfort of our home offices and shared our experiences and ideas with the community.

Pengutronix on ELC 2021

The sun is shining for the last days of summer 2021. It doesn't only mean that autumn is coming, but also that this year's ELC is in preparation.

ELCE 2020 - Recommended Talks

The Embedded Linux Conference Europe (ELCE) is the one biggest meetup of Embedded Linux developers in Europe. As usual Pengutronix has attended this conference - but this year from the warmth of our homes.

Pengutronix at the Embedded Linux Conference Europe

The schedule for this year's Embedded Linux Conference Europe (ELCE) has just been released. As in the last years Pengutronix contributes talks to current topics around Embedded Linux.

ELC Europe 2016, Berlin

At the ELC Europe 2016 in Berlin our colleagues Jan Lübbe and Marc Kleine-Budde are talking about two interesting and important presentations about Kernel longterm maintenance strategies and verified boot.