Saturday, October 5, 2013

FunkOS Is Dead

While most would probably assume it due to its inactivity for over 3 years, I'm going to state the obvious: FunkOS is dead. Well, maybe it's dead in the sense that I'm no longer supporting it – but the source is still freely available, and folks can do with it as they please – hey, that's the magic of open-source!

And with that proclamation I'd like to take the opportunity to reflect on the project, the state of the RTOS' market in general, and where I'd like to see it go from here.

Requiem for an RTOS

First off - despite the inactivity of FunkOS, people are still using it, and its popularity has stabilized at around 120 downloads per month (if you believe the project's download stats on Sourceforge). I still get regular emails from folks all over the world looking for tips and advice on how to implement systems using FunkOS, and I'm very impressed to see some of the things that people have built with it – hobbyists, students, and professionals alike. I've also been encouraged by the positive feedback I've received from users, commenting on how much they've learned from going through the code, and how it “just works”. And recently, I was shocked to learn that some engineering course in India featured an assignment on various RTOS', and FunkOS was part of the comparison! It's been truly gratifying to connect with so many people through the power of open-source, and to be able to contribute something positive to the community.

In hindsight, FunkOS was written very naively – I had never written an RTOS before, and had never really looked at the source for other RTOS' in any detail prior to that. My only knowledge of RTOS implementation was trial and error, and a recorded presentation that Jean Labrosse (of Micrium) had made at ESC one year. As a result, there are a lot of things that are “theoretically” wrong with FunkOS. First – it really doesn't scale well; all threads are stored in a single linked-list, and the more threads you have (whether or not they're active or ready), the longer it takes to run the scheduler. And the scheduler actually runs in the middle of the context switch interrupt – which is not great either, since it's not deterministic (not good for an ISR). Also, there's a lot of code duplication - since the code for each of the blocking objects (Semaphores, Mutexes, etc.) are largely copied from each other. Practically, none or these issues are likely to cause problems for users running it on AVRs - but these issues always bugged me, and if I were to do it all again, I'd do it differently (in fact, I already have – and I'll get to that later).

Despite its shortcomings, FunkOS is still a reasonably popular project; at least by SourceForge standards. All in all, I'd estimate that FunkOS has been downloaded around 7500 times in total from, and other locations. No doubt, most of the popularity of FunkOS has enjoyed is owed to the shameless self promotion that I've managed to get from the Markade's appearance on back in 2010, as well as recent publicity as a result of the Markade setting a Guinness World Record for being the world's smallest playable arcade machine in 2012. Its compatibility with the mega AVR's used in Arduinos likely contributes to its popularity as well.

However, I'd like to think that the quality of the code, documentation, ease-of-use, and the truly open license have helped to drive its adoption as well.

Quality, Cost, and Licensing

Speaking of which, anyone who's taken a look at the source code for some of the other open-source RTOS kernels out there can tell you just how ugly RTOS code can be. Poor source documentation, unintuitive and incomprehensible naming conventions, and smatterings of random #ifdef's are hallmarks of the myriad of open-source RTOS' out there. Heck, even some of the commercial offerings have been panned for these issues! One only needs to see Jack Ganssle's comments regarding the uCOS-II (Referenced Here) and TI's TI-RTOS (See Here) to see just how wildly the quality of “commercial-grade” code can vary.

And then there's the matter of “openness” and “freedom”. While many RTOS' kernels claim to be “free”, the licensing terms range from fairly-open, to extremely restrictive. In reality, most of the RTOS' out there claiming to be “free” or “open” come with a catch. On one side of the spectrum, strict GPL licensing terms are found in a number of these projects, and have viral consequences for applications which are linked as single binaries to the affected RTOS libraries. Others have opted for various flavors of open licenses (or dual-license models), which as a result of restrictions guide users to paid versions of the projects. Yet other still are free only for certain types of uses- with hobbyist or academic development falling under one free license and commercial work falling under a paid license. Some go far enough to give users a free-for-some-uses license, but sell the documentation to show users how to actually work with it.

Look - I'm not against people making money off of their work in open-source, or licensing their code however they wish; but licensing often complicates the decision-making process for what is otherwise a technical exercise. In addition, the non-standard licensing found in most open-source RTOS' is a quagmire that's difficult for average developers to navigate without competent legal counsel– and the more unfamiliar the license, the more difficult it can be to convince an individual or corporation to agree to the terms, regardless of the technical merits. When faced with a choice between “free, but you might owe somebody something – including your IP” license, and a commercial or proprietary license – you'd better believe that the lawyers will choose the most protective, least ambiguous option every time.

For example, when I had first written FunkOS, I had hoped that my employer would pick it up for use in our products; but the conditions associated with the license (an attribution statement in product documentation and code submissions for certain RTOS-port code) were still too restrictive. Yes - even though I had developed it for free in my spare time, the license was permissive, and I was willing to support it personally, they wouldn't go for it because the license wasn't completely unconditional. I was extremely disappointed by that, and I'd be lying if I said that it didn't contribute to my decision to leave the company shortly thereafter (and also to stop working on FunkOS).

What I find ironic is that we also used a variety of “free” RTOS kernels shipped as object code by our parts vendors – and we trusted in the quality of that code implicitly because vendors are “the experts in that sort of thing”. But in reality, it was blind faith – we were actually using software of unknown pedigree, which was often buggy and full of surprises. Worse than that, the vendor-specific nature of these RTOS' had the side effect of promoting vendor/platform lock-in.

Where Do We Go From Here?

Let's face it – the RTOS is no longer a niche product; RTOS' are commodity software, and the market is fairly mature. There are numerous popular commercial offerings available for all sorts of platforms, include such well-known options such as ThreadX, uC/OS-II and III, Keil RTX, and TI/BIOS (the larger offerings for MMU-based CPU's such as VxWorks and QNX are out of scope for this discussion). A quick search on SourceForge reveals that there are about 100 RTOS projects hosted on that site alone, with about 20 of them listed as “active”. This list including most of the big “free” players - such as FreeRTOS, ChibiOS/RT, and NuttX (FunkOS is still in the top 10 by popularity, by the way). And still other popular embedded OS's such as Contiki and ucLinux could be considered here as well.

So does the world really need more RTOS competition? I don't know that we do, but for a new RTOS to be successful in this mature market, it would need to be disruptive. In my estimation, it would need to challenge the incumbent commercial offerings in terms of features and performance, while additionally throwing down the gauntlet against the free options in terms of total-cost-of-ownership, quality, and licensing.

Here's a comprehensive wish-list of what I'd really like to see in a new open-source RTOS:

Truly free -No cost to download anything – including the source, build environment, documentation or otherwise. -Permissive single licensing model, allowing for usage in binary form in any project or product, with or without modifications, without affecting IP. -No royalties, no attributions -No guarantee for fitness -Development is not-for-profit

Truly open -Community contributions are welcomed -Institute a process for integrating changes from community into mainline

Sophisticated Design
-Developed using object-oriented methodologies to minimize lines-of-code and maximize code reuse
-Set a new minimum-standard for code quality in an open-source RTOS
-Target key architectures in each category
--32-bit: ARM Cortex-M0, M0+, M1, M3, M4
--16-bit: MSP-430, PIC24/dsPIC
--8-bit: Atmel AVR
-Feature competitive with the best the commercial offerings
--Fully-deterministic scheduling
--Implement priority inheritance for Mutexes
--Variety of blocking objects, events, and IPC mechanisms
--Tickless, dynamic timers for maximum power efficiency
--Dynamic threads
--Threadsafe, determinstic, fixed-block-size heaps
--Device driver APIs to abstract hardware interfaces from code
-Designed as a Platform for middleware
--Graphics, GUI, filesystems, network stacks, and virtual machine services can be implemented
--Accompanying cross-platform software design tools and utilities
-Lightweight standard library equivalents
-Per-feature configuration that can be generated by tools

Easy to use, easy to learn, easy to understand
-APIs are written in plain English, with readability in mind
-Code is partitioned into modules by feature and logically abstracted
-Well documented source code – not just API documentation, but annotated implementation.
-All documentation is generated from source using standard tools, such as doxygen
-Easy-to-understand examples for all core features
-Minimum use of #ifdefs, limited to compile-time feature configuration. -CPU-specific code is clearly separated into separate modules

Support a variety of use cases
-Export as source, and compile directly into projects
-Build RTOS as a library, link against projects
-Build application as part of an integrated build system

Reliable, Robust, and Performant
-All features have unit tests that are easily run on any supported target or simulator
-Performance profiling suite available for all key benchmarks
-Employ extensive use of secure and defensive coding techniques to ensure reliability
-Built-in debugging and instrumentation to catch critical errors and trace execution at runtime

I think that sounds like a pretty good starting point – and nothing I've seen in the open-source community provides all of these. Who knows - maybe I should take up that challenge.

While that's my wishlist - what would you like to see in your ideal free-and-open RTOS?

1 comment:

  1. What are your thoughts on chibios/RT

    One of my bigger complaints is that the lead developer has not adopted git. I think this reduces integration. But he is very quick to pick up patches submitted.
    And it sort of lacks a good bootloader sub-project.