FunkOS Is Dead

While most would probably assume it due to its inactivity for over 3 years, I'm going to state the obvious: FunkOS is dead. Well, maybe it's dead in the sense that I'm no longer supporting it – but the source is still freely available, and folks can do with it as they please – hey, that's the magic of open-source!

And with that proclamation I'd like to take the opportunity to reflect on the project, the state of the RTOS' market in general, and where I'd like to see it go from here.

Requiem for an RTOS

First off - despite the inactivity of FunkOS, people are still using it, and its popularity has stabilized at around 120 downloads per month (if you believe the project's download stats on Sourceforge). I still get regular emails from folks all over the world looking for tips and advice on how to implement systems using FunkOS, and I'm very impressed to see some of the things that people have built with it – hobbyists, students, and professionals alike. I've also been encouraged by the positive feedback I've received from users, commenting on how much they've learned from going through the code, and how it “just works”. And recently, I was shocked to learn that some engineering course in India featured an assignment on various RTOS', and FunkOS was part of the comparison! It's been truly gratifying to connect with so many people through the power of open-source, and to be able to contribute something positive to the community.

In hindsight, FunkOS was written very naively – I had never written an RTOS before, and had never really looked at the source for other RTOS' in any detail prior to that. My only knowledge of RTOS implementation was trial and error, and a recorded presentation that Jean Labrosse (of Micrium) had made at ESC one year. As a result, there are a lot of things that are “theoretically” wrong with FunkOS. First – it really doesn't scale well; all threads are stored in a single linked-list, and the more threads you have (whether or not they're active or ready), the longer it takes to run the scheduler. And the scheduler actually runs in the middle of the context switch interrupt – which is not great either, since it's not deterministic (not good for an ISR). Also, there's a lot of code duplication - since the code for each of the blocking objects (Semaphores, Mutexes, etc.) are largely copied from each other. Practically, none or these issues are likely to cause problems for users running it on AVRs - but these issues always bugged me, and if I were to do it all again, I'd do it differently (in fact, I already have – and I'll get to that later).

Despite its shortcomings, FunkOS is still a reasonably popular project; at least by SourceForge standards. All in all, I'd estimate that FunkOS has been downloaded around 7500 times in total from sf.net, avrfreaks.net and other locations. No doubt, most of the popularity of FunkOS has enjoyed is owed to the shameless self promotion that I've managed to get from the Markade's appearance on hackaday.com back in 2010, as well as recent publicity as a result of the Markade setting a Guinness World Record for being the world's smallest playable arcade machine in 2012. Its compatibility with the mega AVR's used in Arduinos likely contributes to its popularity as well.

However, I'd like to think that the quality of the code, documentation, ease-of-use, and the truly open license have helped to drive its adoption as well.

Quality, Cost, and Licensing

Speaking of which, anyone who's taken a look at the source code for some of the other open-source RTOS kernels out there can tell you just how ugly RTOS code can be. Poor source documentation, unintuitive and incomprehensible naming conventions, and smatterings of random #ifdef's are hallmarks of the myriad of open-source RTOS' out there. Heck, even some of the commercial offerings have been panned for these issues! One only needs to see Jack Ganssle's comments regarding the uCOS-II (Referenced Here) and TI's TI-RTOS (See Here) to see just how wildly the quality of “commercial-grade” code can vary.

And then there's the matter of “openness” and “freedom”. While many RTOS' kernels claim to be “free”, the licensing terms range from fairly-open, to extremely restrictive. In reality, most of the RTOS' out there claiming to be “free” or “open” come with a catch. On one side of the spectrum, strict GPL licensing terms are found in a number of these projects, and have viral consequences for applications which are linked as single binaries to the affected RTOS libraries. Others have opted for various flavors of open licenses (or dual-license models), which as a result of restrictions guide users to paid versions of the projects. Yet other still are free only for certain types of uses- with hobbyist or academic development falling under one free license and commercial work falling under a paid license. Some go far enough to give users a free-for-some-uses license, but sell the documentation to show users how to actually work with it.

Look - I'm not against people making money off of their work in open-source, or licensing their code however they wish; but licensing often complicates the decision-making process for what is otherwise a technical exercise. In addition, the non-standard licensing found in most open-source RTOS' is a quagmire that's difficult for average developers to navigate without competent legal counsel– and the more unfamiliar the license, the more difficult it can be to convince an individual or corporation to agree to the terms, regardless of the technical merits. When faced with a choice between “free, but you might owe somebody something – including your IP” license, and a commercial or proprietary license – you'd better believe that the lawyers will choose the most protective, least ambiguous option every time.

For example, when I had first written FunkOS, I had hoped that my employer would pick it up for use in our products; but the conditions associated with the license (an attribution statement in product documentation and code submissions for certain RTOS-port code) were still too restrictive. Yes - even though I had developed it for free in my spare time, the license was permissive, and I was willing to support it personally, they wouldn't go for it because the license wasn't completely unconditional. I was extremely disappointed by that, and I'd be lying if I said that it didn't contribute to my decision to leave the company shortly thereafter (and also to stop working on FunkOS).

What I find ironic is that we also used a variety of “free” RTOS kernels shipped as object code by our parts vendors – and we trusted in the quality of that code implicitly because vendors are “the experts in that sort of thing”. But in reality, it was blind faith – we were actually using software of unknown pedigree, which was often buggy and full of surprises. Worse than that, the vendor-specific nature of these RTOS' had the side effect of promoting vendor/platform lock-in.

Where Do We Go From Here?

Let's face it – the RTOS is no longer a niche product; RTOS' are commodity software, and the market is fairly mature. There are numerous popular commercial offerings available for all sorts of platforms, include such well-known options such as ThreadX, uC/OS-II and III, Keil RTX, and TI/BIOS (the larger offerings for MMU-based CPU's such as VxWorks and QNX are out of scope for this discussion). A quick search on SourceForge reveals that there are about 100 RTOS projects hosted on that site alone, with about 20 of them listed as “active”. This list including most of the big “free” players - such as FreeRTOS, ChibiOS/RT, and NuttX (FunkOS is still in the top 10 by popularity, by the way). And still other popular embedded OS's such as Contiki and ucLinux could be considered here as well.

So does the world really need more RTOS competition? I don't know that we do, but for a new RTOS to be successful in this mature market, it would need to be disruptive. In my estimation, it would need to challenge the incumbent commercial offerings in terms of features and performance, while additionally throwing down the gauntlet against the free options in terms of total-cost-of-ownership, quality, and licensing.

Here's a comprehensive wish-list of what I'd really like to see in a new open-source RTOS:

Truly free -No cost to download anything – including the source, build environment, documentation or otherwise. -Permissive single licensing model, allowing for usage in binary form in any project or product, with or without modifications, without affecting IP. -No royalties, no attributions -No guarantee for fitness -Development is not-for-profit

Truly open -Community contributions are welcomed -Institute a process for integrating changes from community into mainline

Sophisticated Design
-Developed using object-oriented methodologies to minimize lines-of-code and maximize code reuse
-Set a new minimum-standard for code quality in an open-source RTOS
-Target key architectures in each category
--32-bit: ARM Cortex-M0, M0+, M1, M3, M4
--16-bit: MSP-430, PIC24/dsPIC
--8-bit: Atmel AVR
-Feature competitive with the best the commercial offerings
--Fully-deterministic scheduling
--Implement priority inheritance for Mutexes
--Variety of blocking objects, events, and IPC mechanisms
--Tickless, dynamic timers for maximum power efficiency
--Dynamic threads
--Threadsafe, determinstic, fixed-block-size heaps
--Device driver APIs to abstract hardware interfaces from code
-Designed as a Platform for middleware
--Graphics, GUI, filesystems, network stacks, and virtual machine services can be implemented
--Accompanying cross-platform software design tools and utilities
-Lightweight standard library equivalents
-Per-feature configuration that can be generated by tools

Easy to use, easy to learn, easy to understand
-APIs are written in plain English, with readability in mind
-Code is partitioned into modules by feature and logically abstracted
-Well documented source code – not just API documentation, but annotated implementation.
-All documentation is generated from source using standard tools, such as doxygen
-Easy-to-understand examples for all core features
-Minimum use of #ifdefs, limited to compile-time feature configuration. -CPU-specific code is clearly separated into separate modules

Support a variety of use cases
-Export as source, and compile directly into projects
-Build RTOS as a library, link against projects
-Build application as part of an integrated build system

Reliable, Robust, and Performant
-All features have unit tests that are easily run on any supported target or simulator
-Performance profiling suite available for all key benchmarks
-Employ extensive use of secure and defensive coding techniques to ensure reliability
-Built-in debugging and instrumentation to catch critical errors and trace execution at runtime

I think that sounds like a pretty good starting point – and nothing I've seen in the open-source community provides all of these. Who knows - maybe I should take up that challenge.

While that's my wishlist - what would you like to see in your ideal free-and-open RTOS?

Embedded Device Drivers in FunkOS – Example

The following is an example of how to use the FunkOS UART driver on an ATMega328p, as requested in the comments of the previous post.  Using the newly-added UART driver for from FunkOS R3 as an example, this example demonstrates both  how to configure and initialize a device, as well as how to use it in a multithreaded environment.

#include "types.h"
#include "kernelcfg.h"
#include "driver.h"
#include "drvUART.h"
// Define the sizes of the UART TX and RX FIFOs 
#define UART_TX_SIZE        (64)
#define UART_RX_SIZE        (64)
// Define the UART TX and RX FIFOs
static UCHAR aucRxData[UART_RX_SIZE];
static UCHAR aucTxData[UART_TX_SIZE];
// Configure the UART's default driver state
static UART_DRIVER_STRUCT stUART = 
{
    {
        //--[Driver info]--------------------------------------------------------
    "UART0",             //*szName;                
    DRIVER_UNINIT,        //eState;                
    DRIVER_TYPE_UART,    //eType;                
    //--[Driver functions]---------------------------------------------------
    UART_Init,            //pfConstructor;
    UART_Start,            //pfDriverStart;
    UART_Stop,            //pfDriverStop;    
    UART_Control,        //pfControl;    
    UART_Read,            //pfDriverRead;    
    UART_Write,            //pfDriverWrite;
    // Do not initialize mutex
    },
    
    UART_TX_SIZE,    //ucTxSize;                        //!< Size of the TX Buffer
    0,                //UCHAR ucTxHead;                //!< Head index
    0,                //UCHAR ucTxTail;                //!< Tail index 
    
    UART_RX_SIZE,    //UCHAR ucRxSize;                //!< Size of the RX Buffer
    0,                //UCHAR ucRxHead;                //!< Head index
    0,                //UCHAR ucRxTail;                //!< Tail index
    
    0,                //BOOL bRxOverflow;                //!< Receive buffer overflow
    
    aucRxData,        //UCHAR *pucRxBuffer;            //!< Receive buffer pointer
    aucTxData,        //UCHAR *pucTxBuffer;            //!< Transmit buffer pointer
    
    57600            //!< Baud rate
};
static TASK_STRUCT stMyTask;
static UCHAR aucMyStack[128];
static TASK_STRUCT stIdleTask;
static UCHAR aucIdleStack[128];
void MyTask(void *this_);
void IdleTask(void *this_);
int main(void)
{
    Task_Init();                    // Initialize the kernel (must be first)
    Driver_Init(&stUART);            // Initialize the UART driver
    
    Task_CreateTask(    &stMyTask,    // Create the active task
                        "Task1",
                        aucMyStack,
                        128,
                        1,
                        MyTask);
    Task_CreateTask(    &stIdleTask, // Create the idle task
                        "Idle",
                        aucIdleStack,
                        128,
                        0,
                        IdleTask);
    // Add the active and idle threads to the scheduler
    Task_Add(&stMyTask);
    Task_Add(&stIdleTask);
    Task_Start(&stMyTask);
    Task_Start(&stIdleTask);
    // Start the kernel (program continues @ threads)
    Task_StartTasks();
}
//-------------------------------
// Active thread function
//-------------------------------
void MyTask(void *this_)
{
    USHORT usWritten;
    UCHAR aucData[16];
    int i;
    // Start the thread by starting the driver
    Driver_Start(&stUART);        
    {
        // Set the baudrate using the baudrate command
        ULONG ulData = 57600;
        Driver_Control(&stUART, CMD_SET_BAUDRATE, &ulData);
    }
    // Set the data to keep spamming to the UART (ABCDEFGHIJKL..)
    for (i = 0; i < 16; i++)
    {
        aucData[i] = 'A' + i;
    }
    aucData[15] = 0;
    // Keep writing the same string to the UART over and over again
    while(1)
    {
        usWritten = Driver_Write(&stUART, aucData, 16);    
        Task_Sleep(200);
    }
}
//-------------------------------
// Idle thread function
//-------------------------------
void IdleTask(void *this_)
{
    volatile int i = 0;
    while(1)
    {
        i++;
    }
}

The Value of Embedded Device Drivers

Today's embedded processors feature scores of easy-to-use peripherals that can usually be accessed directly through a series of memory-mapped registers, which are easily read and written through C/C++ or assembly code.

Flip a few bits here and there and all of a sudden your processor is sending data, toggling I/O lines, counting time, and making all sorts of logic dance! It's an elegant, flexible, and powerful way to control a wide variety of hardware, but it certainly doesn't help developers produce elegant, portable, or maintainable code.

To use a cliche - "with great power comes great responsibility."

What I'm referring to is the practice of separating peripheral I/O code from application code using device drivers.

And just like a teenager getting behind the wheel of Dad's car for the first time, many embedded developers take the "keys to the register set" and run directly into the proverbial garbage can before making it out of the driveway.

Chanting the mantra of code space overhead and the inefficiencies of "needless" layers of abstraction, we tell ourselves that sprinkling platform-specific I/O operations in the middle of applications is a good practice - because using functions to perform those operations would result in larger, less efficient code. And for a while it does make sense - especially on tiny microcontrollers where code space is at an absolute premium.

But as applications grow, features get added, and routines start fighting for resources, this approach falls apart. Our "streamlined" code becomes increasingly difficult to read and maintain, portability goes out the window, and reentrancy issues start to pop up making it difficult to guarantee the reliability of the system. Developers then resort to all sorts of trickery and work-arounds to plug the holes on a sinking ship.

It's at this point that programmers typically start looking at grouping related peripheral functions into modules to offer some level of abstraction. The result is a series of modules containing all of the I/O operations for the various peripherals written as functions - and this is often sufficient to get the application back into a usable shape.

While it's a good start, ad-hoc libraries (AHLs) are not device drivers.

The difference between ad-hoc libraries and a device driver is in how they are used at the application level:

While a board support package provides functions to control peripherals, each will typically have a separate API, and are generally specific to the peripheral. As a result, applications written to use AHLs are tied to a given platform, making portability difficult to achieve. Applications attempting to be portable using AHLs are typically easy to identify by the unreasonable amount of #ifdefs scattered throughout; a hallmark of unreadable, unmaintainable code.

These problems can all be avoided by abstracting AHLs through a device driver framework. While it might intuitively sound like there would be a lot of overhead involved in implementing a driver "framework", it really doesn't take much effort at all (and I'll demonstrate it in this article). In fact, most capable RTOS's include a formalized device driver framework which saves you the work.

A device driver needs to be able to perform the following operations:

• Initialize a peripheral
• Start/stop a peripheral
• Handle I/O control operations
• Perform various read/write operations

At the end of the day, that's pretty much all a device driver has to do, and all of the functionality that needs to be presented to the developer.

As a result, we need an API that contains the following functions:

• Initialize
• Start
• Stop
• Control
• Read
• Write

A basic driver framework and API can thus be implemented in six function calls - that's it! You could even reduce that further by handling the initialize, start, and stop operations inside the "control" operation.

In C, we can implement this as a data structure to abstract these event handlers, and a series of wrapper functions to call them, described below.

First, we define function pointer types for the different handlers in the framework:

typedef BOOL (*DRIVER_INIT)(void *pstThis_);
typedef BOOL (*DRIVER_START)(void *pstThis_);
typedef BOOL (*DRIVER_STOP)(void *pstThis_);
typedef USHORT (*DRIVER_CONTROL)(void *pstThis_, USHORT usID_, void *pvData_);
typedef USHORT (*DRIVER_READ)(void *pstThis_, UCHAR *pucData_, USHORT usLen_);
typedef USHORT (*DRIVER_WRITE)(void *pstThis_, UCHAR *pucData_, USHORT usLen_);

Which allows us to design a generic device driver "base" structure as follows:

typedef struct
{
// Function pointers to driver event handlers
DRIVER_INIT pfInit;
DRIVER_START pfStart;
DRIVER_STOP pfStop;
DRIVER_CONTROL pfControl;
DRIVER_READ pfRead;
DRIVER_WRITE pfWrite;

// Variables common to all drivers go below (driver name, type, state)
} DEVICE_DRIVER;

The API functions to operate on this driver structure can then be declared:

BOOL Driver_Init(DEVICE_DRIVER *pstDriver_);
BOOL Driver_Start(DEVICE_DRIVER *pstDriver_);
BOOL Driver_Stop(DEVICE_DRIVER *pstDriver_);
USHORT Driver_Control(DEVICE_DRIVER *pstDriver_, USHORT usEvent_, void *pvData_);
USHORT Driver_Read(DEVICE_DRIVER *pstDriver_, UCHAR *pucData_, USHORT usLen_);
USHORT Driver_Write(DEVICE_DRIVER *pstDriver_, UCHAR *pucData_, USHORT usLen_);

These functions simply act as wrappers for the handler functions set in the DEVICE_DRIVER struct. For example, the Driver_Init function might look something like this:

BOOL Driver_Init(DEVICE_DRIVER *pstDriver_)
{
return pstDriver_->pfInit((void*)pstDriver_);
}

Now, in this example we aren't checking that pstDriver_ is valid (which you would want to do in a *real* implementation), but it does illustrate the concept of wrapper functions.

You'll notice that the handlers function types each have a void pointer as the first argument, where the wrapper API calls use DEVICE_DRIVER pointers. There's a good reason for this, and it's related to inheritance.

In this framework, each individual device driver is built on top of the the basic device driver type. Since C doesn't have built-in inheritance found in C++ or Java, we must implement this feature using the knowledge of how data is aligned in structures. Implementing a device driver based on the "base" driver type is shown in the following example:

typedef
{
DEVICE_DRIVER stDriver; //!! Must be first

// data unique to this kind of device driver goes below...
USHORT usMyData;
} MY_PERIPHERAL_STRUCT;

BOOL MyPeripheral_Init(void *pstThis_);
BOOL MyPeripheral_Start(void *pstThis_);
BOOL MyPeripheral_Stop(void *pstThis_);
USHORT MyPeripheral_Control(*DRIVER_CONTROL)(void *pstThis_, USHORT usID_, void *pvData_);
USHORT MyPeripheral_Read(void *pstThis_, UCHAR *pucData_, USHORT usLen_);
USHORT MyPeripheral_Write(void *pstThis_, UCHAR *pucData_, USHORT usLen_);

Specified handler functions for the driver are assigned to the function pointers when declaring the structure as follows:

MY_PERIPHERAL_STRUCT stMyPeripheral =
{
{ // set the handler functions for the base driver struct
MyPeripheral_Init,
MyPeripheral_Start,
MyPeripheral_Stop,
MyPeripheral_Control,
MyPeripheral_Read,
MyPeripheral_Write
},
0xBEEF // initialize driver pecific data
};

This looks simple- and it is, but something very interesting happens when we create a structure this way. In C, the first element in a struct will always have the same address as the struct itself, so if we have a struct of type MY_PERIPHERAL_STRUCT, it is completely valid to re-cast the struct to type (DEVICE_DRIVER*). This re-casting to a parent type is the essential element of inheritance which gives us the ability to create a series of device drivers that are all accessed through a consistent interface.

As a result, any specialized device driver type can be cast back to the DEVICE_DRIVER type and used with the API functions we have already defined! The driver's handler functions are responsible for re-casting these pointers back to the appropriate type when used internally (i.e. to MY_PERIPHERAL_STRUCT).

Consider a system with drivers for I2C, SPI, and UART peripherals - under our driver framework, an application can initialize all of these peripherals and write a greeting to each using the same simple API functions for all drivers:

Driver_Init((DEVICE_DRIVER*)stMyI2C);
Driver_Init((DEVICE_DRIVER*)stMyUART);
Driver_Init((DEVICE_DRIVER*)stMySPI);

Driver_Start((DEVICE_DRIVER*)stMyI2C);
Driver_Start((DEVICE_DRIVER*)stMyUART);
Driver_Start((DEVICE_DRIVER*)stMySPI);

Driver_Write((DEVICE_DRIVER*)stMyI2C, "Hello World!", 12);
Driver_Write((DEVICE_DRIVER*)stMyUART, "Hello World!", 12);
Driver_Write((DEVICE_DRIVER*)stMySPI, "Hello World!", 12);

To me, the benefits of using this kind of device driver framework are obvious:

• It provides a consistent interfaces for controlling and accessing peripherals. All a developer needs is to learn a single driver API, with a few specific control events for each peripheral type.
• The internals of the device driver are completely obscured from the user - all that needs to be provided to the application is a pointer to each driver.
• Creates an automatic hardware abstraction layer, leading to better code reuse for the drivers and portability of application code.
• Reduced module coupling in application code, as the app doesn't need to call AHL functions directly.
• The wrapper functions can incorporate resource protection mechanisms to prevent deadlock in a multi-threaded system.

This approach was used in the Mini-Markade project to successfully abstract the display and joystick drivers to allow the games and frontend to compile under Borland C++ Builder on windows as well as the Atmel atmega328p platform without requiring any modifications or #defines to the application code. The overhead on the embedded side was roughly 100 bytes of code space, and in the context of a 32kB application, the benefit greatly outweighs the cost.

See the latest release of FunkOS for more information on how formal device drivers can be used in embedded systems applications.

The Mini-Markade Lives!

It's the first post on the blog, and I thought I'd start off by sharing a project of mine.

I've been working on a tiny arcade machine based on an AVR atmega328p microcontroller in an attempt to win a Guiness record for the category of "world's smallest arcade machine". Well, it turns out that Guiness doesn't give awards for "smallest" anything anymore, but that hasn't stopped me from building it all the same.

Originally, I got the idea after seeing a number of "Paper Arcade" models popping up on the 'tubes. Given the dimensions from those cabinets, I figured that it wouldn't be too hard to shoehorn an AVR board and an OLED display into that form factor - and while it was a challenge, I've been surprised at how well everything has come together.

The end result is what you see below - a fully functional version of the "Paper Arcade" capable of playing my own interpretations of classic arcade games.

My arcade machine (dubbed the "Mini-Markade") currently has versions of Tetris, Space Invaders, and Breakout - all selectable from a frontend application. Smooth 60fps graphics are displayed on a 1.5 inch OLED display (by 4D Systems), while controls are input through a custom Atari-compatible joystick made from random parts off digikey. Powered by 2xAAA batteries, the system will run for 12-15 hours between changes.

The Markade also runs FunkOS, my very own RTOS for low-resource microcontrollers, which you can download from my sourceforge page (http://funkos.sourceforge.net).

Between my RTOS, my own implementations of Tetris/Invaders/Breakout, and all the device driver and frontend code, the system uses about 1800 bytes of RAM and all but 80 bytes of the available 32KB of flash. Not bad!

The pics below show the cabinet almost fully assembled - all that's left to do is tack the back panel on, add an external power switch and give it some paint. Standing around 3.5 inches tall, the mini-markade replicates the design of an old Defender cabinet - with all dimensions implemented to scale.

More to come in the coming days and weeks!