Anyone with experience in open-source firmware in medical devices

I’m looking for some guidance applying FDA’s software validation recommendations to open-source software used in an embedded application within a class III medical device.

Much of the operating system and device drivers come for free, downloaded off the web, from the open-source community (controlled by organisations such as “kernel.org”). Another open-source community has provided adaptations to allow the software to run on non-Intel processors for embedded applications. The tools to build the kernel are also open-source and free, and in most cases are built specifically for the target from “standard” resources.

As you can imagine, there’s a lot of software here that comes with no explicit warranty, and cannot be easily inspected or validated other than when tested as a system running our application. Our intention is to describe, review and test only the code that we’ve written. The rest would be tested during system validation. At the moment, even code reviewing our code appears unrealistic given the time it requires. Do you have any comments, or recommendations regarding this?

I have experience of open source software, but not of medical devices. As I understand it, class 3 medical devices support or sustain human life, so I imagine you’d like the software not to fail :wink:

On open source software in general: software costs time and effort to maintain. Many companies that use open source software engage engineers to maintain it, because they cannot be sure the open source community will fix the bugs they care about and add new features in the timescales their customers demand. They find it’s not really “free” and the costs of maintaining it, in terms of software engineering effort, can be as high as those of purchasing a commercial product. (I did the math once, it was scary.) That’s not to say open source is bad, it can be good and there can be benefits; but the apparently-free downloads can be deceptive in terms of costs. What this means is you might want to identify one or a few engineers to ride herd on the open source stuff, and thoroughly validate new versions before incorporating them into your product.

On verifying it: you could inspect the code since it’s open source, but I imagine that would be expensive and low in value because there would be too much to learn. I assume you’ll test it. I would caution you on managing changes: when you download a new version, regression test it very carefully to make sure that everything still works properly. One open source product I dealt with a few years ago was notorious for being unstable: every new feature would break one or more existing features. The “goodness” of the software quality depends very much on the team leading the open source development.

On inspecting your own code: research indicates that code inspection is very cost-effective at finding bugs. In real time embedded systems, which are common in medical devices, there are bugs (to do with synchronising real time events and memory management) that simply cannot be found in testing, occur once in a blue moon (that once could kill someone) and can easily be found in code inspections (e.g. for every allocation of memory, did we return it to the heap when finished?). Cost-effective code inspection doesn’t usually mean inspecting all the code. Typically I’d recommend sampling the complex code, the brand new code, and stuff written by novices; often, the engineers will know what they’d like someone to take a second look over. If you find errors, sample more of that kind of code; if not, less.

Here’s an example of code inspection: will this program work?

main ()
{
printf ("hello world);
}

Almost everyone spots the missing closing quotes – even people who can’t read C.

A final thought: What’s the cost if you let bugs out into the field? That would have some bearing on what it’s worth to manage the software quality, I’d imagine.

Hope this helps,