Code Review

In terms of software regulated as a medical device is it essential for source code to be reviewed by a second party as part of software validation?

We currently have one Software Engineer who writes our code. Nobody else is qualified to read the actual code and look for errors. Our validation strategy mostly involves rigorous bug testing and is controlled by the Software Engineer.

I have read the FDA document “General Principals of Software Validation” which provides some guidance on source code evaluation but doesn’t dictate who must do it. We review and evaluate the source code, however it is by the same person who wrote it.


First I view code review as a verification effort and not validation.
You are not required to have code reviewed. Code review is a way to show that the output of the coding effort meets its input (a design). (Code review generally does not show that the code meets the intended use of the product.) There are other ways of showing that the code output meets the design input. One way is with unit and integration testing run by the engineer.

If you only have one coder, then I would suggest using unit/integration testing. Having the coder as the sole review the code would not be defensible as a code review. If you have another smart individual, you can improve the use of unit/integration tests by having them reviewed by someone else. The review of the tests can be at a much higher level than a code review by having the review of the tests against the design.
You can strengthen the process and your defensible position through the use of tools like static analyzers, and code coverage analysis.

I am sorry, the statement that code reviews are not required is so wrong, it just has to be corrected.
During an inspection of a very large US based manufacturer, the FDA inspector was reviewing protocols for autoclaves, when he suddenly started taking notes. There was something that attracted his attention. He asked for the code reviews, but he still was not satisfied. He asked for a copy of the code in use, to be uploaded from the autoclave onto a disc, and overnight he had it analysed.
The outcome confirmed his suspicions some of the completed tests in the protocols were not possible, since the software was not capable of the functionality. The protocols were fraudulent. The outcome of this; people were fired and prosecuted by the FDA, and the company was fined very severely.
This is not fiction the warning letters are in the FDA, Freedom of Information file.

Somewhere I have a sad anecdote that was used to point out to inspectors that even a relatively simple software module, with three infinitely variables of 0 to 100 taken to two decimal points, has many millions of test points if 100% testing is undertaken.
So you test for accuracy and response at agreed set points, and you review the code for scope, completeness, range and continuity.

If I can find the tale I will publish it.

Alex Kennedy


What regulation requires code inspections? Is 100% code inspection required?

I do agree that code inspections are a useful tool in the verification toolbox. We use them on just about every project (if not all). I believe, though, that a good unit/integration/functional test suite, IMO, is more effective and efficient at confirming requirements compliance than a code inspection. We generally limit code inspections to “safety critical modules” or to requirements that cannot be verified any other way.

I would also assert that a company bent on fraudulent activities, could do so in code inspections as well.

The anecdote I mentioned in a previous posting.

When have you tested enough?

A new car is in development, each day it is out on the test track, each day the driver is pushing the car to new extremes. The car uses the very latest computer technology in all the on board systems. Everyone is relaxed and quite jubilant because after thousands of hours of bench testing, the car has progressed to the final stages of performance testing and is still ahead schedule.

One of these computer systems is used in the braking system. This braking system computer uses a module to interface the air temperature with the road speed and disc temperature. The module output is then used to apply the maximum safe retardation effort.

It is a crisp dry winter morning, and the car is out on the track and consistently lapping at an average speed of over 100 MPH. They watch as the car comes down the main straight at maximum speed, and then quite unexpectedly does not appear to attempt to slow down, and goes straight across the approaching bend, through the barriers and into fencing, resulting in a horrendous impact, which tragically, the driver does not survive.

Months later, the accident inquiry revealed; the software used in the brake system module, applied a predetermined formulae to an air temperature range from 50ºC centigrade to 0.1ºC, at which point this formulae changed and a different predetermined formulae was applied from -0.1º to -50ºC. The temperature on the day of the accident was exactly 0ºC.

Everyone had missed the fact that between +0.1 and -0.1, the software would do nothing, and since the air temperature was defined to two decimal points, there were in fact nineteen points at which the car braking system was inhibited. Nineteen points when the car would be without brakes.

So the moral of the story - you can test and test and test, but you don’t know what the code will do until you read it.

Sadly the story is true.

Alex Kennedy

I would suggest that you have your code independently reviewed. I’m sure I could review it as could many other consultants on this site - your pick - obviously the required expertise with your programming language, controller platform etc. is required.

You need to have SQAP, SCMP documents in place that any reviewer would refer back to just to make sure that you are following your own rules. You can get all the required guidelines from the IEEE in addition to your GAMP kit. If you’re missing these documents, then the reviewer should be able to provide them.

A few years ago it would have been less necessary, but if some sort of audit happens then you’re goose is cooked. Also, it is just GEP to have a source code review as part of each project. Whereas regulations may not ask for it GAMP, IEEE etc. certainly do.