Why software development is important in the regulated industry

The FDA will not audit you unless you’re directly regulated (e.g. a medical device manufacturer), and in any case won’t bless your software for other companies to use.
I’ll tell you why I do software supplier audits and in general what I look for.

First, if my company is writing their own software, the expectation is that we’ll document what the user needs, what the software must do, how the software will be designed, how the software was built, how the software was tested, how the testing mapped to the requirements, how we handled any bugs or other testing discrepancies, how we’ll track bugs that remain in the software after release, and so forth. There are other things that surround and/or sit on top of the software development process that also have impact, such as whether our software development lifecycle is documented and repeatable, how we handle configuration management of the software and documentation, how we handle the training of developers and testers, whether we have separation of responsibilities for development and testing, whether we have an independent QA group that conducts internal audits, how we manage our work instructions and/or standard operating procedures, etcetera
ad infinitum.

Because we designed the software and we wrote the code and know how it’s structured, we’re expected to have design controls and software quality assurance in place. Our software specification documents should clearly define what the software must do and how it does it. Our design should undergo an review by technical folks who are independent of the effort being reviewed. Our code should be reviewed by programmers who didn’t write it. Our testing should have a lifecycle that includes (as appropriate to the technology being used) unit testing, integration testing, and acceptance testing. Our testing should include functional (input/output black box testing) and structural (code tracing white box testing). Because we know the inner structure of the software, the data we use when we run functional tests should be specified with an eye towards what might cause the software to break.

Now let’s say that instead of writing the software, we want to buy it from you. We’re still responsible for assuring that all of the above gets done, but we’re no longer the ones doing it. So we instead audit you and document what we found, and store that audit documentation along with whatever else we can do to assure that the software is fit for our purpose (the typical expectation is supplier audit + specification of our process needs + specification of how we combine your software installation with servers, networks, and other other supporting infrastructure + our own functional and performance testing). When we audit you we look for evidence of all the stuff we would expect to have in place and do ourselves had we written the code. We’ll look at your software development artifacts (to gain an assurance that the software is sound) and your quality management system (to gain an assurance that future releases will be sound, too).

If you validate your software per current industry standards it helps me in that it makes my review of your development artifacts faster. If also you declare your software to be Part 11 compliant it helps me, too, since I should be able to find all of the Part 11 technical controls as requirements in your specification documents and therefore easily traceable to the functional tests that challenged them.

Regards,

Graham,

How would you go about auditing a larger software provider, such as SAP? For instance, you plan to leverage SAP’s PM module to support your plant maintenance effort. If the structure of the module is not customized in any way or form, so essentially, only the configuration of the module has been done, to what extent would you a) need to audit the software provider b) the application of the configured module?

Regards,

Kevin

Not sure I can agree with the last part about a vendor declaring software Part 11 compliant. You can only determine if something is Part 11 compliant as you have installed and are using it. The vendor can only state that controls / mechanisms are in place to allow your use to be part 11 compliant.

[quote=SV Student]Graham,

How would you go about auditing a larger software provider, such as SAP? For instance, you plan to leverage SAP’s PM module to support your plant maintenance effort. If the structure of the module is not customized in any way or form, so essentially, only the configuration of the module has been done, to what extent would you a) need to audit the software provider b) the application of the configured module?

Regards,

Kevin[/quote]

For a company like SAP I would be pretty confident before I audit them that they have good solid quality systems in place to ensure the integrity of their software. A good starting point would be to try and speak to another company who has already audited them, perhaps you could leverage from their documents if they are kind enough to allow you.

For the application of the configured module one of the most important aspects of your audit should be to examine their configuration management process plus the usual list of good engineering practices you should look out for…bug tracking, testing methodology etc

Good point and I agree 100%, a common mistake when software companies market their software.

Good to see your awake! I’m not long day…

Just a cautionary note! In a recent audit, we were under some pressure to explain why we did not audit SAP - our auditor discovered that at that point in time less than 4% of their sales was into the life sciences industries (I don’t know what it is now - the auditor got the information from SAP website!). We did provide a satisfactory verbal explanation, but had not written down our rationale. The auditor was less than impressed.

The moral of the story - use the word “becuase” a lot - helps you write down the why of what you’re doing.

Please note, nothing in this implies any comment or criticism of SAP at all - the audit was of our company, and our documentation. The auditor’s point was that we were not properly documenting decisions - not that we were making poor decisions.

So have confidence in your suppliers (SAP or others) by all means - but document your decisions regarding performing any kind of audit or not. There’s nothing wrong with documenting a decision not to audit - because you have confidence becuase … (add your own words).

Cat,

Thanks for your thoughts and experience.

Kevin

If one thing has been drummed into me regarding audits, its this;

“if you didnt write it down…it didnt happen!”

Hi,

We have got a development project from a client wherein they have also asked us to do CSV. They had provided a deliverable matrix in which they mentioned OQ/IQ/PQ (this is the sequence) after construction (coding). They have not mentioned any FAT,UAT instead the matrix had OQ and PQ.

My question here is that can we do OQ and PQ inplace of FAT and UAT. does a seperate IQ/OQ/PQ is required duing installation. This is for a web based application.

In this case how can we document the CSV.

Regards
shan

Hi Shan

The FAT & SAT are vendor related documents. A new definition of validation has been slowly adopted by the regulators. This new approach starts with the URS and virtually states - if you dont have a URS in place you cannot validate. I have copied in some citations to demonstrate that you will not pass a regulatory inspection unless you have the full set of validation documents in place.

Get your User Requirements Specification (URS) correct and the validation task can be properly scoped. We use a three level URS to achieve the regulatory requirement for traceability from the the individual requirements as detailed in the URS through to the DQ/IQ/OQ/PQ.

Everything below is word for word from the FDA:

Excerpt from guidance document.

A documented software requirements specification provides a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) and (g)).

Excerpt from a recent Warning Letter,
Failure to validate computer software for its intended use according to an established protocol when computers or automated data processing systems are used as part of production or the quality system as required by 21 CFR § 820.70(i). For example:

A) Your firm uses off-the-shelf software (HEAT Help Desk) to manage customer support service calls and to maintain customer site configuration information; however, your firm failed to adequately validate this software in order to ensure that it will perform as intended in its chosen application. Specifically your firm’s validation did not ensure that the details screen was functioning properly as intended. The details screen is used to capture complaint details and complaint follow-up information which would include corrective and preventative actions performed by your firm when service calls are determined to be CAPA issues.

B) Off-the-shelf software (Microsoft SharePoint) is being used by your firm to manage your quality system documents for document control and approval. However, your firm has failed to adequately validate this software to ensure that it meets your needs and intended uses. Specifically at the time of this inspection there were two different versions of your CAPA & Customer Complaint procedure, SOP-200-104; however, no revision history was provided on the SharePoint document history. Your firm has failed to validate the SharePoint software to meet your needs for maintaining document control and versioning.

Regards
Alex Kennedy

Hi Graham/ anyone

I know this isn’t related to this topic but I didn’t know where else to look.
I am trying to get a secondhand copy of the GAMP5 book. Does anyone know where I can get one and any idea of how much i should expect to pay for it?
Any help would be appreciated

Eugene

OK -let’s make this clear …

“Validation is the responsibility of the user - not the vendor”.

Next, lets look at FDA inspections.

These work on a complaints basis - if you complain to your chemist that your blue tablets you bought are actually red, or that there was one missing, or they made you fat etc - you can bet your life a complaints card will be registered and sent to the regulators (MHRA / FDA etc etc). On looking at the complaints, the regulators will go and inspect the facility (good reason, purpose etc)

From a CSV standpoint, we are usually off the hook - data entry is a manual process (thank God) and therefore any data entry errors are usually captured by humans (detectability - I’ll write a section on this later if you like) and as you can’t “validate people” the only problem you will get is training and SOPs - the very things that get chopped when we run out of time on projects :smiley:

So this brings me around to this - “Bread’s Winderful Paradox”.

Don’t expect the documentation of a validated system to stand up by itself in front of an auditor - build it using our own audit defences. Alternatively, we can build the most compliant systems in the world - they probably wont work but they will stand up to regulatory inspection - great for business (not)

Bottom line … don’t cover your arse - cover theirs.