Risk assessment in software developer

GOOD MANUFACTURING PRACTICE (cGMP) RISK ASSESSMENT
1

Hi all. I am new here and wanted to ask a question in regards to risk assessment.

I am currently working for a software developer, and need to assess the risk of all tools used when developing the product. Of course we are planning to fully validate the product (software used in analytical laboratories, used in diagnostic and data management), but I am questionning whether I should validate all tools used when developing.

I have prepared a risk assessment template, trying to give numeric values to answers and grade each entity assessed (i.e.: 0 @ 25 - No risk, 26@45 - low risk, etc…). What should be considered for risk assessment?

  • Coding tools (i.e.: source safe, etc)?
  • Client request/bug report tool?
  • Electronic document management system (used to manage SOP and such docs.)…this one containes e-sign.
  • Any Microsoft apps used (i.e.: Excel for calculations maybe?).

I know how to assess risk, I just need a better idea of the bigger picture. Our product is a medical device, and we are located in Canada (Health-Canada medical device requirement) but also sell to the US (therefore part 820).

Can anybody please shed some light on my questionning? Thanks!

Edit: one more question…as per the FDA, would a software automatically need full validation when it uses e-signatures, even though the risk is very low?

2

Welcome to our community

Validating all tools, ask yourself why would you do this?

Are the tools going to be used in a regulated environment, I would say no so why do they need to be validated!

No need to validate these applications, these should from part of your quality system. Your quality system is the key when you are validated. Companies will audit your quality system not to see if you have validated a coding tool.[/quote]

You can contact me through my company if you need any more help with the project.

Hope this helps somewhat

3

Just a few follow up notes…

I don’t think Graham is making a blanket statement about tools validations in regulated industries (Graham, correct me if I’m wrong). If you are in a regulated industry (and it sounds like you are), the software has a high level of criticality (e.g., can make a diagnosis), and the tools used can directly affect the quality (e.g., a compiler) then there is probably good reason to validate the tool.

I agree that tools such as CM, bug management, etc. pose a much lower risk and probably do not need much in the way of validation, if any.

Mention is made of a doc control system that utilizes e-sigs. If you are regulated under the US FDA then this would fall under Part 11 and validation would be expected. I don’t know about other regulatory agencies.

4

[quote=Cyclop3000]Hi all. I am new here and wanted to ask a question in regards to risk assessment.

I am currently working for a software developer, and need to assess the risk of all tools used when developing the product. Of course we are planning to fully validate the product (software used in analytical laboratories, used in diagnostic and data management), but I am questionning whether I should validate all tools used when developing.

I have prepared a risk assessment template, trying to give numeric values to answers and grade each entity assessed (i.e.: 0 @ 25 - No risk, 26@45 - low risk, etc…). What should be considered for risk assessment?

  • Coding tools (i.e.: source safe, etc)?
  • Client request/bug report tool?
  • Electronic document management system (used to manage SOP and such docs.)…this one containes e-sign.
  • Any Microsoft apps used (i.e.: Excel for calculations maybe?).

I know how to assess risk, I just need a better idea of the bigger picture. Our product is a medical device, and we are located in Canada (Health-Canada medical device requirement) but also sell to the US (therefore part 820).

Can anybody please shed some light on my questionning? Thanks!

Edit: one more question…as per the FDA, would a software automatically need full validation when it uses e-signatures, even though the risk is very low?[/quote]

Yes Yodon I wasn’t making a blanket statement just trying to give flavour of what is involved.

Must have been in a rush…

5

Ok so to be honest, there is a hyped discussion here at the moment, whether we should validate tools or not.

As QA I say we should validate tools that affect our product (yes it is a critical software used in diagnostic, Class 2 medical instrument). I wanted to run a risk assessment on each tool, and see if I can mitigate some risk in order to not have to validate everything.

Management disagrees, and argues that we should not validate any tools, their argument being “we don’t validate Microsoft products, why should we validate the compiler, bug tracking system, e-document management system, etc…”

So I am torn, I believe I am right (after a few years in the validation field I think I must have picked up a few ideas here and there hehe) and I understand why management does not want to validate all. Heck I don’t want to either, less work for me!

So the real question is, do we validate the tools that “could” affect the product, or not? You mentionned not validating the bug tracking system…why not? What if a problem log gets lost in the database, and we don’t look into it, and eventually a doctor somewhere makes a wrong diagnostic caused by this problem and a patient dies? Should we use “probabilities” to explain why we did not validate this bug tracking tool?

And what about e-sign in the electronic doc manager? Just because there are e-sign we have no choice but to validate it, as per FDA? So we would validate this tool, for which I do not see any possible impact on health (we could always find something, but it would be a long shot) and not the bug tracking system, which in my opinion is more prone to causing problems?

I hate when things get subjective like this…everybody argues and everybody’s right in a certain way. Anyways, I appreciate any ideas :slight_smile:

Thank you!

6

I think the tail is wagging the dog here.

You use a risk assessment to gauge the level and depth of validation that is required to verify that your tool is faultless and operates in accordance with the manufacturer’s specification.

So, what is the level and depth of validation that is required?

The question that prompts thinking on this matter is “What testing do I have to do, to verify that all actions carried out by the tool are 100% correct”. If the tool’s function has been a relatively simple one, this should not be difficult. However if the tool’s function has been complex and involved integrating and or differentiating several variables, then complete testing is impossible, since the combination of test points would be infinite. You now have to revert to Life Cycle Validation.

This involves validating the design and development cycle of the tool. I have found on every occasion (and there have been many) that this was impossible, simply because the design development of the tool, was not structure or documented in accordance with the requirements for life cycle validation, and the cost of doing this retrospectively was prohibitive.

This has always been the problem using software manipulating tools. Indeed in many cases it immediately inhibits their use.

Do not get your Risk Assessments (RA) mixed up (there are several occasions when a RA is used). You are required by cGMP’s to justify your validation scope, you are also required to document your justification. The outcome of this RA is your justification and as such is predicate required data.

Alex Kennedy

www.fda-compliant.net

7

Cyclop3000,

With what you have presented here, you are right and management is wrong.

I guarantee that if audited by FDA, they will look to see that the software tools are validated for their intended uses. As the others suggests, you can use a risk-based approach to determine to what extent (depth) you will need to validate the tools and the product software itself. I always recommend a risk-based approach to establish what will require validating. As Alex points out, you’ll need to document your validation strategy including your rationale for a given level of validation.

Also, as you probably know, even your Microsoft products can be brought under FDA scrutiny where they are used to support QMS functions (e.g. CAPA admin, Engineering data analysis, etc.). It sounds to me that your management may be a decade or two out of step. Be patient with them, but help them understand your fuller obligations.

Good luck,

Kevin