I am in the process of writing a paper related to validation execution systems.
This kind of system is recent and there is no company close to me where I can
make questions so I ask for your usual cooperation.
I would like to know the rationale for the implementation, the experience during
the implementation and use of the system, satisfaction level, before & after
comparison. Also I would like to know about implementation time, ROI, etc.
Systems that I know for this purpose are:
Valgenesis
QA Valid
CA Validator
I have talked to many people about this kind of ystem but they are kind of
skeptic about it. I understand that the implementation of such system is
delicate as it requires a cultural change of the process, not to add that almost
each company has its own recipe, even within big corporations, divisions could
have different validation approaches. But I do believe that Validation is a
process overdue to be automated. Any help is appreciated.
Please, have in mind that my purpose is to develop a paper that may be
published. Thanks for your help.
Rationale to automate the validation process is to improve the
consistency, compliance, reduce the validation cycle time and
release the system in to production on time. In fact, we are
experiencing very good improvement in our corporate validation
process after automation. We are little political, need to route the
protocol to various reviewers (Sys owner, validation, QA, IS or Mfg)
for both review and approvals. Just the review process alone takes
25-30 days in the paper based system, with normal review period (5
days). Similarly approval takes another 20 days. Since they are
cross functional users, the review and approvals are not higher
priority for them.
By automating the review and approval workflows (both after protocol
development and testing), we are experiencing 50% reduction in
review and approval cycles. In the automated systems, you could
route the protocol concurrently and they can share their comments.
In addition to time saving, you could find the real time validation
status of any system across the site.
Sorry guys but I don’t think that you are using the word "automation"
correctly. What parts of the process are actually automated?
Call me a nit picker but what you are describing is parallel
processing due to the usage of electronic file formats. This is not
really automation in my book. People are still doing the work not
machines, automatons or robots.
I think you can mislead people to think that validation can actually
be automated, see other post for an expansion on this view point.
Automation in the application we implemented – i.e no human interaction
other than clicking OK:
Creating a document from content (includes any type of “validation”
content, plan, requirements, tests, risks, regulations, in any type of
format or style, into any type of output etc.)
Routing content and documents for review or approval
Traceability matrix generated from links between compliance content
(the links have to be “thought” of but once entered they stay there until
content changes and they become suspect).
Running tests (can link to any script execution or program execution
– which is absolutely not all validation testing but it is some of it.)
Assessing “in or out of compliance” state (content, documents or
projects have a colour status based on their completion – viz a dashboard)
Creating a delta project that differs in some way from the original
(e.g. change control) requires only the manual selection of applicable
requirements – all other components of the protocol are automatically
generated.
Eliminates all paper and therefore automates the task of filing and
storing and retrieving.
Risk assessment. All compliance content can be assigned a risk
factor. Any exposed risks are automatically flagged.
All of the above occurs based on rules and logic programmed within the
software.
I do not quite understand what you mean about “parallel processing using
electronic file formats”, as the software application when implemented does
not need to generate any file format (it can generate output into a variety
of file formats, but the whole concept is that it eliminates any paper,
instead storing the validation protocol as content in a database).
People are still required to do some of the validation work. Not
surprisingly, the work that remains after automation is actually the value
added work for which skilled personnel are needed, but instead of wasting
their time within spreadsheets and Word templates and formats, and routing
and storage and cutting and pasting from one place to another and
controlling versions and creating status reports and executing routine
tests, those tasks have been automated.
Exactly what value the automation of these tasks calculates out to probably
depends on your organization. If you have costs within any of the areas
above, they can be positively impacted (i.e. they will become lower using
the software than not using it). In terms of intangible benefits (savings
that are often difficult to calculate) possibly the largest comes from the
ability to easily generate a new validation protocol when a supplier
releases a patch for the software. As this is somewhat unique to CSV you
would see less benefit in this area when using the software to validate
equipment. Many customers that I have visited would easily spend tens of
thousands of dollars each year though maintaining paper validation protocols
and sadly, all of the content of those protocols lost their value and
continued to lose their value from the day someone thought it up. Whether
you generate paper, or send paper documents to electronic storage vaults you
have not saved any money or realized the benefits having of reusable
compliance content in a database.
Call me a nit picker, but if you can save manual effort using a software
application, then that is considered automation. If you do not do any of the
above tasks, then perhaps we should be discussing what validation is rather
than automation?
Hey Juan
I don’t want to get down on you but there is just no way you can
automate the validation process, in its fullest usage of the
term “validation”. You may well be able to automate the management
of validation documentation, this is merely a document control
system.
Remember testing PC based software is only a small proportion of
validation work (although from some posts on this forum you may get
the impression that only software(systems)validation exists). There
may be some opportunities to auotmate the execution of test scripts
but these have to be written first and the code to do this may
itself be subject to being “validated”, who is going to do this work?
How are you going to automate the brainstorming and creation of a
set of user requirements(URS)? How are you going to automate the
creation of the tests for IQ/OQ/PQ? How are you going to automate
the sign-off and approval of executed documentation. There is no AI,
Expert System or Neural Network powerful enough to do this.
Maybe I misunderstand what you mean by “automated validation”, but
from my experience this would only be possible (as mentioned above)
for a thin slice of validation work, and comes with its own overhead.
[quote=maryacton]Hey Juan
I don’t want to get down on you but there is just no way you can
automate the validation process, in its fullest usage of the
term “validation”. You may well be able to automate the management
of validation documentation, this is merely a document control
system.
[/quote]
I would agree with you Mary. It is only possible to ‘automate’ the validation “Framework/Process” you operate within i.e. an automated documented control system/process used to produce, review and even approve, validation deliverables. You would still require a huge amount of human input to populate all of the specs., protocols, plans and all of the other project deliverables, not to mention the meetings, suppliers and vendors etc.
There is, however, one small exception used within IT Systems Validation sometimes… Mercury Test Tools; like Test Director. These tools allow you to populate your test scripts directly from uploading the requirements into Test Director which also provides you with an automatically produced traceability matrix. BUT EVEN THEN, you would still have to manual draft and load your requirements into Test Director i.e. it’s only an automated testing tool, and does not provide/develop all of the other deliverable around it. Also, it is only used sometimes in ‘IT Systems Validation’ and unlikely be used in execution systems validation.
There also seems to have been some confusion in the previous messages around the terminology of “automation” in the validation of “execution system’s” original message posting. My take on this is as follows:
Automation - It is not possible to automate the whole validation life-cycle. It is only possible to automate the “framework” your validation life-cycle operates within. This is an important difference for people to understand i.e. you still have to do the work! Note: I was involved in a major study with a top 2 Pharma (global) company who ran a project, with lots of money, to try and do this and it could not be done with any real benefits that could justify a change in ways of working etc. Only the framework of the validation process itself could be “automated”….so not much of a saving in reality.
Validation of “Execution Systems” - The terminology of the original post is confusing here. I suspect the original poster is talking about “Manufacturing Execution Systems (MES)” a fairly new technology which ‘automates’ and introduces a ‘paperless production line’ i.e. literally no paperwork on the production lines…just touch screen controllers for the operators to input their responses in to for batch record input and print-outs. This is automation at the production line level on an “already validated system/line” i.e. you would still have to validate the MES system in the normal fashion with humans, before the line itself could be used to “automatically” produce drug product etc.
This is just my interpretation and opinion. I hope it helps.
Did you ever write your paper on this subject? And if so could you post it here so that we may read it? It was a very interesting subject for many people on here. Thanks
IQ/OQ/PQ protocol development
Test Execution
IQ/OQ/PQ report development
Finally the review and approvals.
automating the Validation activity lies in protocol development and report generation, ofcourse approvals can be done by emplyoing electronic signatures. But the test execution and the input for report can be made manually. But using camtasia the full execution can be recorde automatically and that the approvar have to watch the video and give his approval.
Hi THere-
Regarding Camtasia. we have been using it to record our validations…
Have you ran into any trouble with it interfering with the software you are actually testing??
Thanks!
[quote=maryacton]Sorry guys but I don’t think that you are using the word "automation"
correctly. What parts of the process are actually automated?
Call me a nit picker but what you are describing is parallel
processing due to the usage of electronic file formats. This is not
really automation in my book. People are still doing the work not
machines, automatons or robots.
I think you can mislead people to think that validation can actually
be automated, see other post for an expansion on this view point.
Regards
Mary[/quote]
Well said! I was confused there for a while, but realizaty hit me in the nick of time. Where people are involved in validation, electronic communication is essential. Automating validation will take us back 20 years!