Chapter 2. Quality considerations
Key message: To realise fully the potential of in vitro methods and allowing them to become a key tool for a new way of doing toxicology, they need to be developed and applied in a way that scientific integrity and quality is assured.
Key content: Discusses quality assurance versus quality control, quality risk-based assessment and quality control requirements for development and implementation of in vitro methods, the types of documentation needed and quality considerations regarding the integrity of the data.
Guidance for improved practice: Control charts can be used as a powerful and simple statistical tool to help routinely monitor the quality of any quantitative process and to determine if the process is in a state of control.
Recommendations for basic quality risk assessment questions and applicability of integrity checks on cell and tissue cultures are described.
The life cycle of an in vitro method usually progresses from method development to validation to routine use. In vitro method development may benefit from many quality considerations addressed in the GIVIMP guidance document, e.g., recommendations concerning the test system, maintenance and calibration of equipment, qualification of computerised systems and training requirements. In vitro method developers, who do not work in a formal quality system, may also benefit from certain Quality Assurance (QA) requirements such as consistent documentation, an internal QA program and change control policies (OECD, 2016[1]). In summary, it is recommended that method validation be performed in a formal quality environment, while routine use of in vitro methods for safety testing should always be performed in a formal quality system environment, often meeting the requirements of GLP or similar quality systems.
The International Organization for Standardization (ISO) publishes many standards of which ISO/IEC 17025 is the main standard used by testing and calibration laboratories. ISO/IEC 17025 (General requirements for the competence of testing and calibration laboratories), originated in the laboratory accreditation community who prepared a mutually agreed set of criteria that a laboratory should fulfil in order to demonstrate its technical competence. ISO/IEC 17025 is an international standard that laboratories can choose to apply (i.e. voluntary). Increasingly governments are specifying international standards, such as ISO/IEC 17025, as a tool to meet their regulatory and trade objectives across a wide range of fields (OECD, 2016[2]).
The principles of GLP on the other hand are written into law in many countries as a regulatory control mechanism, often as a legal requirement that non-clinical health and environmental safety studies intended for regulatory submission be conducted under GLP. The OECD GLP Principles have gained wide acceptance, also in non-OECD countries. In 2004 the OECD published an Advisory Document on The Application of the Principles of GLP to in vitro Studies, so as to provide guidance specifically of relevance to the application and interpretation of the OECD Principles of GLP to in vitro studies.
Even though there is overlap in many areas between GLP and ISO/IEC 17025 (e.g., training, management of equipment, etc.) each serve, as a result of their evolution and history, very different purposes. The OECD Principles of GLP are specifically designed to be applied to individual studies and to accommodate the complexity and variability of non-clinical health and environmental safety studies, while ISO/IEC 17025 was originally intended for testing according to established or specifically developed methodology. However, laboratory accreditation such as ISO/IEC 17025 can be applied to non-clinical testing, and is increasingly being used by governments to meet regulatory and trade objectives.
2.1. Quality assurance (QA) and quality control (QC)
The definition and roles of both QA and QC will depend for a large part on what quality management system is being followed; however most systems have a defined Quality Assurance Unit (QAU) that acts in an independent role. For the sake of simplification QA may be described as a proactive process for managing quality, while QC may be thought of as a reactive process for recognising quality problems and correcting them. The quality management system should be under ongoing review to ensure current best practices are implemented and to provide continuous improvements in the quality system, even if not formally required for GLP.
GLP has no explicit requirement to undertake QC activities, and QC is not defined or included in the GLP Principles or any of OECD GLP consensus or advisory documents. The OECD GLP Principles refer to a Quality Assurance Programme as a defined system, carried out by individual(s) designated and directly responsible to management who must not be involved in the conduct of the study, that is designed to assure that studies performed are in compliance with the principles of GLP.
Most GLP facilities do include QC activities within their quality system. QC activities are most effective when built into a procedure, e.g., calibration or checking of an instruments performance prior to use in order to identify and correct errors at the earliest opportunity prior to acquisition of study data.
2.2. Quality risk assessment
Risk management includes elements such as risk identification, assessment, mitigation, elimination and communication and may be applied to many laboratory processes, such as setting the calibration interval for specific equipment (e.g., some equipment may require less frequent calibration that others based on the probability of failure, the ease of detection and the severity of the consequences of the failure). Quality risk assessment may also be used for the assessment and evaluation of suppliers or to ensure that the test systems (Section 2.4), reagents and materials (Section 2.5) etc. are fit for purpose.
In a risk assessment the following basic questions should be addressed:
-
What might go wrong?
-
What is the nature of possible risks?
-
What is the probability of their occurrence and how easy is it to detect them?
-
What are the consequences (the severity)?
For an effective quality risk assessment the probability that the event will occur and the severity of the event must be addressed. Other parameters, such as assessing the ease of detection and the frequency of occurrence, may also be included to provide a more fine-tuned approach. The probability can be based on historical data and/or on the users' experience or it may also remain unknown. The severity of the event is addressed by listing the possible consequences of the event in the case it actually occurs. The ease of detection is a more difficult concept and is usually based on experience and thorough knowledge of the process while the frequency of occurrence may be based on historical data or also remains unknown.
The output of a risk assessment is either a quantitative estimate of risk (numeric probability) or a qualitative description of a range of risk (e.g., high/medium/low). The use of historical data is important when evaluating the probability that the event will occur. Therefore, in order to use updated information, the risk must be reassessed periodically.
Based on the outcome of the analysis and the criticality of the level of risk, specific risk controls, such as increased quality controls or QA inspections, should be put in place. The purpose of these quality risk controls is to reduce the risk to an acceptable level, and should be proportional to the significance of the risk.
2.3. Quality control charts
Control charts may be used in certain QC activities, and are a powerful and simple statistical tool used to routinely monitor the quality of any quantitative process to determine if the process is in a state of control. Control charts are typically used for time-series data, e.g. Figure 2.1 but they may also be used for monitoring discrete data sets such as batch to batch variability or operator performance.
Run or trend charts (Figure 2.1) are the most commonly used and easily understood charts. Individual results, e.g., for a reference item or for pipette checking, are plotted using a scatter plot graph versus the time order when the data were produced. The data points are linked by lines to help visualise the trend or changes in the trend. Trend charts are ideal for visually checking the historical performance of a process. Additional information may be placed on the trend charts to aid decision making, such as the true or expected value and specification limits or the average and control limits may be calculated based on historical data.
When using historical data, which gives a true representation of the performance of the process, it is important that the data used is representative of the current process and is based on an adequate sample size, i.e. the smaller the set of historical data used to calculate the average and limits the less representative these are of the overall process. The average (often used as a running average) is plotted with limits set at ± 2 Standard Deviations (SDs) for Upper and Lower Warning Limits (UWL and LWL) and ± 3 SD for Upper and Lower Control Limits (UCL and LCL). For normally distributed data based on a representative data set, the warning limits contain approximately 95% of the data points while 99.7% of the data are contained within the control limits, i.e. the 68–95–99.7 rule. The limits are irrespective of the process specifications or requirements.
Control or Shewhart charts use subgroups of the individual data to smooth out effects of individual data points and as such make it easier to identify trends or changes in the process. The most common types are the X-chart (average), the R-chart (range) and the s-chart (standard deviation). Subgrouping of the data, e.g., into sets of 5, allow the calculation of the standard deviation and/or range providing more information and finer control of the process. The subgroup average is usually set as the central line and the limits are calculated based on 3 SDs.
Control charts are mainly used to identify when a process is out of control or about to go out of control. When the process is out of control, data points fall outside the control limits, or when the process is about to go out of control, i.e. when a trend (e.g., two consecutive points outside the warning limits but still within the control limits) it usually means a new source of variation has been introduced into the process.
This variation may be due to systematic error which is usually seen as a change in the mean of the control values. Systematic error may be due to an abrupt change in the process (out of control), often caused by a sudden failure (e.g., apparatus), due to operator error (e.g., pipetting error) or some other once-off event. Systematic error may also be due to a gradual change which does not cause the process to go out of control, i.e. a trend change. Trend changes are usually harder to identify and indicate a gradual loss of reliability. The warning limits are used to detect the gradual change in the average and should also include some decision criteria on how to handle this change. The decision criteria will depend on the criticality of the process.
Random errors are those which are caused by random and unpredictable variation in a process and may be seen as acceptable (with the normal variation of process) or unacceptable errors, i.e. those that fall outside the control limits.
2.4. Quality control of test systems
It is important that certain key go/no-go points are established during the preparation and use of the test system for an in vitro method. Key quality attributes (e.g., genetic/phenotypic stability, identity and absence of contamination), based on the suppliers’ documentation and the facility's needs, and should be documented, with acceptance criteria, preferably in SOP(s). A QC plan to periodically confirm these attributes on a regular basis should be put in place. In practice it may not be always feasible to assess all "essential characteristics". The in vitro method should therefore include relevant and reliable positive and negative controls, including acceptance criteria, which will be used to establish an historic database of the test system essential functional characteristics. Lack of cell proper authentication, provenance, and characterisation could be grounds for a member country not accepting data that are not adequately documented.
Proprietary in vitro methods and the related in vitro systems may be relatively expensive; therefore their availability for QC testing may be limited by practical considerations, such as cost. In light of these considerations, the user may sometimes be dependent on the supplier to provide as complete as possible documentation regarding the test system, including cell or tissue characterisation and functional performance. The supplier should be expected to provide adequate documentation of quality control testing of each batch manufactured.
The suppliers' documentation should detail appropriate test system integrity checks of the Original Source (Table 2.1), ideally with evidence of test results provided by the supplier or a qualified service provider (Section 5.2.). These checks should also be performed on the cells arriving in the laboratory as soon as samples can be obtained (Early Stocks). Ideally Cell Banks (both master and working) should be established ( (Coecke et al., 2005[3]) but testing may be focused on the master stock with more routine checks applied to working cell banks e.g., mycoplasma and viability (Table 2.1). In addition, the user should carry out quality control checks in the test facility on a regular basis (Routine Culture testing) appropriate to the test system so that the in vitro method performs as expected after transport and handling of the test system.
Where primary cell cultures and tissues are used, variation in properties between individual donors must be considered, and each new batch should be qualified or controlled for key functionality (Meza-Zepeda et al., 2008[4]); Special care should be taken to note any unusual observations in case of contamination or viral cytopathic effects or transformation, and all primary cell cultures should ideally be cryopreserved and screened for mycoplasma. Human and animal tissues and primary cells used for testing will also need to be appropriately documented. As part of QC for tissues, their differentiated state should also be documented, which may require a range of assays including for instance morphology, histochemistry, cell markers, specific tissue function and cell-cell/matrix interactions (Stacey and Hartung, 2006[5]). For primary cells prepared from tissues stored as banks of cryopreserved vials of cells, similar QC approaches can be used as adopted for banks of continuous cell lines (Section 5.5.).
Moreover, records recommended by Good Cell Culture Practice (GCCP) or other relevant guidance documents (e.g., ISO standards, GLP) should be kept. Guidance on cell and tissue culture work is available for either general (Coecke et al., 2005[3]) or specific applications (Andrews et al., 2015[6]; Geraghty et al., 2014[7]; ISCBI, 2009[8]; Pamies, 2016[9])).
2.5. Quality control of consumables and reagents
All consumables and reagents should be evaluated to be fit for the intended purpose(s). Consumables such as flasks, cryovials, culture dishes, culture slides, tubes, cell scrapers, etc. in general will not require any in-house QC, however it is good practice to maintain any relevant documentation provided by the supplier, such as proof of sterility, date of arrival, expiry dates and batch numbers, as the suitability and acceptability of materials may be questioned by the GLP Monitoring Authorities.
Test facilities can perform quality control checks of consumables, but the process how to do this is not always evident. Some test facilities have established procedures whereby a percentage of consumables from each batch/lot number are evaluated prior to use in in vitro work (e.g., for sterility testing). While this approach will not prevent contamination, it can provide data which can be useful for future evaluation of contamination. Ideally, sterile consumables with appropriate certificates should be used where possible. Alternatively, some consumables can be treated with ultraviolet (UV) light, gamma irradiation and/or autoclaved. Viral infection via such biological material as Foetal Calf Serum (FCS) (Section 4.3.1) can be avoided e.g., by gamma ray radiation of FCS (House, House and Yedloutschnig, 1990[10]; Nuttall, Luther and Stott, 1977[11]). These preventive measures may be useful in limiting contamination. Other consumables, such as centrifugal filter units and filtered pipette tips, cannot be pre-treated. In the case where no commercial sterile centrifugal filter units and/or filtered pipette tips are available, establishing a method for detecting contamination from these items is very important.
Certain materials which are critical to the performance of a method may be subject to significant variation, such as growth promoting reagents, hormones and conventional serum products (functional tests including acceptance criteria need to be defined). These critical reagents should be reliably available and sourced from a reputable supplier (where possible alternative sources should be identified), and should either be accompanied with the supplier's Certificate of Analysis (CoA), or appropriate quality controls should be applied in-house (Good Cell Culture Practice (GCCP) and Good Cell Culture Practice for stem cells and stem cell derived models). These controls may include growth or functional characterisation and should be performed by qualified personnel according to documented procedures or formal SOPs.
For some critical reagents it may be necessary to test for batch to batch variability so as to reduce the introduction of unknown variables, which may interfere with assay or overall in vitro method performance. For this purpose a batch is tested first and when approved, a large quantity of the batch can be acquired to reduce variability during the performance of a certain number of assays. Successive batches may be tested in-house and the new batches compared against historical data (e.g., growth rates).
For established reagents, the in vitro method uses the acceptance criteria of negative controls to identify eventual issues related to a new batch of reagents. Similar reagents obtained from different suppliers may each have specific and not necessarily the same acceptance criteria. Acceptance criteria should be established for reagents depending on the degree of risk they represent to the final results. This risk can be assessed by:
-
Considering the potential impact of the perceived risk to prioritise certain reagents.
-
Formally evaluating the Quality Management System (QMS) of the supplier and establishing suitable Agreements (e.g., Service Level Agreements (SLA)) with the provider ensuring quality, availability and shipment of the reagent. Acceptance of individual batches of reagents can be addressed by review of key elements of the certificate of analysis, compliance with specific conditions of the agreement provided by the manufacturer/supplier or a combination of these and supplementary evaluation which may include pre-use testing to assure that individual batches are fit for purpose.
-
Assessing consistency of batch/lot qualification tests on critical reagents.
2.6. Staff training and development
Training is an integral part of all quality assurance systems, and must be formally planned and documented. For example GLP requires the maintenance of records of qualifications, training, experience and job descriptions of personnel (OECD, 1998[12]). Training should be formal, approved (certified), documented to a standard format and typically described in a SOP (WHO, 2009[13]). Training should be proactive, enabling staff to acquire the skills and knowledge that, with experience, makes them competent in the cell and tissue culture aspects of their work or enables them to elicit an appropriate reactive response where necessary. New objectives and new activities or procedures (e.g., SOPs) will always involve some training, and therefore requires new certification of the involved personnel. GLP attaches considerable importance to the qualifications of staff, and to both internal and external training given to personnel, in order to maintain levels of excellence and ensure the procedures are performed consistently by all personnel.
A list of core training for in vitro cell culture laboratory staff is detailed in the GCCP (Coecke et al., 2005[3]). Special aspects of training are also referred to in other sections of this document where relevant.
Documented training plans are useful to define procedures in which staff should be trained before they are considered competent. Regular review by line managers of staff performance is a useful tool for considering ongoing training needs. These may include regulatory requirements (e.g., GLP training), specific in vitro methods and their associated proficiency chemicals, use and storage of documentation, as well as general training in best practice such as indicated in GCCP (Coecke et al., 2005[3]). When new staff is recruited to work in the laboratory, it is important to guide the staff and review and document any training requirements before assignment to carry out any tasks. It may be helpful to demonstrate competence by documenting individual elements of training followed, including competence to perform the procedure(s) independently.
It is good practice to record all training in individual training files, including training records and periodic competency reviews. Supplementary training and education should also be documented to demonstrate maintenance of ongoing professional development to provide assurance that current best practices are maintained.
2.7. Types of documentation
The importance of documentation cannot be over stressed as it is the only way to demonstrate the work performed, i.e. if it is not documented it did not happen. It should enable reconstruction of a study/experiment and is also essential for the interpretation of the results.
Documentation in a quality system typically involves documents and records at several levels. The main document is a high-level, accurate description of the types of work performed by the organisation or group, key policies and standards adopted for delivering the work and the structure of the quality system. In some systems, this may be called a "quality manual". Another level may include overviews of procedures referring to the various specific testing methods involved at the next level. Finally, supporting the SOPs, there will be formal record sheets for test and control data and templates for reporting results. An overview of descriptive and prescriptive documents is provided in the World Health Organisation (WHO) handbook on quality practices in Biomedical Research (WHO, 2013[14]).
The WHO divides documentation into two broad classes:
-
Prescriptive documents that give instructions as to what is to happen during the course of a study, such as SOPs and Study Plans.
-
Descriptive records that describe what actually happened during the course of the study, such as records of raw and derived data, study reports.
Many quality systems require document management to assure that all documents are developed and approved in a formal process, that versions are accurately dated, authored and approved with specific version numbers to avoid inadvertent use of obsolete documents.
Each institution should implement rules regarding the recording and retention of data. Record keeping, whether by hand or making entries to electronic systems, should meet certain fundamental elements of data integrity (Section 10.1).
2.8. Quality considerations regarding electronic data integrity
The integrity of electronic data, and how to assess it, should be described in the quality system. Some of the common issues that repeatedly come up in US Food and Drug Administration (FDA) warning letters are:
-
Common passwords. Sharing of passwords, or use of common passwords, does not allow the true identification of the operator, i.e. it is not attributable.
-
User privileges. The software application is not adequately configured so as to define or segregate user levels and users may have access to inappropriate privileges such as modification of methods and integration or even deletion of data.
-
Computer system control. Access to the operating system is not adequately implemented and users may modify system configurations (e.g., system clock) or allow unauthorised access to modify or delete electronic files; the file, therefore, may not be original, accurate, or complete.
-
Processing methods. Integration parameters are not controlled and there is no procedure to define integration leading to concerns over re-integration of chromatograms at a later time.
-
Audit trails. In this case, the laboratory has turned off the audit-trail functionality within the system. It is, therefore, not clear who has modified a file or why.
See Section 10.1 for a more in-depth discussion on data integrity.
References
[6] Andrews, P. et al. (2015), “Points to consider in the development of seed stocks of pluripotent stem cells for clinical applications: International Stem Cell Banking Initiative (ISCBI)”, Regenerative Medicine, Vol. 10/2s, pp. 1-44, https://doi.org/10.2217/rme.14.93.
[3] Coecke, S. et al. (2005), “Guidance on good cell culture practice: A Report of the Second ECVAM Task Force on good cell culture practice”, ATLA Alternatives to Laboratory Animals, Vol. 33/3, pp. 261-287.
[7] Geraghty, R. et al. (2014), “Guidelines for the use of cell lines in biomedical research”, British Journal of Cancer, Vol. 111/6, pp. 1021-1046, https://doi.org/10.1038/bjc.2014.166.
[10] House, C., J. House and R. Yedloutschnig (1990), “Inactivation of viral agents in bovine serum by gamma irradiation”, Canadian Journal of Microbiology, Vol. 36/10, pp. 737-740, https://doi.org/10.1139/m90-126.
[8] ISCBI (2009), “Consensus Guidance for Banking and Supply of Human Embryonic Stem Cell Lines for Research Purposes”, Stem Cell Reviews and Reports, Vol. 5/4, pp. 301-314, https://doi.org/10.1007/s12015-009-9085-x.
[4] Meza-Zepeda, L. et al. (2008), “High-resolution analysis of genetic stability of human adipose tissue stem cells cultured to senescence”, Journal of Cellular and Molecular Medicine, Vol. 12/2, pp. 553-563, https://doi.org/10.1111/j.1582-4934.2007.00146.x.
[11] Nuttall, P., P. Luther and E. Stott (1977), “Viral contamination of bovine foetal serum and cell cultures”, Nature, Vol. 266/5605, pp. 835-837, https://doi.org/10.1038/266835a0.
[1] OECD (2016), Application of Good Laboratory Practice Principles to Computerised Systems, OECD Series on Principles on Good Laboratory Practice and Compliance Monitoring, No. 17, OECD Publishing Paris.
[2] OECD (2016), OECD Position Paper Regarding the Relationship between the OECD Principles of GLP and ISO/IEC 17025, OECD Series on Principles of GLP and Compliance Monitoring, No. 18, OECD Publishing, Paris.
[12] OECD (1998), OECD Principles on Good Laboratory Practice, OECD Series on Principles of Good Laboratory Practice and Compliance Monitoring, No. 1, OECD Publishing, Paris, https://doi.org/10.1787/9789264078536-en.
[9] Pamies, D. (2016), “Good Cell Culture Practice for stem cells and stem-cell-derived models”, ALTEX, https://doi.org/10.14573/altex.1607121.
[5] Stacey, G. and T. Hartung (2006), “Availability, Standardization and Safety of Human Cells and Tissues for Drug Screening and Testing”, in Drug Testing in vitro, Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany, https://doi.org/10.1002/9783527609611.ch9.
[14] WHO (2013), WHO Handbook: Quality practices in basic biomedical research, World Health Organization.
[13] WHO (2009), Handbook: Good Laboratory Practice (Second Edition) - Quality practices for regulated non-clinical research and development, World Health Organization.