Testing laboratories in the analytical world use proficiency tests (PTs) to comply with their accreditation requirements and evaluate analysts’ performance. PTs are an integral part of a quality management system (QMS) under quality assurance and control (QA/AC). Understanding the core components of the QMS is an important part of passing any PT test. Unacceptable PT results may have little to do with the result itself but reflect the use and application of statistics, standards, and methods. As more cannabis and hemp laboratories join implement quality management systems and become accredited under standards organizations, they also become subject to the need for proficiency testing. It can be argued that many PTs directed toward the general laboratory community are not a good fit for purpose in the cannabis community. In this article we discuss the purpose and best practices for PTs and how they can be better adopted to the cannabis market.
PTs are characterized materials created to represent the types of samples, matrices, and analyte targets being evaluated in laboratories. PTs are assigned reference values which are not disclosed to participants. PT samples are treated like blind samples; in which the nature and quantities are unknown. PT tests may provide basic information regarding the target identity, quantitative range and other information which directs the analyst in how to perform sample preparation or analyses. PT samples can come in many different forms from solid or liquid matrices to extracted oils, or solid dosage formulations like capsules and tablets. Analysts are expected to prepare and treat PTs in the same way similar types of samples would be routinely processed.
PT participants confidentially share their results with the PT provider for final evaluation and grading. In this way, PTs serve as an indicator for the competency of a laboratories staff and analytical performance. Results reported to the PT provider are compared to the established reference values for that PT. A reference laboratory obtains established reference values or averages the values reported by the PT participants (consensus value).
Results can be reported individually, meaning each analyst reports a result, or by calculating an average from all analysts performing the PTs. The individual result entry method is often preferred because it serves to document passing or not passing results at an individual level. This is most helpful for assessing and attributing competency of each lab analyst performing the PT. Participants who achieve passing PT results are ensuring the validity and reliability of their lab’s test results.
PTs should be an integral part of all testing labs’ quality system. When evaluating PT programs, laboratories must consider some critical points such as: the qualifications of the PT administrator, the quality and qualifications of the PT provider, and the accessibility of data. Within QMS systems based on ISO guidelines, laboratories are required to use certified providers for their analytical standards, methods, and PTs. ISO 17025 laboratories must use PT providers accredited to ISO 17043 and CRM providers to ISO 17034.
Methods used for PTs should have previously been validated by the laboratories or standards organizations which have issued those methodologies (i.e., AOAC, ASTM). In cases that standardized methods are not available; the laboratory should validate or verify their methods that will be used for the PT before the study begins. PTs are not a means of methods validation. A suitable QC-known, certified reference material or primary reference standard is employed for validating an analytical method. If a certified reference material is not available; a suitably validated and characterized material (usually by consensus or round-robin testing) canbe validated for use as standard in lieu of a CRM.
Acceptable PT results can serve as method verification when PTs are performed by the lab’s validated method. After the in-house methods have been validated, the methods can be verified by successfully passing a PT from an accredited third-party provider. In this example, PT is integral to the QMS of the participant lab.
PTs serves to measure the ongoing proficiency of independent laboratories through interlaboratory comparison of test results for the same sample. Some regulatory bodies stipulate the frequency of PT participation. Testing laboratories participate in accordance with their accreditation audit schedule. Some accreditation audits only occur once every other year. Thus, the labs would participate in PTs once every two years.
When determining the schedule and budget for PT participation, it is best to consider the accreditation goals and regulatory requirements for the laboratory when determining the schedule and budget for PT participation. It is best for each analyst who is reporting data to perform PT at least annually to monitor performance during a twelve-month period. Longer stretches without PT participation could negatively impact timely identification of non-conformance and delay implementation of effective corrective action. Laboratories that do not achieve acceptable results on their PT should examine analyst training and method procedures to check for deficiencies.
PTs calculations and results
Once participants analyze the PT, the participants report their results to the PT provider for review and evaluation. PT providers use customized software programs to evaluate participants’ results using programmed algorithms and statistics that meet the accrediting bodies guidelines. ISO guidelines (ISO 13528) have two basic methods for statistical evaluation of proficiency tests: the En-value and the z-score. An En-value is most commonly used in interlaboratory comparisons where the laboratories report their uncertainty calculations. (Equation 1).
Equation 1. En-Value Equation:Xi is lab reported value versus the Xref value reported by PT provider divided by the square root of the combination of uncertainties from the lab value and the reference value expressed using k=2 with a 95% coverage factor. Results with En-value between 1 and -1 are acceptable and agree with the reference values. Results with En-values greater than one or less than -1 are not acceptable.
The most practical of the statistical functions for chemical and biological analyses is the z-score. (Equation 2).
Equation 2. Z-score equation where µ is the mean subtracted by the Xi lab reported value; divided by the standard deviation, s.
The z-score can be used within interlaboratory comparisons without an uncertainty calculation; The z-score assumes all reported samples are from the same population, batch, or sample set and have the same uncertainty.
The results are calculated after the outliers are removed or are obtained from robust statistics. Z-scores less than two are considered acceptable. Scores between two and three are considered suspect while scores greater than three are unacceptable.
Once participant results are compared to reference values and assigned acceptable or unacceptable scores, those scores are provided to the laboratories. Results of a particular PT scheme may be deemed suspect if a clear bias or trend is apparent. Even if the results themselves are passing, the PT results can be considered unsatisfactory if there is a clear and measured bias. All failures or unsatisfactory results should be directed towards root cause analysis and corrective actions indicated by the lab’s QMS.
In the event that analysts or laboratories fails PTs, there should be a procedure for review along with a plan of action to correct the problems found. First, a root cause analysis is conducted to identify and document the problems. If a root cause analysis finds deficiencies within a system, then corrective action is devised to remedy issues. A retesting of a corrected system must be conducted to determine if the correction was successful.
Points of review should include laboratory processes and all internal quality control data. Points to reexamine during a system and procedure review for a PT failure include:
Figure 1. Unit conversions
Figure 2. Dilution tips
Best practices for PTs Preparation
The proper handing and processing of PT samples can be critical in passing PTs. There are other additional areas which play key roles in passing these tests including instrument calibration, maintenance, and updating procedures and methodologies. Here is a list of helpful points to assist laboratories and analysts in passing their PTs. (Figure 2)
Figure 3. Best practices for PTs
PTs are a necessary and helpful tool in the QMS arsenal of a laboratory. But they do have some limitations and misuses. In PT schemes where interlaboratory tests are used to establish values, there is often a retrospective aspect to the tests. There are often routine samples or analyses not reflected by the PT schemes either because the PT is offered in a different sample format or matrix than what the lab routinely tests.
For example, if a laboratory tests pesticides in cannabis extracts in ethanol but the PT is pesticides in plant material, the PT doesn’t match the preparation routine of that laboratory. In this case, the laboratory must document all the steps taken to prepare the PT sample and the ways in which it could differ from the preparation of actual samples. The best practice is to try and match the PT to the types of routine samples run by the laboratory. If a commercial PT program does not offer an obviousmatch in an established PT program; the laboratory can work with the PT provider to create the matrix-matched PT suitable for that laboratory’s needs.
An unacceptable PT only indicates a problem exists. It does not identify the root cause of the problem. Conversely, an acceptable PT result does not indicate competence in all areas of QMS. PTs are not substitutes for using quality controls and reference standards. They all work collectively within the QMS.
Conclusions - Tying it all together
PTs serve to measure the ongoing proficiency of independent laboratories through interlaboratory comparison of test results for the same characterized material. PTs participation is often stipulated by the regulatory authority overseeing laboratories operations or an accrediting body to which laboratories comply, or often both authorities concurrently. It is best to consider the laboratories accreditation goals and regulatory requirements when determining the schedule and budget for PTs participation. It is best to perform PTs at least annually to monitor lab performance during a twelve-month period. Longer stretches without PTs participation could negatively impact timely identification of non-conformance and delay implementation of effective corrective action. PTs are useful as tools to verify continuous improvement, resolve or monitor effective corrective actions, and demonstrate training and competency for lab analysts. PTs are a multi-purpose external quality assessment tool and should be an integral part of the lab’s QMS.