I Can’t Afford Good Science: The Battle Between Economics and Method Development in the Emerging Cannabis Market

How is “good” science achieved when governments take no active role in establishing limits, validating methods, or issuing guidelines? How can standards organizations fill the void by not only trying to establish guidelines, but also to create policy in the face of the need for scientific accountability and the reality of a highly economically driven and price-conscious analytical environment? How does one demand “good” science and high accuracy when there is no legal reason for accountability, little consensus of official guidance or methods, and a driving force of economic competition between laboratories that rewards fast, cheap results that favor the manufacturers and distributors? The balancing act between “good” science and scientific accountability is widespread in the cannabis community, from the growers to the testing laboratories and beyond.

One winter afternoon, a group of professionals and scientists came together to discuss the development of test methods for cannabis. These individuals represented a variety of organizations and viewpoints, from testing laboratories and standards providers to instrument manufacturers. Several government entities, while not officially sanctioned to work with cannabis, participated as interested voices in the process. Since then, this group, which is part of an internationally recognized standards organization, convenes on a regular basis to discuss and debate the testing of cannabis in ways that will eventually lead to standardized test methods for analytes of interest. Time and again the discussions progress down a similar path of proposal, debate, and compromise, which is an integral part of the consensus and peer-reviewed method development process.

Standardized analytical methods are constructed from templates or guides, specifying key areas that are integral to the method such as scope, applicability or use, preparation and analytical procedures, analytical needs (laboratory equipment, instrumentation, standards, and so forth), performance requirements, and statistical analysis (precision, bias, and validation). In addition, current or proposed applicable regulations and requirements are examined and incorporated into the methodology. There is an understanding that the methods will be built on the rules of “good” science; they will follow the scientific method, be proven reproducible, and represent the sample or the testing appropriately. The method should function in real-world use without any agenda other than producing accurate results, which achieve satisfactory precision and bias requirements (1,2).

These methods are commonly based on either governmental or industry-driven specifications and regulations. When a new analytical sample type or product enters the testing world, it is routed to a well-established market or industry (such as environmental, consumer safety, pharmaceutical, or food safety), which then takes over its governance. Once routed, the method creation procedure is followed and appropriate governmental and accreditation agencies put their marks on the process before it is incorporated into a laboratory workflow. As testing progresses over time, methods are updated and requirements are refined but, still, the consistency of the science and the data is expected to maintain its demonstrated level of quality. The methods are supposed to represent the best available test procedures for the intended use and reflect technological advances and requirements (2).

The steps of the standard process are well-proven. For many new industries the road to regulation and testing, despite some bumps or minor detours, is a fairly straightforward path. But, for an industry such as cannabis, these traditional guidelines pose difficulties and roadblocks because cannabis, by its very nature and position in culture, is not easily adopted by regulatory and governmental parent organizations that drive testing regulations.

Early Cannabis Testing

Less than a decade ago, the cannabis market moved from an underground black market business into the light of being a quasi-legal industry. Dave Egerton, the Vice President of Technical Operations for CW Analytical in California, saw the early testing for potency as a marketing tool, used by growers purely for economic reasons. The early years in the market were filled with small laboratories started by those people with an interest in cannabis as a new industry, but who also knew very little about laboratory testing. They were focused on taking in samples and putting out reports. It was difficult, in the beginning, to find veteran scientists willing to jump into an unknown and uncertain testing industry. “Experienced chemists are more risk averse, and it can be challenging to bring them into the market,” said Carl Carnagey, CEO of Juniper Analytics, a cannabis testing laboratory in Oregon.

The initial cannabis testing laboratories did not have methods to follow. They did not have to adhere to regulations or prove their competency. In many cases, procedures were created from literature searches, analysts’ personal experiences, or the type of instrumentation purchased (oftentimes secondhand). In the United States, the federal regulatory agencies had no official guidance for the cannabis industry, which created a “wild west” mentality in the community. Laboratories were competing for the business of growers and dispensaries, whose primary focus at the start was not necessarily safety, but more often, low-cost testing and a “scientific” stamp of approval. Carnagey explained that those times were challenging for scientists. “Even labs that wanted to do things right were challenged because they did not know what ‘right’ was,” he said. “It became a quest to find what is right.”

The Future of Cannabis Testing

As the legalization of cannabis became part of the U.S. national and world debate, voices of concern grew louder regarding the safety and quality of cannabis products. The scientific and regulatory communities were recruited to help legitimize the safety and quality of these new products by the scientific method, without the benefit of procedures established in other legally recognized industries. Nearly a decade since the mainstream introduction of cannabis, the regulatory and standardization organizations are now stepping up to fill the void and standardize the testing of cannabis.

The goal of these cannabis method committees is not unlike their more established counterparts: to create methods that meet the needs and demands of the industry and analytical community. The difference between this industry and the ones that have come before it is that the method development process that is typical for other scientific fields creates challenges at every level, driven by commodity value, legal limitations, and financial burdens, that are not usually seen in other laboratory industries. 

Take, for example, the first steps of a method development process-that is, defining the scope and applicability of a method. The definition of a method scope for a target such as potency is fairly straightforward. However, for other targets-pesticides, for example-it becomes a more difficult task. In the absence of regulations, the early laboratories did not know which pesticides to test for or what levels to consider as limits. States that legalized cannabis products began to separately issue some guidance for pesticide targets. Now, several years into cannabis testing, several states in the United States, along with Canada, have issued pesticide lists.

As the standardization organizations look at creating methods for cannabis, it becomes a question of which lists and limits most accurately reflect what is needed for the safety and testing of cannabis products; the lists generated are all very different from one another. One state list has less than a dozen pesticides, whereas the list from Canada is close to 100 target pesticides. Another state has a limit for a particular pesticide at 1 ppm, while a second state has a limit of 20 ppb.

Returning to that winter meeting of the cannabis committee, the debate continued without resolution. At the heart of the discussion: What pesticides should this international pesticide method for cannabis contain? Participants from the accreditation groups and the contributing members of governmental agencies worked from the perspective that pesticide testing in other industries, such as food, can screen for hundreds of pesticides. These members believed that the required testing list should encompass all the pesticides on all the available cannabis lists to date.

The opposing groups, mostly composed of members directly involved in cannabis testing or the cannabis industry, believed that requiring laboratories to test for a large list of pesticides would hurt the implementation of the method into widespread use. Their point of view was that most cannabis testing laboratories are only interested in meeting their own state’s requirements, or the requests of their customers. One member of the committee who works in a testing laboratory explained that his laboratory would not be allowed to develop methods beyond his own state’s requirements because of cost in both time and materials. This opinion was not unique to the cannabis laboratory members of the committee. When asked, both Carnagey and Egerton agreed that the focus of cannabis laboratories is meeting the regulations and rarely trying to exceed them.

Another point of contention within the standard methods process happens when it is time to standardize the sample collection and preparation processes. A food testing laboratory very rarely has to fight for adequate amounts of sample to conduct all the tests required. So, when a method is being written for a new food product, generally, previous methods can be adopted. Many of these methods dictate large sample sizes and multiple sampling and preparation schemes to ensure homogeneity and allow for multiple replicates. But when the product is a high-value commodity, it can be a fight with the sample supplier to obtain the necessary amount of material to ensure a homogenous sample and produce accurate results. Egerton explained that there has always been a hesitation of sacrificing product for testing. In the beginning, laboratories had to make do with what they were given to try to get an accurate result. As regulations change and more governments get involved, it becomes easier to justify the amount of sample needed. Even with increased input from regulatory bodies, it falls to the laboratories to educate the nascent industry on concepts such as variability, homogeneity, sampling tables, and representative samples.

The newly formed cannabis methods committees must weigh the economics of sample collection and sample replicates into the value of the method being produced. There is an economic cost to what most scientists would consider a sufficient analytical sampling and testing batch. There is a cost to multiple sample preparations and multiple sample testing replicates, which some cannabis suppliers and manufacturers are unwilling to incur, creating resistance to adopting methods requiring larger sample sizes.

At the point at which committees finally get to the actual analytical methods, there are then debates around performance requirements, instrumental and laboratory requirements, and reference materials. All of these requirements have challenges to the industry and unexpected associated economic impacts because of the gray areas in which the cannabis industry finds itself.

Carnagey believes that a large part of the cannabis testing business is dealing with the “gray” areas, such as finding leasable laboratory space that is willing to risk its mortgage or financing by renting or selling to a cannabis business. Cities will sometimes block attempts of cannabis businesses from renting or purchasing space, and governments charge these businesses and laboratories extra fees and require additional permits. Carnagey thinks that some of these extra requirements and fees are because of the misconception that the entire cannabis industry is “making tons of money.” In his experience, cannabis testing laboratories are operating on the same margins as other analytical laboratories, yet the cannabis laboratories face more hassles and fees. These financial burdens and legal limitations become part of the method development discussion when economics factor into analysis. The additional financial and legal challenges drain capital that would be better served by reinvesting in the equipment and technology required to meet the new standards.

Many modern analytical methods for targets such as pesticides require instruments that can quantitate smaller and smaller amounts of analytes and provide results with exacting performance requirements. Regarding equipment needs for cannabis, Carnagey said that pesticides in the cannabis laboratory should be done on a triple-quadrupole liquid chromatography–mass spectrometry (LC–MS/MS) system. However, many cannabis laboratories are struggling with the need for LC and gas chromatography (GC) triple-quadrupole instruments. It is not only the cost of these instruments that is an issue; the purchase of capital equipment and instruments can be a challenge in a gray-area industry, battling federal banking laws and facing boycotts by some scientific suppliers and instrument manufacturers.

Standards organizations do consider the cost of the method development process, but in many cases, especially in terms of safety, the cost incurred by a method is not a primary consideration (2). Many analytical method developers consider instruments such as LC–MS, GC–MS, inductively coupled plasma–mass spectrometry (ICP-MS), and in the past few years, triple-quadrupole versions of these, to be part of a laboratory’s standard resources-not an added burden for use in methods. The ideal balance is supposed to be “technically relevant results” proportional to the cost to determine those results (2).

This point brings up another hurdle in the methods process for cannabis testing: reference materials. There are a multitude of problems around securing and using relevant and appropriate reference materials in the cannabis industry. The first issue is the limited number of commercially available standards, specifically for cannabis potency. Each of the cannabinoids of interest is a U.S. federally scheduled compound, strictly regulated by the Drug Enforcement Agency (DEA). Reference material manufacturers must secure licenses to possess the materials and to manufacture the standards. There are strict limits imposed on the quantities of materials that can be purchased, and all aspects of the purchasing, manufacturing, and sale of the final products must be monitored and documented. The reference material manufacturers, like some of the cannabis laboratories, can be denied permits by local and state governments, or pay additional fees. The cost to manufacture a cannabis standard is much higher than that for most other type of standards. After licenses are obtained, there is the additional expense and difficulty of finding and purchasing quality starting materials for scientific use. All of the legal wrangling makes it difficult and expensive to produce cannabis reference standards, which creates higher costs passed down to the testing laboratories.

The second problem is that the transportation and sale of cannabinoid standards is restricted to concentrations of less than 1000 ppm, which does not necessarily match the analytical needs of potency testing for materials that have percent levels of cannabinoids. Additional dilutions and sample preparations must be employed to use standards at concentration levels that are hundreds, if not thousands of times more dilute than the actual samples.

Some analytical methods require a solid reference standard or a matrix-matched material to validate or confirm the analysis. In the case of cannabis, the transportation of a truly representative cannabis material containing high levels of tetrahydrocannabinol (THC) is often prohibited by federal law. This prohibition makes it difficult to impossible for standards manufacturers to sell a product into laboratories not located within their home state, or to the rest of the world. Substitute matrices, such as hemp, are available but are not identical to the ideal cannabis reference material.

A final point in any methods process revolves around the validation of the method and the definition of a method’s precision and bias. Precision and bias validation studies, in many cases, involve interlaboratory studies where a reference or study material is distributed to a variety of laboratories around the country-or, indeed, around the world-for analysis by the proposed method. In the case of cannabis, the studies would have to be conducted using a substitute material, or be confined to one state, because of the transportation prohibition.

Conclusion

For the time being, economics plays a significant role in the development of cannabis methods; at least until larger federal governmental agencies step in and take control of the regulation and reduce the prohibitions. Until such a time, the testing debates will continue with economics as part of the discussion. When the methods are finally agreed upon in some form it can be hoped that they will meet the needs of the cannabis industry and reflect the practices of good science-and are not just a reflection of the price this new industry is willing to pay.

References:

1. Standard Format and Guidance for AOAC Standard Method Performance Requirement (SMPR) Documents (Version 12.1; 31-Jan-11).
2. ASTM International, Form and Style for ASTM Standards, October 2017.

Patricia Atkins is a Senior Applications Scientist with SPEX CertiPrep in Metuchen, New Jersey and a member of both the AOAC and ASTM committees for cannabis. Direct correspondence to: patkins@spex.com

How to Cite This Article

P. Atkins, Cannabis Science and Technology1(2), 10-16 (2018).