Quality Variations

Defining and Measuring Quality in Coffee From Production to Preparation

This article was original published by ROAST Magazine in the January/February 2020 issue and has been converted into an audio article podcast on SoundCloud.

Until the late 1990s and early 2000s, there was a generally accepted definition of green coffee quality designated by each producing country. Each place determined its own quality designations and terminology.

For example, in Colombia there is Usual Good Quality (UGQ), European Preparation (EP) and Supremo, and in Brazil grading either follows the Santos or New York scale. In Kenya there is AA, AB, B, etc. Ethiopia and Indonesia use Grade 1, Grade 2, Grade 3, and so on, although these countries have different definitions for each grade. Quality variations were once a seller’s market, meaning sellers defined what was available and how coffee quality would be measured. Buyers selected green coffee from what was offered.

Slowly, trading houses—often as a result of demand from roasters—began to establish slightly customized interpretations of these standard grades as a point of differentiation for their supply chain. Thus, a Strictly Hard Bean quality from Central America would have different bean size or defect counts from different exporters; however, they could each still be identified as Grade 1 or standard Strictly Hard Bean quality. Some considered this market evolution as market confusion, while others simply embraced the differences.

As the coffee industry expanded in the early 2000s, trade associations created industry standards and educational programs for green coffee quality evaluation and grade determination. At this point, the market completed the shift from a seller’s to a buyer’s market, changing the paradigm of green coffee sourcing. Sellers now customize grade and cup profiles to meet the demands of buyers, product developers and roasters; buyers are no longer limited to the quality that was offered and available.

“Wholesalers are starting to pick roasters who align themselves with their core values,” says Jeanine Niyonzima-Aroian, founder of JNP Coffee and Burundi Friends International. “Green coffee producers and traders are starting to work with roasters that specifically seek sustainability practices to meet the growing demand of consumers.”

Maria Esther Lopez-Thome, co-creator of Coffea School in Miami, Florida, and head coach at Café Registrado in Buenos Aires, Argentina, says, “Programs such as those developed by the Coffee Quality Institute and ACE/Cup of Excellence provided new perspectives of ‘quality’ in producing countries, which contributed, by itself, to the fact that as many producers as buyers began a process of higher quality demands.”

These variations expanded the understanding of quality. Some would say this segmented the industry into an infinite number of quality discrepancies, while others applaud the increased diversity of coffee quality by bolstering the variations to meet each roaster’s quality personality.

Eliana Cossio, who holds a doctorate in food science and coffee, and works at Wisconsin’s Trilliant Food and Nutrition, adds, “The definition of quality is not that simple because it can be seen from several different perspectives. Ernesto Illy taught us that ‘quality’ is a word that everyone uses, but no one defines. He said it is a frame of mind and an exchange of culture and technical know-how within all the stakeholders of the entire coffee production flow.”

A natural progression for quality variations occurs throughout the supply chain, beginning with coffee buyers, then roasters, and ending with baristas. At each control point, new technology, innovation, and creativity increased quality based on what was possible and how each company could differentiate its coffee for an individual quality personality. Each coffee has its own identity and expectation for physical attributes and sensory characters.

“Thanks to direct trade, relationships between buyers and producers have also changed,” adds Lopez-Thome. “Roasters are traveling to origin countries and are creating sustainable relationships with farmers, and consequently the quality they seek has changed also, which is now more focused on how the green coffee will behave once it is affected by heat transfer.”

Measuring, identifying and categorizing coffee quality is a fundamental aspect of our industry. Quality concerns are relevant to both sourcing and sales, while categorizing and justifying quality are pertinent to both sales and finance. Company reputations are built on coffee quality by self-identification and consumer reviews.

EXPLORING QUALITY VARIATIONS

Quality can be quantitative or qualitative. It can be measured based on cherry, parchment, export cargo, arrival inventory, roasted products, prepared beverages and ready-to-drink items. However, coffee quality is never solely an absolute value.

Starting with the local and regional roasting companies in the late 1800s up to the boutique roasters of today, and from the industrial functionality of cowboy coffee boiled over a campfire to the refined elegance of slow-drip siphons, quality has always been part of the coffee conversation. What tastes great? The ideal of quality remains the same: a never-ending quest for flavor perfection. The quest for coffee perfection is quite different at the various control points in coffee’s supply chain.

Variations in green coffee quality result from differences in variety or cultivar, terroir, processing and milling. Roasted coffee variations in quality arise from blending, profile roasting and modulating flavor development. Beverage quality variations derive from the personality of the brand or are created by the barista with the brewing method, coffee-to-water ratio, water temperature, and more. Each process, controlled procedure, and designed result exponentially increases today’s concept of quality, which is much different than the quality variations in the past.

From the consumer perspective, the expectation is for the coffee to look, smell and taste the same with each purchase. This is achieved using great attention to detail, communication throughout the supply chain, and rigorous testing. Often specialty coffee is promoted and described by the particular flavor attributes from the individual harvest or micro-lot.

“In specialty coffee, the oscillation in quality year after year is generally accepted and even valorized. The consumer is more knowledgeable about coffee and appreciates the differences between crops and production year that influence the flavor notes,” Cossio says. “The main characteristic of commercial coffee is consistency. The consumer is expecting exactly the same taste profile year after year, so it is responsibility of the roaster to create a blend or a single-origin coffee that can be reproduced in a consistent way through sourcing and industrial processing.”

Strategically speaking, quality has two main measurements—status as an outright rating, or comparable to a specification. The quality rating that sets one coffee apart from others is commonly used when publicizing a sensory score or product rating and is often communicated as an absolute value. This is communicated to extol the virtues and general excellence of the coffee, its aroma, or perhaps the absence of imperfections. Quality as a comparable measurement corresponds to the consistency of the product and how it measures in relation to expectations for appearance, aroma and taste. In short, quality variations relate to how good the coffee is or how it compares to what it should be.

 Intrinsic Quality

The attributes of the coffee inherent in the bean as a result of botany and farming—the quality of coffee by its very nature—is intrinsic quality. For example, intrinsic quality is determined by the specific variety or cultivar, the growing altitude, components of the soil, amount of sunlight and rain, and more. Variation in intrinsic quality will include details than cannot change, but that can be sorted or separated. Intrinsic quality contains all the imperfections and defects as well as sugars, organic acids, carbohydrates, and other chemical compounds created by the plant that make up the quality potential. Intrinsic quality is not an absolute; it is reset with each harvest due to botany and farming variations.

Climate and geographic conditions will help produce a coffee emblazoned with the fingerprint of its growing place, says Niyonzima-Aroian. “Hence, the very importance of separating day lots, when possible, to preserve the uniqueness of terroir.”

Intrinsic quality is the starting point for the development of coffee into the final product. The coffee’s distinctive character potential has already been developed by the plant on the farm. The absolute value of the quality determination cannot be manipulated or created in excess of the coffee’s intrinsic quality.

Produced Quality

From this point on, the variations in quality—measured in absolute value or compared to a specification—can either be maximized based on the intrinsic quality, or reduced due to human interaction and/or processing. Produced quality can be affected by operational decisions and equipment operation on the farm, wet mill or dry mill.

“Absolute value would also require a singular reference point or definition, which is in the eye of the beholder. Supply chain calibration/alignment and good communication are tools for defining and maintaining produced quality,” says Bonnie Hall director of commodity procurement and quality at S&D Coffee &Tea in Concord, North Carolina.

Produced quality is the beginning of coffee customization. Decisions on the farm regarding application of fertilizer or controls for pests, fungus or unwanted plants contribute to the variations in produced quality. Produced quality is also dependent on the processing method selected; for example, fruit-dried, pulp-dried, parchment-dried and seed-dried options will yield different sensory characteristics in green coffee. Operational direction for dry milling is determined by buyers in the required cup and grade specification in their purchase contracts. What was once milling to a country standard has developed into milling to order by the buyer’s demand. Milling variations include the amount of imperfections as a result of cleaning and sorting, bean size as a result of screening, and cup/sensory profile as a result of blending or by selecting the appropriate parchment lots for processing.

“Supply-chain quality includes all the actors that support the movement of coffee from producer to roaster,” Hall says. Coffee roasters work tirelessly to both understand the quality variation and manage the consistency to mitigate changes to both intrinsic and produced quality.

Manufactured Quality

Once the green coffee is delivered to the roaster, variations in quality change from processing impact to the control of the manufacturing plant. Management decisions in quality assurance, product development, storage, and plant operations will directly contribute to maintaining the intrinsic and processed quality inherent in the green beans upon delivery, or they will change the outright quality rating for the coffee as well as the coffee’s ability to meet specifications.

Manufactured quality begins with the inherent quality of the coffee as a result of farming and processing and develops it further based on the intrinsic quality potential. The priority in manufactured quality is consistency based on physical and sensory expectation, which is established by the buyer’s product specification.

Quality assurance establishes quality requirements and works to prevent the purchase and manufacture of coffee that is outside acceptable specifications. Quality assurance determines control points and establishes standards for process controls and production. Part of quality assurance is quality control, which is responsible for fulfilling the quality requirements via inspections. These assessments determine adherence to specification and conformity to standards through rigorous measuring and analysis.

Manufactured quality, or controlling coffee as a roasted product, is typically our first thought when coffee quality control is mentioned. All prior control points are part of farming or supply chain, leaving manufacturing quality at the junction of product development and manufacturing, creating the coffee products presented to the consumer. The absolute value of roasted coffee sensory quality is designed and determined during the product development process, including blending and profile roasting. Process control decisions for the flow of coffee through the manufacturing plant as well as for equipment operation will contribute to quality variations for roasted coffee. Post-roasting decisions for grind particle size, resting and degassing, packaging, warehousing and inventory rotation will affect the manufactured quality and alter the quality variation of the finished product.

Prepared Quality

During the coffee’s transformation from a dry product into a beverage, the variations in quality increase exponentially. Quality variations as a result of logistics, storage and inventory rotations—including ambient conditions for temperature and humidity, length of supply chain and other factors—may be included in manufactured quality variations. Even though these issues, either controlled or uncontrolled, occur after the coffee leaves the manufacturing’s plant or warehouse, they are not prepared quality variations because they occur prior to brewing and serving.

Prepared quality is the realm of the barista. In cafes and foodservice operations, baristas have the authority and responsibility to determine and manage prepared quality. In retail operations, consumers will prepare the beverages to their liking, and when coffee is an ingredient the processor has discretion over the prepared quality.

Prepared quality is evaluated mainly by qualitative sensory analysis, which is the primary determination of beverage quality for both outright ratings and comparable measurements. Testing laboratories evaluate beverages for qualitative analysis using trained and expert sensory panels, and they also employ quantitative analysis using scientific equipment to measure the physical and chemical attributes of the beverage for adherence to product specifications or to measure the outright quality.

Detailed knowledge of the Golden Cup Standard and understanding the Specialty Coffee Association (SCA) Brewing Control Chart contribute to the management of prepared quality. When the brewed beverage is plotted on the SCA Brewing Control Chart relative to brew strength and solubles extraction yield, this is an example of quality variations. It is under the barista’s control. The actual plot point can be considered both the absolute value of quality for the beverage and the determination of quality as relevant to adherence to product specification. Was the coffee beverage created as designed and expected?

“The Brewing Control Chart is more related to the sensory experience than as an absolute value; it is possible to be within the plot and the coffee experience to be undesirable. Or be [an outlier] and have sensory quality to the customer’s liking,” Lopez-Thome says, “especially today with all the innovation in terms of equipment, green coffee process and extraction. You can use the chart as a guide, but at the end it is the sensory experience that will define cup quality.”

Emotional Quality

Quality variations by emotional and external sensory influences are real and will affect the outright quality rating of coffee. However, emotional quality is not intrinsic to the coffee; it is not produced, manufactured or prepared quality. Emotional quality is not quantitative; it is a subjective, qualitative measurement that is not transferable from one analysis to another. Emotional quality can either elevate a qualitative sensory score from positive emotions or reduce the score from negative emotions.

“Emotional quality is particularly evident at the consumer’s level,” says Niyonzima-Aroian. “For example, we have seen customers choosing sustainable coffee products over others because it makes all the difference in the world. For example, our International Women’s Coffee Alliance (IWCA) Burundi coffees have a deeper connection with our consumers, particularly due to the appreciation of our programs for our women farmers.”

Within supply chain operations, buyers and roasters develop relationships with coffee producers and traders. An affinity for a particular coffee, a coffee farm or the producer can be a result of a positive personal interaction or relationship. This affection will influence the quality rating of the coffee as determined by a sensory analysis. Unpleasant interactions, business frustrations, stress, or simply an argument will change one’s emotions and negatively affect the subjective quality determination. These emotions felt by the buyer or roaster cannot be transferred to another person who did not share the same experience or relationship.

External business influences also factor into emotional quality and can alter a coffee’s quality determination. For example, “market factors like currency exchanges and political factors,” Hall says, “are part of emotional quality.”

Multi-sensory flavor perceptions in coffee, as presented by Fabiana Carvalho, who holds a doctorate in psychobiology and is a post-doctoral researcher, explains how external influences change the way we determine the quality rating of coffee. Sensory influences (ranging from what cuppers and consumers hear to what they see when tasting coffee) change the sensory perceptions and quality ratings. Once the multi-sensory influences are removed, the variation in quality returns to normal.

Variations created by emotional quality can be mitigated by making quality determinations in controlled environments—your lab at your company, not at the farm or exporter’s office during a business trip. Evaluating coffee by a sensory panel and not as a single individual will identify the emotional quality rating as an anomaly and inconsistent with the scores by panelists who do not share the same emotion. It’s important to control the information provided to sensory panelists during evaluations by removing any supply chain or product information, or physical attribute measurement that may bias or influence the sensory results.

Perspective Quality

A distinct difference occurs when the cupper has a sense of ownership for the coffee. A coffee’s seller has an unmistakable perspective, which is different from that of a buyer. Quality is best determined by the buyer and not the seller, since emotional quality, ownership and perspective each play a role in subjective evaluations.

Quality variations between professionals and consumers exist in quality ratings, preference determination, and purchase intent decisions. Perspective quality can be found throughout the food and beverage industry—professionals who are constantly exposed to product flavors and aromas seek out the items that are different, are the exception to standard, and have intensity and extraordinary characteristics. Consumers do not always seek out and sometimes actually dislike these intense, exotic, contrasting characters. Coffees that have cupping scores in the high 80s and 90s do not always resonate with consumers who may not find the flavor and aroma appealing, while some professionals may consider coffees with sensory scores in the mid-80s as commonplace, based on their perspective.

“For many, unique is the enemy,” Hall says. “Many times, our job as coffee professionals is to ensure consistency.”

Perspective quality may create quality variation based on expertise, experience and training among professionals. For this reason, qualitative scores are often scrutinized while the buyers or interested parties consider the source of the quality rating. Different from the perspective of professional and consumer, or buyer and seller, this example of perspective quality calls into question reliability of the quality score and credibility of the cupper. After taking a cupping training course for a few hours in a formal classroom by a certified instructor or an informal training led by a consultant or employer, the cupper should be able to follow basic protocols for sensory analysis and successfully complete the cupping score sheet.

“It takes years to fully develop as a trained coffee sensory expert,” Hall says. Limited expertise from the completion of classroom training, even with practical education activities, or limited professional experience will not correct for quality variation based on perspective. Additional experiential training is needed to produce score alignment with veteran sensory specialists and calibration of flavor identification using flavor standards, or simply with increased experience.

TOOLS OF MEASUREMENT

Coffee industry professionals follow precise methods for analysis, record measurements, write reports and interpret data to quantify a coffee’s quality. Quantitative quality uses structured techniques, such as published methods and procedures, and calibrated instruments to determine a measured value of the coffee. Some examples of physical quantitative quality are net weight, moisture content, water activity, bean size, density, grind particle size, color, and headspace measurements for oxygen and carbon dioxide.

Quantitative tests are not all the same, and the results may vary depending on the methods or equipment. For example, there may not be a correlation in results for beverage total dissolved solids (TDS) testing between the three main analysis technologies: gravimetric, using an oven to evaporate the liquid; refractometer, measuring the light change in the liquid; and conductivity, measuring how strongly the liquid conducts or resists electric current. When a primary method is determined and published by a scientific authority such as the International Standards Organization (ISO), Association of Official Agricultural Chemists (AOAC), or American Society for Testing and Materials (ASTM), analysis using other (secondary) technologies or methods must be calibrated to the primary method to be able to correlate and compare the results.

Quality variation may occur during the measurement of data by differences in technicians. For example, when testing for bean size using hand screens the arc, torque and velocity of the swing when shaking the screens will impact the quality determination. 

On the other hand, qualitative quality uses observation. Often, trained individuals, who are aligned with each other and who have been calibrated to a standard, determine an understanding of the coffee and explain the effects of the quantitative results. Qualitative quality tests to increase understanding of the effects of measurements recorded in quantitative data collection. Examples of qualitative quality are sensory evaluations for color, fragrance, aroma and taste.

Sensory training is required to ensure that quality variations are a result of changes in the coffee and not issues with the sample preparation or analysis. Sensory panelists have a tendency to alter their scores over time to align with other panelists—there is a natural aversion to being the outlier. Also, junior panelists seek to align their scores, through experience training, to senior panelists. To prevent these quality variations due to score alignments, regular calibration with sensory standards are required to ensure the accuracy of qualitative sensory results.

Calibration to standards and alignment to other sensory panelists help prevent quality variations as a result of emotional quality and multi-sensory perceptions. Quality alignment occurs when the absolute value of coffee quality matches the expectation of quality as defined by the product specification. When trained and calibrated sensory specialists determine the sensory score of a coffee to be within the acceptable range of the specification, the quality determination meets both outright rating and comparison. 

Quality variations are confusing when the outright rating exceeds the specification or expectation—the coffee can be considered both “high quality” and “not quality” at the same time.

ANALYSIS VARIATIONS

To ensure that quality variations are a result of the differences in the coffees and not a result of the testing, there are several factors to consider. Quality variations may change the management of a coffee, causing an acceptable coffee to be rejected or a coffee that should be rejected to be accepted. Either instance has the potential to damage a company’s reputation based on quality inconsistency, which could adversely affect its financial stability.

“Companies should identify the goal of the analysis,” Cossio says. “Is it to create a quality product? To control the quality? Or to track data that needs to be analyzed and keep the quality?”

Testing equipment should be kept clean and in like-new condition, and be calibrated to certified standards by the operator or confirmed for standard operation by the manufacturer on a scheduled basis. Published methods for analysis should be followed for testing; different methods or technologies will produce different results that may not correlate to the product specification or compare to tests completed at other points in the supply chain.

“Quality can be visualized as a long chain, done by 113 rings connected together. If just one of them is weak and breaks, the final quality is affected,” Cossio says. “That’s the reason why it is so important to take care of each single step of production flow, always considering the final goal and the final consumer.”

Staff should be trained on the both the operation of testing equipment and how to interpret the data to make operational decisions. Data interpretation includes understanding emotional quality and multi-sensory flavor perceptions, as well as controlling the testing environment and the testing process to minimize misleading results. The variations of quality may be dramatically different as a result of intentional or unintentional human impact, management decisions or operational processes—or simply designed by the buyer’s request.

When discussing coffee quality, it is important to designate whether the topic is outright quality tested for absolute value or comparable to standards and adherence to specifications. Additional details regarding coffee quality should include the type of quality variation—intrinsic, produced, manufactured, prepared, or emotional quality—to mitigate any potential confusion.

#########################

SPENCER TURER is vice president of Coffee Enterprises in Hinesburg, Vermont. He is a founding member of the Roasters Guild, a licensed Q grader, and received the SCAA Outstanding Contribution to the Association Award. Turer is an active volunteer for the Specialty Coffee Association, the National Coffee Association USA, and is an Ambassador for the International Women's Coffee Alliance.