Selecting labor job codes in field surveys

Labor markets across the world bring with it nuances that are difficult for researchers to standardize, especially for the ones in developing countries. The informality and structure of the economy may allow for occupations to exist that are extremely country-specific. Such idiosyncrasies of labor markets make the lives of researchers (and their RAs) a ~bit~ difficult. In order to draw a meaningful picture of a particular country and tie it to the development narrative of that particular region, ideally one would like to bunch up jobs in “standardized labor codes” that enable comparisons to be made. But what are these job codes and how do you assign one to a particular job?

How enumerators select a job code

Surveys that ask questions related to labor market outcomes usually collect the job description of the individual, and assign a “job code” relevant to that. Enumerators--the real heroes of research-- are given the mammoth task of extracting every iota of information from a household which typically ranges from what their household ate in the last month to what types of assets they own. In a standard labor market module, enumerators pick a relevant job code based on the information provided by the respondent. But how do they do so? Enumerators collect information on what type of work the respondent does (agriculture or not), how much they earn, how frequently they are paid (hourly v monthly v daily) and a barrage of other questions. Based on this information as well as the verbatim description of the activity the respondent does, the enumerators select a job code from a pre-existing list they are given. Sounds like a lot, right??

The status quo

Researchers, even the demanding ones, usually try to reduce the cognitive burden on the enumerators since their research is only as good as the data collected. Selecting a job code as part of a survey may take roughly 30 seconds to 1 minute, but it does eat away the enumerator’s energy which may subsequently affect their performance in later questions. Additionally, there is a chance of enumerators selecting an incorrect code. So what do you do?

Introduction of a labor recoder

In an effort to see whether it makes sense to ask enumerators to select the job code while they conduct the survey, we used a pilot survey of a recent study to check their performance in comparison to someone choosing the job code from the comfort of their air-conditioned workspace. The enumerators chose a job code like they usually do during such surveys and we asked a “labor recoder” to also go through the job description collected and choose a job code they think fits.?

What did we find?

The enumerators can pick out job codes well--but are often confused. Enumerators were able to select the correct job code for 58% of the sample. Before we point fingers at their performance, it is important to understand what the different codes tell us. Are they actually incorrect or do we not have information they have from the field? If we break down the answers, we can attribute 26% to completely different job codes selected. For example, a respondent says that they are a landlord, but the enumerator’s selected job code says that this person is a farmer.?

The other two buckets of different answers are interesting. The enumerators were thinking along the right lines but ended up choosing a different code than the labor recoder. An important distinction when collecting labor market outcomes is whether the respondent is a wage worker or has their own business. In our survey, we explicitly asked a question where we collect this information. However, it is difficult for the enumerator to always be cognizant of what information they collected.? In 8% of the cases, enumerators correctly identified the type of work the person does, but when they selected a job code, they selected the one that denotes a wage worker for that type of work, and not a business owner. Let me elaborate. A person who works in a parlor is a beautician/salon/parlor worker. But a person who owns the parlor can also be a beautician/salon/parlor worker, but they are a shop owner of a service business--we had a job code that allows for this to be denoted. The enumerator is not 100% incorrect, but they are unable to make an important distinction.?

The last tranch of different job codes is where the enumerators can claim that the job code they selected is indeed correct, the labor recoder is the one who is incorrect. Field surveys in developing countries bring with it nuances that cannot be foreseen. Information collected by the enumerators may not entirely capture the candid conversation that took place between them and the respondents. Maybe the conversation gave them an indication to select a job code that is different to the one selected by the recoder. For example,? a person who has a poultry farm is different from a person who rears poultry. In essence, they are the same, but the scale and intention is different. In cases such as this where you can advocate for both sides, it is important to ask the enumerator what made them select the code they selected.?

So who selects the job code??

Do we let the enumerators do it in the field or have a labor recoder do it in parallel to data collection? A piece of important information I glossed over is that the work of the labor recoder is overseen and corrected by an RA, who adds another layer of screening to the codes. It is obvious that this method will yield more accuracy; but it requires an increased level of effort from the RA to oversee both the data collection and labor recoding. Is a higher level of accuracy needed? Enumerators are the primary points of contact with respondents and absorb more information than our surveys do. As researchers, we should help enumerators reduce friction on the field as much as possible by reducing any extra analytical work required for them. Before selecting a labor code, enumerators can be prompted with a snapshot of labor market indicators they collected (wage work v business owner, hourly wage v monthly wage etc). A refresher such as this can help reduce (2) and (3). But (4) is something that you can reduce, but not eliminate. The utility of a labor recoder hinges on the discretion of the researcher. For large surveys, it makes sense to add another pair of hands to help siphon through the responses of the enumerators. It ensures better data quality, but adds to cost and need for supervision. For smaller surveys, just make the RA do it--they are probably not working and writing a blog post.