Quarterly Labour Force Survey 2011

Data Processing

Introduction :
The purpose of data processing is to ensure that the information collected from the sampled primary sampling units, dwelling units and households (i.e. the boxes containing QLFS questionnaires) are physically received, stored and processed. The aim is to produce a clean dataset that has all the information contained in the questionnaires. Except for the scanning system, all other elements of the data processing system were developed in-house. One important innovation that is central to the smooth operation of the entire system is the development of barcodes that are linked to a unique number on each questionnaire. This information provides the link between the information recorded in the Master Sample database and other processes such as editing and imputation as well as weighting and variance estimation.

Processing phases :
QLFS data processing is continuous, starting on the second week of every month. Data processing for each quarter must be completed by the first Friday of the subsequent month to ensure that the four-week deadline for publication of the QLFS results is met.

The phases listed below occur sequentially.

Receiving of questionnaires :
The contents of the boxes containing questionnaires sent from the regional offices are verified when received at the DPC. The questionnaire barcodes captured in the provinces are captured again at the DPC to ensure that all questionnaires have been received.

Primary preparation :
The purpose of primary preparation is to ensure that all questionnaires are correctly stacked and positioned prior to being guillotined.

Guillotining:
The purpose of the guillotine process is to cut off the spines of the questionnaires in order to have pages separated for scanning.

Secondary preparation :
The purpose of secondary preparation is to ensure that the questionnaires are correctly stacked and positioned for scanning. At the same time, quality assurance takes place on the work done during the primary preparation and guillotining processes.

Scanning :
The purpose of scanning and recognition is to convert the questionnaires into an electronic format and Tagged Image File Format (TIFF) images.

Verification :
The purpose of scanning verification is to manually correct un-interpretable characters, missing data and errors detected by validation rules.

Electronic coding:
Industry and occupation codes are assigned using the electronic coding system which converts the respondents' industry and occupation descriptions into numeric codes based on Standard Industry Classification (SIC) and South African Standard Occupation Classification (SASCO). If the system fails to assign a code for either industry or occupation, the coding is assigned manually.

Automated editing and imputation :
QLFS uses the editing and imputation module to ensure that output data is both clean and complete10. There are three basic components, called functions, in the Edit and Imputation Module:

Function A: Record acceptance
Function B: Edit and imputation
Function C: Clean up, derived variables and preparation for weighting
Function A: Record acceptance

This function is divided into three phases:

First phase: Pre-function A :
The first phase ensures that the records contain valid information in selected Cover Page questions required during edit and imputation and during the subsequent weighting and variance estimation. Any blanks or other errors that need to be corrected are done here before processing of the record can proceed.

Second phase: Function A record acceptance :
The second phase ensures that there is enough demographic and labour market activity information to ensure that editing and imputation can be successfully completed.

Third phase: Post Function A clean up :
This phase ensures that certain data are present where there is evidence that they should be. This for example, involves:
• Ensuring that if there is written material in the job description questions then there are corresponding industry and occupation codes for them.
• Ensuring that partial blanks or non-numeric characters that appear in questions where the Survey Officer is required to enter numbers are validated.
• Ensuring that where there is written material in the space provided for "Other - specify" that the corresponding option is marked.

Function B: Edit and imputation :
Having determined in Function A that the content of the record would support extensive editing and imputation, this function carries out those activities. Editing is the detection of errors in the captured questionnaire. Imputation is the correction of the detected errors.

Function C: Clean up, derived variables and preparation for weighting :
Function C includes all of the "post E&I clean up" functions such as "Off-path cleaning", "Result Code validation", verification of the presence of industry and occupation codes, and the generation of all derived variables.

Electronic data processing systems have been developed to ensure that the key QLFS results are published four weeks after the end of data collection each quarter. The system is fully automated and includes the seven sub-systems discussed in detail in the subsections that follow (9.3.1 to 9.3.6): Real Time Management System (RTMS)
RTMS serves two important functions. Firstly, it is a management tool for personnel involved with field operations to monitor progress. Secondly, it provides an important link between field operations and data processing as follows:
• Ensures that publicity information at the PSU and dwelling unit level can be rapidly assessed, thus allowing for speedy intervention should the need arise.
• Enables the tracking and monitoring of PSU listing books and questionnaires from the provinces to HO as well as through the processing phase.