The following instructions summarize the data verification process that all GDMP Program Managers are expected to perform when reviewing MMP survey responses.
Verification Overview
The following sections provide a high-level overview of the verification process and the stakeholders involved.
Program Managers
Each GDMP customer can assign up to two (2) program manager accounts (referred to as “Authorized Users” in Customer Terms of Service). GDMP program manager accounts (referred to as "program managers") are configured to access MMP data for all members in the customer’s jurisdiction. The primary role of GDMP program managers is to review data submitted via the MMP’s surveys to identify inaccurate data or response discrepancies.
If you haven’t received an invitation to access your program manager account, please contact mmp@re-trac.com
Data Verification Workflow
The following section describes the data verification process that program managers should follow when reviewing MMP survey response data.
The Recycling Partnership will provide basic data verification training to all new program managers.
The Recycling Partnership will also perform a final review of MSW Collection Programs and Recycling Collection Programs survey responses to confirm data quality and provide feedback to program managers as necessary.
BioCycle will perform verification of all Yard & Food Waste Collection Programs survey responses and provide feedback to confirm data quality and provide feedback to GDMP program managers as necessary.
The general flow of the verification process is as follows:
An MMP member (participating municipality, county, or regional agency) marks all surveys as “complete”.
The program manager is notified by automated email that the MMP member’s surveys have been marked as "complete".
The program manager logs into Re-TRAC™ to review the completed survey responses.
The program manager follows the steps outlined in the Data Verification Protocol (see Data Verification Protocol section below).
As the survey response is reviewed, program managers can enter notes into the "Comments" section at the bottom of each survey. Program managers should use this feature to document important review notes including changes made to the survey response and the steps taken to confirm the accuracy of the data. The "Manager Notes" field located under the "Manage Response" section should not be used for this purpose.
Once the program manager has completed a review of the MMP member’s survey response, including contact with the member to address any data accuracy issues, they should use the Manager Team Sign-Off section located towards the bottom of each survey to indicate if the survey response is still under review or ready for BioCycle and The Recycling Partnership to perform final data verification procedures.
Click on the radio button "No, still under review" if the program manager review has started but is not yet complete.
Click "Yes, ready to verify" if no further member follow-up is needed.
The Recycling Partnership and BioCycle will perform a final review of the MMP member’s completed survey response, they will review any program manager review notes, and address any outstanding questions directly with the program manager.
The Recycling Partnership and BioCycle will mark the MMP member’s survey responses as “Verified”.
Example of a survey response that is still under review by the program manager.
The MMP Surveys
Full participation requires that MMP members complete the three core MMP surveys. The Recycling Partnership and BioCycle will verify each survey response afterthe program manager review has been completed.
The MSW Collection Programs survey collects data about the member's single-family curbside, multi-family, and residential drop-off MSW collection programs.
The Recycling Collection Programssurvey collects data about the member's single-family curbside, multi-family, and residential drop-off recycling collection programs.
The Yard & Food Waste Collection Programs survey collects data about the member's single-family curbside and residential drop-off yard & food waste collection programs.
All three (3) surveys in marked as "Completed"
Getting Prepared
We recommend that program managers establish a system to monitor and track member survey responses, regularly updating an inventory of submissions to then schedule verification activity. It is further recommended to conduct verifications when all surveys are marked as "complete".
Problematic data may require direct contact with MMP members to verify data accuracy and/or make corrections. Before opening a survey response for review, it’s good practice to establish a method to track all problematic data points to be addressed with the MMP member in one summary contact.
Helpful Tips
Members cannot mark survey responses as “complete” until data has been entered into all required fields. Therefore, all essential program information will be available for the program manager's review when survey responses are marked as “complete”.
Verification should focus on the appropriateness of the numerical data, both in raw form (i.e. entered directly into a field) and also in the calculated form (e.g. pounds per household metric).
This section provides helpful tips for identifying common data entry errors and a guide to completing the verification process for each survey.
Watch For These Common Errors
The following section addresses general verification steps for questions that appear in all three MMP surveys.
Conflicting Response Options
Each survey starts with a series of checkbox questions instructing the member to select the agency or agencies responsible for providing service for each of the residential collection programs. Members can select any or all of the response options provided but one of the options is listed as "Service is not available in my jurisdiction" so if a member selects this option, please ensure that none of the other response options have been selected. If "Service is not available in my jurisdiction" is selected alongside any of the other options, please contact the member to confirm whether or not the service is available to the residents in their community.
Invalid checkbox response example
Incorrect Use of "Other"
Where "Other" is available as a response option, check the text provided in the adjacent "Please Specify" text box. If the “please specify” field contains information that is already represented by the available choices, contact the member to confirm the information and/or directly change the answer in the completed survey response.
Example: A member checks “Other” for the types of containers used in their single-family curbside recycling program and writes "recycling carts" in the “Please Specify” field despite "Roll carts" being available as a response option.
Example: "Please Specify" has text matching available options so the response should be corrected.
Opinion-based Questions
Each survey includes opinion-based questions that can be reviewed but should not be changed by the program manager.
.
Example 1: Top challenges related to delivering MSW collection programs
Example 1: The "top challenges" question would be considered opinion-based because the reporter is providing a subjective response.
Example 2: Please share any relevant supplemental information about your single-family curbside MSW collection service.
Example 2: Questions that invite the member to provide supplemental information would also be considered opinion-based.
This section provides instructions for completing the verification process for each survey
MSW Collection Programs Survey
This survey collects data about the member's 2022 curbside and drop-off MSW collection services.
1. Single-family curbside MSW tonnage (Lbs/HH)
According to The Recycling Partnership, single-family households generate an average of 2,000 pounds of MSW per year. If the "Total SF Lbs/HH MSW" field is 50% higher (3,000 or more pounds per household) or lower (1,000 or fewer pounds per household), the response should be flagged for review to confirm the total tons of MSW collected by the single-family curbside program, with the member.
Lbs/HH MSW greater than 3,000 or less than 1,000 should be flagged for review with the member.
2. Multi-family households served
Review the multi-family household number provided. If it looks like an unusually low number, the participant may have entered the number of multi-family buildings (e.g. apartment blocks) and not the number of multi-family units (i.e. number of households). Follow-up with the member may be required.
Recycling Collection Programs Survey
This survey collects data about the member's 2022 curbside and drop-off recycling collection services.
According to The Recycling Partnership, the average household generates 440 Lbs/Household of recycling per year. If the "Total Lbs/HH recycled" field exceeds 650, or is under 150, make a note to contact the member to confirm their recycling tonnage information.
Total Lbs/HH recycled greater than 600 or less than 150 should be flagged for review with the member.
2. Containers used for curbside recycling
If the MMP member checks more than one option in "What types of containers are used for your curbside recycling program?", make a note to contact the member for clarification.
Multiple options selected for containers used in single-family curbside recycling collection.
3. Multi-family Households Served
Review the multi-family household number provided. If it looks like an unusually low number, the participant may have entered the number of multi-family buildings (e.g. apartment blocks) and not the number of multi-family units (i.e. number of households). Follow-up with the member may be required.
Excessively high (600 lbs+) or low (100 lbs) is a signal to contact the member to confirm tonnage and household figures.
Issues to explore with the MMP member related to lbs/hh values:
Is the number of households served by the program accurate?
Is the annual tonnage number accurate?
Is the number missing any tonnage that should be attributable to the program?
Does the number include tonnage that should not be counted for the program (e.g. drop-off or multi-family collection tonnage entered as single-family curbside tonnage)?
Documenting Results After Contacting Members
After contacting the member to clarify their survey responses:
If the member provides reasonable justification for household and tonnage numbers, make a note in the "Comments" section. You can now consider this survey response reviewed and ready for final verification. Check the "Ready to verify" check box.
If the household or tonnage numbers need to be changed, enter the correction directly into the survey response and click "Save". Remember to document the conversation and outcomes in the "Comments" section.
