Data Verification Protocol

Data Verification Protocol Icon

MMP DATA VERIFICATION PROTOCOL

The following instructions summarize the data verification process that all GDMP Program Managers are expected to undertake when reviewing program assessment survey responses submitted by MMP members in their respective jurisdictions.

Program Managers

Each GDMP Customer will assign up to two (2) Program Manager Accounts (referred to as “Authorized Users” in Customer Terms of Service). GDMP Program Manager Accounts are configured to access Municipal Measurement Program (MMP) data for all members in the Customer’s jurisdiction. The primary role of GDMP Program Manager’s is to review data submitted via the MMP’s program assessment surveys to identify inaccurate data or response discrepancies.

If you haven’t received an invitation to access your Program Manager Account, please contact mmp@re-trac.com


The Importance Of Data Verification

The following section describes the data verification process that GDMP Program Managers are expected to undertake when reviewing MMP members' program assessment survey response data.

The Basics

  • The Recycling Partnership will provide basic data verification training to all new GDMP Program Managers.
  • The Recycling Partnership will also perform a final review of all MMP program assessment survey responses to confirm data quality and provide feedback to GDMP Program Managers as necessary.
  • The general flow of the verification process is as follows:
  • An MMP member (participating municipality) marks all relevant surveys as “complete”.
    • GDMP Program Manager is notified, by automated email, that the MMP member’s surveys have been marked as "complete".
    • GDMP Program Manager logs into Re-TRAC Connect™ to review the completed survey responses.
    • GDMP Program Manager follows the steps outlined in the Data Verification Protocol (see below).
    • Once the GDMP Program Manager has completed a review of the MMP member’s survey response, including contact with the municipality to address any data accuracy issues, the GDMP Program Manager will use the “Audit only” Manager Team Review and Sign-Off Window on the last page of each survey to:
      • Provide the full name of the GDMP Program Manager who reviewed the survey response.
      • Specify how changes to the survey response were confirmed (as applicable).
      • Document important review notes including changes made to the survey response and the steps taken to confirm the accuracy of the data.
      • Specify the date that the survey response was reviewed.
      • Click on the radio button “Yes, ready to verify” to indicate that the survey response is ready for The Recycling Partnership to perform final data verification procedures. Note: do not use the “Manage Response” window at the bottom of the page for your review; that section is for Recycling Partnership use.
        • The Recycling Partnership performs a final review of the MMP member’s completed survey response, reviews the GDMP Program Manager’s review notes, and addresses any outstanding questions with the GDMP Program Manager.
        • The Recycling Partnership marks the MMP member’s survey responses as “Verified”.

MMP Surveys

Full participation requires that municipalities complete the three core MMP surveys. The Recycling Partnership will verify each survey response after the GDMP review has been completed.

  1. The About Your Municipality survey collects general demographic information about the municipality and information about its education and outreach programs.
  2. The Residential Curbside Program Assessment survey collects information about the curbside programs available to residents in the municipality, including municipal solid waste (MSW), single-family recycling, multi-family recycling, and organics collection.
  3. The Residential Drop-Off Program Assessment survey collects information about the drop-off sites operated by the municipality, including MSW, recycling, source-separated organics, household hazardous waste (HHW), scrap tires, electronic waste, construction & demolition (C&D), and scrap metal / white goods.

There are two optional surveys that municipalities can complete to enhance their program performance analysis. The Recycling Partnership may review but will not verify this survey data after GDMP review.

  1. The Public Space Program Assessment survey collects information about where public space bins are located, the types of containers used, and tonnage information for recycling and MSW collected from public spaces in the municipality.
  2. The Processing Facilities Assessment survey will collect information about how your municipality works with facilities to manage the materials it collects from residents. It also collects information about the percentage of materials that are diverted versus disposed. By completing this survey, municipalities will unlock the Environmental Benefits Report.

Getting Prepared

  • Bookmark and become familiar with using: U.S. Census Bureau Data
    • This tool will replace the previously used: U.S. Census American Fact Finder 
    • It should be used to verify household and population data submitted by MMP members.
    • It’s also useful when making calculations to catch program data anomalies that need to be addressed.
  • It is recommended to establish a system to monitor and track member submittals, regularly updating an inventory of submittals to then schedule verification activity. It is further recommended to conduct verifications when all of the expected reports are in from a submitting member.
  • Problematic data may require direct contact with MMP members to verify data accuracy and/or make corrections.  Before opening a program assessment survey response for review, it’s good practice to establish a method to track all problematic data points to be addressed with the municipality in one summary contact.

Helpful Tips

  • Members cannot mark program assessment survey responses as “complete” until data has been entered into all required fields. Therefore, all essential program information will be available when GDMP Program Managers review survey responses marked as “complete”.
  • Verification should focus on the appropriateness of the numerical data, both in raw form (i.e. entered directly into a field) but also in the calculated form (e.g. pounds per household metric).
  • Conduct baseline checks against numerical data:
    • Check population and household counts against U.S. Census Bureau Data for the community.
    • Check for both single-family and multi-family units according to how the MMP member defines each one.
    • If household counts are outside of 20% difference with the U.S. Census Bureau Data, plan to contact the MMP member to discuss.

Addressing Specific Survey Questions

The following section addresses key verification questions in the three core MMP surveys: About Your Municipality, Residential Curbside Program Assessment, and Residential Drop-off Program Assessment.

About Your Municipality

Every municipality will fill out the About Your Municipality survey. It collects general demographic information about the municipalities in your jurisdiction and high-level information about the types of waste and recycling programs available to residents in your jurisdiction.

What is the population of your municipality?

Reference the U.S. Census Bureau Data to check for the correct number.

If the number entered by the MMP member seems too high or too low (i.e. outside of 20% difference with number listed on U.S. Census Bureau Data), make a note to contact the MMP member to ask if they have more up-to-date information. Specifically, ask about the source of the data and ask them to describe any factors that would cause such a large difference when compared to the U.S. Census Bureau Data. Note that the survey does ask about seasonal populations as a possible context for any discrepancies.

  • If the MMP member provides reasonable justification for the originally submitted number, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
  • If the number needs to be changed, enter it directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.
GDMP Data Review: Population

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following two questions are classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

  1. What are the biggest waste management and recycling challenges for your municipality? (select all that apply)
  2. What other waste management and recycling challenges does your municipality contend with? (This question is displayed when the municipality selects "Other" in the preceding question)
Waste Management Challenges

RESIDENTIAL CURBSIDE PROGRAM ASSESSMENT SURVEY

This survey is separated into six sections designed to summarize waste & recycling programs in the municipality.

  • Page 1 gathers basic information about curbside collection programs
  • Page 2 collects information about MSW curbside collection
  • Page 3 collects information about single-family curbside recycling collection
  • Page 4 collects information about multi-family recycling collection
  • Page 5 collects information about curbside organics collection
  • Page 6 collects information about the person completing the survey

HELPFUL TIP:
Where "Other" is available as an answer option for any given question, check the text provided under “Please Specify”. If the “Please Specify” field has information that is already provided in the answer choices, contact the MMP member to get the correct information and directly change the answer in the completed survey response.

Example: An MMP member checks “Other” for types of containers used in their single-family curbside recycling program but then notes under “Please Specify” that they use "recycling carts".

Option already available

PAGE 1: ABOUT CURBSIDE COLLECTION PROGRAMS IN YOUR MUNICIPALITY

Do residents in your municipality have access to any curbside collection services (see definitions below)?

The first question in this survey is the most important because it affects every subsequent survey page. If the municipality checks “No” to this question, then none of the detailed program questions on pages 2 through 5 will be available for data entry (though each page will need to be saved by the member before the survey can be marked as complete).

If you are aware that the municipal participant does indeed have programs in the area of curbside solid waste collection, curbside recycling collection, multifamily collection, or organics collection, then this answer should be changed to "Yes" and the participant should be contacted to fill out the applicable sections.

Curbside Program Assessment - Question 1

Based on the definitions you provided and referencing the U.S. Census Bureau Data, please provide the total number of single-family and multi-family households in your municipality. 

Reference the U.S. Census Bureau Data as you review this question. The number of unit choices in the question is more granular than the unit numbers available in the Census data but they are designed to help participants indicate the specific features of their program eligibility.

Single-family and multi-family household numbers

In conducting verification, check the U.S. Census Bureau Data against the submitted single-family and multifamily numbers. If the number of households seems too small or too large (i.e. outside of 20% difference with Census number), make a note to contact the MMP member to ask if they have better data. Specifically ask about the source of the data and ask them to describe any factors that would cause such a large difference when compared to the Census.

  • If the MMP member provides reasonable justification for the numbers they originally submitted, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
  • If the number needs to be changed, enter it directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.
GDMP Review Households

PAGE 2: ABOUT YOUR RESIDENTIAL CURBSIDE MUNICIPAL SOLID WASTE (MSW) COLLECTION PROGRAM

Residential curbside MSW tonnage

The tonnage submitted on page 2 should be a focus of attention for verification. Verification staff will see a field that is hidden to participants but that is key to reviewing this data: Total Lbs/household MSW.

In general terms, households generate an average of 2,000 pounds per year. If the submitted answer is 50% higher (3,000+ pounds) or lower (1,000- pounds), it is a signal to check with the municipal participant on the data.

Reasons for problematic answers in this question may include:

  • Issues with household counts
  • Issues with MSW tonnage
MSW Tonnage

After contacting the participating member:

  • If the MMP member provides reasonable justification for the tonnage they originally submitted, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
  • If the number needs to be changed, enter it directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.
GDMP Review MSW Tonnage

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following question is classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

Is there any additional information you'd like to share about the way Municipal Solid Waste collection is provided in your municipality? Tell us your story! Please provide as much detail as possible.

Opinion-3

PAGE 3: ABOUT YOUR RESIDENTIAL SINGLE-FAMILY CURBSIDE RECYCLING PROGRAM

What type of curbside recycling collection do you offer residents?

If the MMP member checks more than one option, make a note to contact the MMP member for confirmation and clarification.

Multiple curbside recycling collection options selected

What types of containers are used for your curbside recycling program?

If the MMP member checks more than one option, make a note to contact the MMP member for confirmation and clarification.

Mulitple container types selected

Are the following types of PAPER accepted in your municipality's curbside recycling collection program?

If the MMP member selects "Other paper", a text box will be displayed with the following question: What other types of PAPER are accepted in your municipality's single-family curbside collection program? Are there any exceptions? Tell us your story!

Check to make sure the answers provided in this text box are already listed as a paper type above.

  • If the option is available in the list of paper types, click "YES" for that paper type and delete any reference to it in the text box.
  • If answer appears to be problematic, make a note to contact the MMP member for confirmation or clarification. Once the correct paper types have been confirmed, enter them directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

Example: An MMP member selects "YES" for "Books" and "Mail", but also checks "YES" for “Other paper” and writes "We also accept junk mail and text books". These options should be covered by the "Books" and "Mail" so the comment should be deleted.

Paper selection
Paper Selection is wrong

Are the following types of PLASTICS accepted in your municipality's curbside recycling collection program?

If the MMP member selects "Other plastics", a text box will be displayed with the following question: What other types of PLASTICS are accepted in your municipality's single-family curbside collection program? Are there any exceptions? Tell us your story!

Check to make sure the answers provided in this text box are already listed as a plastics type above.

  • If the option is available in the list of plastics types, click "YES" for that plastics type and delete any reference to it in the text box.
  • If answer appears to be problematic, make a note to contact the MMP member for confirmation or clarification. Once the correct plastics types have been confirmed, enter them directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

Example: An MMP member selects "YES" for “Other plastics” and writes "We accept #1 and #2 plastic bottles and containers". These options would already be covered by the "PET (#1) bottles and jars", "PET (#1) non-rigid containers and other packaging", "HDPE (#2) bottles and jars", and "HDPE (#2) non-bottle rigid containers and other packaging" so the comment should be deleted and "YES" should be selected for each of the aforementioned categories.

Plastics Selection
Plastics detail should be captured by existing options

Are the following types of GLASS accepted in your municipality's curbside recycling collection program?

If the MMP member selects "Other glass", a text box will be displayed with the following question: What other types of GLASS are accepted in your municipality's single-family curbside collection program? Are there any exceptions? Tell us your story!

Check to make sure the answers provided in this text box are already listed as a glass type above.

  • If the option is available in the list of glass types, click "YES" for that glass type and delete any reference to it in the text box.
  • If answer appears to be problematic, make a note to contact the MMP member for confirmation or clarification. Once the correct glass types have been confirmed, enter them directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

Example: An MMP member selects "YES" for “Other glass” and writes "We accept beer and wine bottles". These options would be covered by the "Glass containers" so the comment should be deleted and "YES" should be selected for "Glass Containers".

Glass Selection
Glass selection wrong

Are the following types of METAL accepted in your municipality's curbside recycling collection program?

If the MMP member selects "Other metal", a text box will be displayed with the following question: What other types of METAL are accepted in your municipality's single-family curbside collection program? Are there any exceptions? Tell us your story!

Check to make sure the answers provided in this text box are already listed as a metal type above.

  • If the option is available in the list of metal types, click "YES" for that metal type and delete any reference to it in the text box.
  • If answer appears to be problematic, make a note to contact the MMP member for confirmation or clarification. Once the correct metal types have been confirmed, enter them directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

Example: An MMP member selects "YES" for “Other metal” and writes "We accept pop and beer cans". These options would be covered by the "Aluminum cans" so the comment should be deleted and "YES" should be selected for "Aluminum cans".

Metal selection
Metal selection wrong

Can you provide 2019 tonnage information specifically for your municipality's single-family curbside recycling program?

Follow this protocol if the MMP member selects one of the following four responses:

  1. Yes, I can provide total annual tons specifically for single-family curbside recycling
  2. Yes, but single-family curbside recycling tonnage is combined with tons from other programs (e.g. drop-off multifamily, commercial, school programs, etc.)*
  3. Yes, I can provide annual tons broken out by material category (paper, plastic, glass, metal)
  4. No, I don't have any tonnage data for the curbside recycling program

*If the municipality selects number 2 above, they will be asked to identify which other programs are included in the tonnage number provided. They will also be asked to estimate what percentage of the tonnage number belongs to the single-family curbside program.

The verification objective in this section is to check for excessively high or low Total Lbs/household recycled numbers.  Verifiers will see a “For Audit purposes” field (which is hidden from members) that presents this calculation.

If the Lbs/Household number exceeds 650, or is under 150, make a note to contact the MMP member to confirm and clarify these numbers.

Issues to explore the MMP member:

  1. Is the number of single-family households served by the curbside recycling program accurate?
  2. Is the annual single-family curbside recycling tonnage number accurate?
    • Is the number missing any tonnage that should be attributable to the program
    • Does the number include tonnage that should not be counted for the single-family curbside program (e.g. drop-off or multi-family collection tonnage)?  This may be indicated by a member answer that says the tonnage includes non-curbside program materials. In that case, the member’s estimate of the percentage of tonnage attributable to the curbside program may need to be adjusted through engaging with the member.
SF Curbside Recycling Tonnage
GDMP Review SF Curbside Recycling Tonnage

After contacting the participating member:

  • If the MMP member provides reasonable justification for household and tonnage numbers, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
  • If the household or tonnage numbers need to be changed, enter the correction directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following question is classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

Is there any additional information you'd like to share about the way single-family curbside recycling collection is provided in your municipality? Tell us your story! Please provide as much detail as possible to explain your answer.

Additional-information-about-single-family-curbside-recycling

PAGE 4: ABOUT YOUR RESIDENTIAL MULTI-FAMILY RECYCLING PROGRAM

How many multi-family households are served by municipal recycling collection services?

It's not uncommon for MMP members to enter the number of multi-family complexes (buildings) served when they should be entering the number of multi-family households served (i.e. one complex may have six units or households).

If the number of multi-family households served is abnormally low (i.e. single or double digits), make a note to contact the MMP member for clarification.

  • If the MMP member provides reasonable justification for the originally submitted number, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
  • If the number needs to be changed (e.g. MMP member entered the number of multi-family complexes instead of households), enter the correct number directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.
  • If the MMP member cannot provide an exact or accounted number, ask them to provide their best estimate. Enter the estimated number directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes; especially the fact that this number is an estimate!
Multi-family households served

Can you provide 2019 tonnage information specifically for your municipality's multi-family recycling program?

If the MMP member selects "Yes, but multi-family tonnage is combined with other programs (e.g. commercial collection program, single-family curbside recycling collection, etc.)", check the response to the follow-up question "Which other programs are included with your multi-family recycling tonnage data?" and then check for the percentage estimate the member has applied under "Estimated % multi-family recycling"

Directly below the tonnage questions, you will see a field displaying the Total Lbs/multifamily. This field is only visible to verifiers and it's a helpful calculation to check tonnages. Excessively high (600 lbs+) or low (100 lbs) is a signal to contact the member to check on the tonnage and household figures being used for this automatic calculation.

  • It's important to document the conversation and outcomes in the Review Notes; especially the fact that these numbers are estimates!
  • If the MMP member cannot provide a reasonable estimate, make a note in the Review Notes. You can consider this data approved for verification by The Recycling Partnership.
Multi-family Tonnage

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following question is classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

Is there any additional information you'd like to share about the way multi-family recycling collection is provided in your municipality? Tell us your story! Please provide as much detail as possible to explain your answer.

Additional-information-about-multi-family-curbside-recycling

Note: The Recycling Partnership will review page 5 (About Your Residential Organics Collection Program) but will not be focused on verification. GDMP members that rely on or otherwise use the data collected on page 5, should pay particularly close attention to their review process.

Some of the same approaches used to review the previous pages may be relevant here, too. For example, if a member selects multiple containers, it may be worth checking to see if that was the intended answer.

PAGE 5: ABOUT YOUR RESIDENTIAL ORGANICS COLLECTION PROGRAM

Are the following types of material types accepted in your municipality's curbside organics collection program?

If the MMP member selects "Other organics", a text box will be displayed with the following question: What other types of ORGANICS are accepted in your municipality's curbside organics collection program? Are there any exceptions? Tell us your story!

Check to see if the answer provided in this text box is already listed as material in the list above.

  • If the option is available in the list of organics types, click "YES" for that organics type and delete any reference to it in the text box.
  • If the answer appears to be problematic, make a note to contact the MMP member for confirmation or clarification. Once the correct organics types have been confirmed, enter them directly into the survey response and click "Save". Remember to document the conversation and outcomes in the Review Notes.

Example: An MMP member selects "YES" for “Other organics” and writes "We accept stumps, grass, and leaves". These options would be covered by "Yard trimmings (e.g. grass clippings, leaves, bundled tree trimmings)" so the comment should be deleted, and "YES" should be selected for "Yard trimmings (e.g. grass clippings, leaves, bundled tree trimmings)"

Organics Selection
Organics selection wrong

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following question is classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

Is there any additional information you'd like to share about the way organics collection is provided in your municipality? Tell us your story! Please provide as much detail as possible to explain your answer.

Additional-information-about-the-way-organics-are-collected

RESIDENTIAL DROP-OFF PROGRAM ASSESSMENT SURVEY

This survey is designed to collect information about the material drop-off services each municipality offers to residents. Only information about drop-off locations managed directly by the MMP member should be included. MMP members should not include information about drop-off locations managed by a 3rd party.

NOTE: If a member does not have a drop-off program, they will answer “No” to the first question in the survey. If GDMP verifiers are aware that indeed the municipal member does have drop-off services, they should contact the member to mark the answer as “Yes” and then fill out the additional questions in the survey. If the answer “No” is accurate, fill in the information in the Audit Only Manager Team Review and Sign-Off window and mark “Yes, ready to verify.”  Again, GDPM verifiers should NOT fill in any data in the Manage Response window.

Approximately how many residents have access to drop-off sites operated by your municipality?

This question is asking for the population and not household numbers. The answer may exceed the specific population figure listed on U.S. Census Bureau Data for legitimate reasons, including that the drop-off service is open to use by county residents or residents of other municipalities. It is recommended that GDMP reviewers check this issue if the population is excessive to understand more about the program. Enter relevant notes in Review Notes section of the Manager Team Review & Sign-Off section.

Population served by drop-off program

DO NOT CHANGE ANSWERS TO OPINION-BASED QUESTIONS

The following questions are classified as "opinion-based" and therefore, GDMP Program Managers are not permitted to make changes to the answers MMP members provide. However, the answer may provide some data that is useful for the verification process so it should be reviewed.

Would you be interested in learning about how you can create free listings for your municipal drop-off locations (and more!) on RecycleSearch.com?

RecycleSearch-Listings

Is there any additional information you'd like to share with us about how your municipality delivers drop-off programs to residents? Tell us your story!

Additional-information-about-drop-off-programs

THANK YOU!

Your assistance/participation in the verification process is much appreciated.

The work you do is vital to ensuring the MMP is collecting good quality data! Questions about data verification can be directed to Scott Mouw at The Recycling Partnership (smouw@recyclingpartnership.org )