CBMS Resources

Chapter 5

Map Data Processing

Contents


Ensuring the quality and correctness of geotagged data is an essential part of the Community-Based Monitoring System operations. This data will be used to navigate and identify which households within a community are in need the most of the programs that will be developed by our government. Hence, the Data Processing System (DPS) application was developed to make sure that the accuracy and quality of data will be checked and corrected before sharing the gathered information to our community and stakeholders.

placeholder

The system contains several modules and functionalities that will assist the data processors and map data processors in determining and correcting the possible errors, anomalies, and inconsistencies in the data that was collected during the CBMS enumeration phase. In addition, it has modules that allow users to download geotagged outputs or shapefiles from the server. The system also has an uploading functionality that will be used once the validation of these shapefiles is completed.


5.1 Map Data Processing Workflow

placeholder

5.1.1 Provincial Statistical Office

The following map data processing activities will be conducted in the PSO:

  1. Digitization of paper maps, if paper maps were used during geotagging of household buildings;
  2. Map data validation of shapefiles using QGIS;
  3. ID validation of data file using DPS;
  4. Consistency check of data file and shapefile using DPS;
  5. Merging of processed shapefiles to barangay level using QGIS; and
  6. Submission and uploading of barangay level shapefiles via DPS.

5.1.2 Regional Statistical Service Office

After uploading of barangay level shapefiles by the provincial map focal, the regional map focal will download the shapefiles using the MIS and will do the following activities:

  1. Validate and check the submitted barangay level shapefiles using QGIS;
  2. If the barangay level shapefiles have passed the validation procedure, the regional map focal will merge the barangay level shapefiles into city/municipal level; and
  3. Submit and upload the city/municipality level shapefiles to the assigned cloud storage.

5.1.3 Central Office

The Geospatial Management Division of the CBSS will manage the post-map data processing of the submitted city/municipality level shapefiles by doing the following activities:

  1. Verify and validate shapefiles using QGIS;
  2. Check consistencies of BSNs and X,Y coordinates of shapefiles and data file;
  3. Adjust or update boundaries, if necessary; and
  4. Generate map data processing report.

5.2 Preliminary Map Data Processing Procedures

The main objective of the map data processing is to give precise and correct location data on the CBMS Form 2 data file. Before proceeding to the map data processing, below are the preliminary activities to be conducted by the following personnel:

5.2.1 CBMS Focal / Map Focal / ISA / SS II

  1. Check the local server and all workstations have the same local network;
  2. Ensure that all workstations are registered in the MIS;
  3. Verify all processors have user accounts to be used in the DPS;
  4. Provide the Form 1 - Listing Record and other necessary forms to MDP; and
  5. Ensure that the shapefile to be processed is not corrupted.
    An error message of “Shapefile: HPQppmmbbbeeeeee is corrupted” will be prompted if the shapefile of selected EA to be downloaded is corrupted.

    pp - province code, mm - municipality code, bbb - barangay code, eeeeee - enumeration area
    Reminder
  6. If the downloaded shapefile is corrupted or cannot be uploaded using the CBMS mobile application, manually import or put the shapefile in the workstation of MDP.
    • Connect the tablet which contains the shapefile of assigned EA to the workstation of MDP.
    • Create a new folder under the following directory:

      C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES
    • Rename the newly created folder to HPQppmmbbbeeeeee.
    • From the workstation of MDP, go to the Internal Storage\PSA-GIS\ppmmbbbeeeeee and copy the following files to the HPQppmmbbbeeeeee folder in C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES:
      • ppmmbbbeeeeee.cpg
      • ppmmbbbeeeeee.dbf
      • ppmmbbbeeeeee.prj
      • ppmmbbbeeeeee.shp
      • ppmmbbbeeeeee.shx
      placeholder
  7. Make sure that all applications listed below are installed in the server and workstations of MDP and DP.
    Workstations of MDP and DPLocal Server
    Java(TM) Platform SE Binary version 8.341 Java(TM) Platform SE Binary version 8.341
    Visual C++ Redistributables Visual C++ Redistributables
    CSPro 7.7.3 CSPro 7.7.3
    QGIS 3.22 R
    CBMS Data Processing System - Client RStudio IDE
    IP Messenger (optional) CBMS Data Processing System - Server
    RCBMS

5.2.2 Map Focal

  1. Ensure that the following files to be used in the map data validation and scrutinization of geotagged buildings are in the workstations of the Map Data Processors:
    • ppmmbbbeeeeee.gpkg - satellite image used in the preparation of QField package projects for CBMS Form 2
      • Place the gpkg file inside the directory:

        C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\QFieldPackage\*EnumerationArea*\
      • To automatically copy all gpkg files to the abovementioned directory, run the following command in the command prompt:
      placeholder
    • ppmm_ea2022.shp - EA boundary shapefile used in the preparation of QField package projects for CBMS Form 2.
    • ppmm_bldg_point.shp - Building points shapefile used in the preparation of QField package projects for CBMS Form 2.
      Place both ppmm_ea2022.shp and ppmm_bldg_point.shp inside below directory:

      C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\*CITY/MUN_NAME*
      Reminder
  2. Verify that the following plugins, modelers and layer styles are added in the QGIS.
User TypeLocal ServerPluginsLayer Style
Map Data Processor 1. CBMS Form 2 Error Reports [PSO]
  • Go2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer Saver
 CBMSHPQDPSStyle.qmlCBMSHPQDPSStyle.qml
CBMS Provincial Map Focal / CBMS Provincial Assistant Map Focal
  1. CBMS Form 2 Error Reports [PSO]
  2. CBMS Check_Overlap_Query [PSO]
  3. CBMS EA_Merge_to_Brgy [PSO
  • Go2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer SaverGo2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer Saver
 CBMSHPQDPSStyle.qmlCBMSHPQDPSStyle.qml
CBMS Regional Map Focal 4. CBMS Brgy_Merge_to_City/Mun [RSSO]
  • Go2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer SaverGo2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer Saver
 CBMSHPQDPSStyle.qmlCBMSHPQDPSStyle.qml

5.3 Digitization of Paper Maps

The digitization of paper maps will be conducted by the map data processors if paper maps are used in the geotagging of household buildings. The map data processors will utilize QGIS in digitizing paper maps.

5.3.1 Preliminary Activities

The CBMS Focal and or alternates will need to check the completeness of paper maps by arranging and matching the number of paper maps to the total number of sheet numbers, including the EA maps and block maps. In addition, the CBMS Focal and or alternates need to secure a copy of the accomplished CBMS Form 1 - Listing of Household Record.

  1. Copy the Map Legends folder, then paste it inside the Map Legends of C:\PSA-GIS folder.
  2. Copy the PAPI-Shapefile folder, then paste it inside the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\References
  3. Open the QGIS v3.22.x application.
  4. Add the CBMS Load Layer and Style Modeler.
    • In the Menu toolbar, click the Processing menu > Toolbox.
      placeholder
    • The Processing Toolbox panel will appear on the right side. Click Models > Add Model to Toolbox…
      placeholder
    • In the Open Model window, browse and locate the C:\PSA-GIS\Map Legends\CBMS Load Layer Style.model3, then click Open to addin the toolbox.
      placeholder
  5. Enable the Snapping Option
    • In the View menu, hover down to Toolbars and tick the check box beside the Snapping Toolbar.
      placeholder
    • The Snapping Toolbar will appear in the toolbar. Click the 🧲 Magnet icon to activate snapping.
      placeholder

5.3.2 Adding of Layers and Style using the CBMS Load Layer and Style Modeler

In this step the necessary layers and style will be added to the QGIS using a modeler.

  1. In the Processing toolbox, click on the Search box and type CBMS Load layer and Style.
    placeholder
  2. Click the CBMS Load layer and Style to open the modeler.
  3. In the CBMS Load Layer and Style window, browse the following shapefiles to be loaded in the QGIS layers panel.
Shapefile *post delineated shapefilesLocation
ppmmbbbeeeeee.shp C:\PSA-GIS\*PROV_NAME*\CBMS\2022\References\PAPI-shapefile
bgy.shp* C:\PSA-GIS\*PROV_NAME*\PROJECT_NAME\*CITY/MUN_NAME*\shapefiles
ea2019.shp*
block.shp*
road.shp*
bldg_point.shp*
placeholder
  1. Add the pre-generated google earth image.
    • In the Menu toolbar, click the Layer menu and hover over the Add Layer, then click Add Raster Layer…
      placeholder
    • In the Data Source I Raster window, click Browse in the source panel. Browse the file directory of the clip google image used in the generation of QField package projects.

      C:\PSA-GIS\*PROV_NAME*\CBMS\2022\QFieldPackage\FORM 2\ppmmbbbeeeeee
    • Highlight the ppmmbbbeeeeee.gpkg, then click Open.
      placeholder

5.3.3 Tagging of Building Points in QGIS

In this process, the CBMS Form 1, and block maps will be used as references by the MDP in tagging the building points in QGIS.

  1. Refer to the CBMS Form 1 and select a BSN starting from the first entry of the listing record.
  2. Locate the geotagged point of the selected BSN in the block map.
  3. In the Map View panel of QGIS, zoom in to the block that refers to the block map and locate the building of selected BSN.
    placeholder
  4. In the Layers panel, highlight ppmmbbbeeeeee by clicking the layer.
    placeholder
  5. Click Toggle Editing icon in the Menu bar then click the Add Point Feature icon button.
    placeholder
  6. To tag the building of the selected BSN, left click at the center of the roof of the building of the selected BSN in the Map View panel of QGIS.
    placeholder
  7. After clicking on the Map View panel, a Feature Attribute window will be shown. Copy the Region Code, HUSN, HSN and Remarks of the selected BSN from the CBMS Form 1 to the corresponding Region Code, HUSN, HSN and Remarks in the Feature Attributes window then click OK.
    placeholder
  8. Continue digitizing and filling out the attribute field until all BSNs indicated in CBMS form 1 and block maps are tagged in the Map View panel.
  9. Save the edits from time to time by clicking on Save Layer Edits button.
    placeholder

5.3.4 Saving CBMS Form 2 Shapefile

  1. Right-click the ppmmbbbeeeeee layer, hover over Export then click Save Feature As…
    placeholder
  2. In the Save Vector Layer As… window, click Browse.
    placeholder
  3. Browse the file directory to the following location:

    C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES
  4. In the Save Layer As window, click the New folder.
    placeholder
  5. Rename the New Folder with HPQ + geocode (ppmmbbbeeeeee).
  6. Double-click the HPQppmmbbbeeeeee folder.
  7. In the File name, type the geocode (ppmmbbbeeeeee) then click Save.
    placeholder

5.4 Map Data Validation (MDV) using QGIS

The first process in the map data processing of CBMS Form 2 to be conducted by the Map Data Processor (MDP) is the map data validation. This process utilizes QGIS as the primary tool in scrutinizing geotagged outputs.

The following modelers and plugins will be used to scrutinize CBMS Form 2 shapefile:

  • CBMS Form 2 Error Reports [PSO]
  • Go2NextFeature3
  • NumericalDigitize
  • Batch Vector Layer Saver

Additionally, the following must be checked and or corrected during map data validation:

  • Inconsistent Region Codes
  • Duplicate Coordinates
  • Unmatched BSN from the referenced building points used in geotagging

Furthermore, the following reports can be used as reference by the MDP and Map focal:

  • Duplicate HUSNs and or HSNs
  • Overlapped geotagged Form 2 with another EA

The MDP must also check that all geotagged building points are at the center of the roof of the household building.


5.4.1 Preparation of Working Directory

Copy the assigned CBMS Form 2 Shapefile from C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES

placeholder

to C:\PSA-GIS\**PROVINCE_NAME*\CBMS\2022\DPS

placeholder

5.4.2 Adding/Importing of Layers and Style

Step 1: Loading of ppmmbbbeeeeee GPKG Image from QField Package Folder

  1. Open the QGIS v3.22.x application. Click on Layer menu > Add Layer > Add Raster Layer
    placeholder
  2. Browse and locate the ppmmbbbeeeeee.gpkg from the C:\PSA-GIS\* PROVINCE_NAME*\CBMS\2022\QFieldPackage\FORM2\*ppmmbbbeeeeee*, then click Open.
    placeholder
  3. From the Data Source Manager | Raster window, click Add button.
    placeholder
  4. The raster layer has been added to the map viewer.
    placeholder

Step 2: Loading of Auxiliary Layers

  1. In the Menu toolbar, click on the Layer menu > Add Layer > Add Vector Layer
    placeholder
  2. Click Browse and find the location of ppmm_ea2022.shp and ppmm_bldg_point.shp in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\*CITY/MUN_NAME*
    placeholder
  3. Click to highlight the layers of ppmm_ea2022.shp and ppmm_bldg_point.shp, then click Open.
    placeholder
  4. In the Data Source Manager | Vector window, click the Add button.
    placeholder
    placeholder
    The MDP may opt to change the layer style of ppmm_ea2022 layer for better map presentation. Right click on the ppmm_ea2022 layer > Properties > Symbology, then choose the desired symbology.
    Reminder

Step 3: Loading the CBMS Form 2 HPQ Layer (ppmmbbbeeeeee)

  1. Click on the Layer menu, find Add Layer > Add Vector Layer. In the Data Source Manager | Vector, browse on the location of the downloaded CBMS HPQ Form 2 shapefiles.
    placeholder
    placeholder
  2. Browse in C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\DPS\*HPQppmmbbbeeeeee*. Click the ppmmbbbeeeeee.shp > Open > Add to load the shapefile layer.
    placeholder
  3. Click to highlight the layers of ppmm_ea2022.shp and ppmm_bldg_point.shp, then click Open.
    placeholder
  4. In the Data Source Manager | Vector window, click the Add button.
    placeholder
    placeholder
    The MDP may opt to change the layer style of ppmm_ea2022 layer for better map presentation. Right click on the ppmm_ea2022 layer > Properties > Symbology, then choose the desired symbology.
    Reminder

Step 4: Loading of CBMS HPQ DPS Layer Style

  1. In the Layers panel, right-click on the ppmmbbbeeeeee layer and click Properties.
    placeholder
    Alternatively, try double clicking the layer to open its properties.
    Reminder
  2. The Layer Properties will be displayed, click the Style button > Load Style.
    placeholder
  3. In the Database Styles Manager dialog box, browse in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\DPS, and select the CBMSHPQDPSstyle.qml, then click Open.
    placeholder
  4. To load the selected style, click Load Style button > OK.
    placeholder
    placeholder

5.4.3 Generation of CBMS Form 2 Error Reports

  1. In the Processing Toolbox, type GMD in the search box, then click the 1. CBMS Form 2 Error Report [PSO] modeler to open the processing panel.
    placeholder
    CBMS Form 2 Error Report [PSO] Modeler is already added.
    ASSUMPTION
  2. In the 1. CBMS Form 2 Error Report [PSO] panel, provide the necessary layers, then click Run.

    CPH HPQ Form 2: ppmmbbbeeeeee
    EA Layer: ppmm_ea2022
    Bldg_point : ppmm_bldg_point
    placeholder
  3. Temporary layers will be added to your layers panel, as shown below:
    placeholder
    Verify the Attribute Table of generated layers. If the attribute table is empty, remove the layer.
    REMINDER
  4. List of Generated MDV Error Reports:
    1. Inconsistent Region Code - list of geotagged Form 2 not consistent with the respective region code of the workload
    2. Duplicate Coordinates - list of geotagged Form 2 with duplicate X,Y coordinates
    3. Unmatched CBMSHPQ BSN with Bldgpoint BSN - list of geotagged Form 2 that does not match with the referenced building points used in geotagging
    4. Duplicate HSN - list of geotagged Form 2 with duplicate HSNs
    5. Duplicate HUSN - list of geotagged Form 2 with duplicate HUSNs
    6. Geotagged F2 Overlapped with other EA - list of Form 2 geotagged outside the EA boundary
    7. pppmm_bldgpoints with remarks - list of geotagged Form 2 with remarks encoded in QField
    placeholder

5.4.4 Validate CBMS Form 2 Map Data

Step 1: Inconsistent Region Code

  1. Open the attribute table. Select the Duplicate Coordinates layer, then click the Open Attribute Table button.
    placeholder
  2. To check the duplicate coordinates in the map, select the rows of the geotagged Form 2 with duplicate coordinates by clicking the row number and click the Zoom map to the selected rows icon button.
    placeholder
    Use the sort function of attribute table by clicking the X column or Y column to easily see the list of geotagged Form 2 with duplicate coordinates.
    Reminder
  3. The geotagged Form 2 with duplicate coordinates will be zoomed in on the map.
  4. Analyze and scrutinize which geotagged Form 2 will be deleted.
    placeholder
    The MDP may use as reference the Remarks in Form 1 (PAPI) to know which geotagged Form 2 to be deleted.
    Reminder
  5. Delete the duplicate geotagged Form 2.
    • Right-click on the geoID column and select copy cell content of the duplicate geotagged Form 2.
      placeholder
    • Open the attribute table of ppmmbbbeeeeee layer and search for the geotagged Form 2 by clicking Show All Features, then select GEOID Mobile App.
      placeholder
    • Paste the copied geoid in the text box filter then press Enter.
    • Select the row of the geotagged Form 2 to be deleted, then click the Delete selected features button.
      placeholder
      The other four generated reports (Unmatched CBMSHPQ BSN with Bldgpoint BSN, Duplicate HUSN, Duplicate HSN, and pppmm_bldgpoints with remarks) will be used as references in the Completeness and Consistency Check process.
      Reminder

5.4.5 Scrutinization of Geotagged CBMS Form 2

Check if the geotagged points are at the center of the roof of the building.

  1. Open the Go 2 next feature plugin by clicking the Plugins menu then click Go 2 next feature.
    placeholder
  2. The Go2NextFeature panel will be displayed in the lower left pane of the QGIS application.
  3. Supply the following information:

    Layer : ppmmbbbeeeeee
    Attribute : Building Serial Number
    Action : Zoom
    placeholder
  4. Review and assess the position of each geotagged point. If the geotagged point is not positioned at the center of the roof of the building, move the geotagged point.
    placeholder
    To move to the next or previous geotagged point, click the Back or Next button.
    Reminder
  5. Moving of geotagged point
    • Select the ppmmbbbeeeeee layer then click the Vertex Tool button.
      placeholder
    • Click the geotagged point, then click at the center of the roof of the building.
      placeholder
    • Save the edits by clicking the Save Layer Edits button.
      placeholder
  6. Repeat steps a to f until the last geotagged point is reviewed.
    placeholder

5.4.6 Export MDV Error Report

The exported MDV error reports will be used by Map Focal and DP in further checking and reviewing of Form 2.

  1. Click the Vector toolbar > Batch Vector Layer Saver > Batch save of vector layers.
    placeholder
  2. Select the following layers to be exported:
    • Duplicate HSN
    • Duplicate HUSN
    • Geotagged F2 Overlapped with other EA
    • Ppmm_bldgpoints with remarks
    • Unmatched CBMSHPQ BSN with Bldgpoint BSN
    placeholder
  3. For the File Type, select CSV.
  4. For the Destination Folder, browse and locate the following directory then click OK.

    C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES\*HPQppmmbbbeeeeee*
    placeholder
  5. A CSV folder containing MDV Error Reports will be created in the selected destination folder.
    placeholder

5.4.7 Save CBMS Form 2 Layer in the CBMS Local File Folder

  1. Right-click on the ppmmbbbeeeeee layer > Export > Save Feature As…
    placeholder
  2. Browse and locate C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES\*HPQppmmbbbeeeeee*\*ppmmbbbeeeeee.shp*, then click Save button.
    placeholder
    placeholder
  3. Uncheck Add saved file to map, then click OK.
    placeholder
  4. A prompt “The layer already exists. Do you want to overwrite the whole file or append features to the layer?” will appear. Click the Overwrite File button. A notification of successful export will be displayed in the Map Viewer.
    placeholder

5.5 ID Validation (IDV) and Consistency Check (CCK) of CBMS Form 2 Shapefile and Data File

The second process in the map data processing of CBMS Form 2 to be conducted by the MDP is the ID validation. This utilizes the CBMS Data Processing System (DPS) in checking the GeoID of CBMS Form 2 data file.

The ID validation process generates the following reports:

  • List of Duplicate Case IDs
  • List of Cases with Incomplete Geocode Data
  • List of Unused HUSNs
  • List of Unused HSNs
  • List of Cases with Incorrect Geocode (Region code, Province Code, Municipality Code, Barangay Code, and EA Number)

5.5.1 ID Validation Procedure

  1. Double-click the CLIENT to launch the CBMS DPS.
    placeholder
  2. Log in to the CBMS DPS. Type the Username and Password of the MDP, then click LOGIN.
    placeholder
    • The workstation of MDP is already registered to the MIS.
    • The MDP has user account registered to the MIS.
    • The MDP has an assigned workload.
    ASSUMPTIONS
  3. To view the list of assigned workload, click the Map Data Validation menu.
  4. Right-click on the selected EA to be validated and select ID Validation.
    placeholder
  5. The following list of ID Validation Reports will be displayed:
    • List of Duplicate Case IDs

      The list of Duplicate Case ID Report has four (4) columns:
      • Case ID - unique identifier of the downloaded cases
      • Error Message - describes the error displayed
      • Classification - defines whether the error message displayed is error or warning
      • Longitude - refers to the X coordinate of a point.
      • Latitude - refers to the Y coordinate of a point.
      • Action - composed of two (2) action buttons:
        1. Add Remarks and
        2. View Members and Household Address
      placeholder
    • List of Cases with Incomplete Geocode Data

      The list of Cases with Incomplete Geocode Data report has three (3) columns:
      • Classification - defines whether the error message displayed is error or warning
      • Case ID - unique identifier of the downloaded cases
      • Error Message - describes the error displayed.
      placeholder
    • List of Cases without Case ID Label
      placeholder
    • List of Unused HUSNs

      The list of Unused HUSNs report has two (2) columns:
      • Classification - defines whether the error message displayed is error or warning
      • HUSN - unique identifier of the downloaded cases
      placeholder
    • List of Unused HSNs

      The list of Unused HSNs report has two (2) columns:
      • Classification - defines whether the error message displayed is error or warning
      • HSN - unique identifier of the downloaded cases
      placeholder
    • List of Cases with Incorrect Geocode (Reg, Prv, Mun, Bgy, and EA)

      The list of Cases with Incorrect Geocode report has two (2) columns:
      • Classification - defines whether the error message displayed is error or warning
      • Case ID - unique identifier of the downloaded cases
      placeholder
    placeholder
  6. In the List of Duplicate Case IDs, the MDP may view the list of HH members and address to further verify if the case is duplicate by clicking the HH Members icon under the Action column.
    placeholder
    placeholder
    The MDP may sort the list of duplicate case IDs by clicking the column names of the table.
    Reminder
    placeholder
  7. If the MDP verifies that the cases are duplicate, the MDP must put “For verification: Duplicate case” as a remark by clicking the Add Remarks button.
    placeholder
    placeholder
  8. If the MDP verifies that the cases are NOT duplicate, the MDP must update either the BSN, HUSN, and/or HSN.
    ScenarioError MessagePossible Solutions
    Two or more cases have the same set of ID (HSN, HUSN, BSN are the same) with different household heads and members Duplicate ID with different household heads, members, and addresses. Update BSN, HUSN and/or HSN
    1. Refer to the remarks of Form 1 and Shapefile and update the BSN, HUSN and or HSN accordingly.
    2. Check the coordinates of both cases, if the coordinates are the same, update the HSN. Increment the last regular HSN used in Form 1.
    3. If the coordinates are not the same, verify the corresponding BSN of the coordinates in the QGIS.
    Two or more cases have the same regular HUSN but with different BSN Duplicate HUSN but different BSN Refer to the remarks of Form 1 and Shapefile then update the HUSN accordingly
    Two or more cases have the same regular HSN Duplicate HSN Increment the last regular HSN used in Form 1. Use the incremented HSN as the new HSN of one of the cases with duplicate HSNs
    • If there are HUSNs and HSNs skipped during enumeration using CBMS Mobile Application BUT the caseIDs are still UNIQUE, the MDP may opt not to update the HUSN and HSN.
    • Make sure that HUSN and BSN of households within the same housing unit have the same HUSN and BSN.
    • After updating the BSN using DPS, make sure to also update the corresponding geotagged Form 2 in QGIS.
    Reminder

5.5.2 Updating of BSN, HUSN, and HSN using DPS

  1. Click the Map Data Validation menu.
  2. Right-click on the selected EA to be validated and select Completeness/Consistency SHP File.
    placeholder
  3. A prompt shp validation success will appear. Click the OK button to proceed.
    placeholder
  4. In the Consistency Check HTML Viewer, click the Show All Cases button.
    placeholder
  5. The Modify Data File window will be displayed. Under the Action column of the list of cases, click the Edit button.
    placeholder
    To search for a specific case in the list of all cases, use the Search function and paste the geoid of the case
    Reminder
  6. A data entry for updating the BSN, HUSN, and HSN will be shown. Encode the correct BSN, HUSN, and HSN, then click Save Changes button.
    placeholder

5.5.3 Adding of Remarks

After updating the BSN or HUSN or HSN, the old case will be retained in the list of all cases. Make sure to add the following remarks to the old case:

  • If the BSN is updated, add remarks “For Deletion: Updated BSN”.
  • If the HUSN is updated, add remarks “For Deletion: Updated HUSN.
  • If the HSN is updated, add remarks “For Deletion: Updated HSN.
  1. In the ID Validation Check HTML Viewer, click the Show All Cases button.
    placeholder
  2. The Modify Data File will be displayed. Under the Action column of the list of cases, click the Add Remarks button.
    placeholder
  3. A data entry for adding remarks will be shown. Encode the correct BSN, HUSN, and HSN, then click Save Changes.
    placeholder

5.5.4 Consistency Check of CBMS Form 2 Shapefiles and Data File using DPS by Map Data Processor

Consistency Check is the third process in the map data processing of Form 2. This process employs the CBMS DPS in checking the consistency of GeoID and location data of CBMS Form 2 data file and shapefile.

The following reports will be generated in this process:

  • List of Geotagged Building Points and Form 2 with Inconsistent Coordinates (X,Y)
  • List of Geotagged Building Points without Form 2
  • List of Form 2 without Geotagged Building Points
  • List of Same Geopoints but Different BSN in Data File
  • List of Same BSN but Different Geopoints in Data File

Step 1: List of Geotagged Building Points and Form 2 with Inconsistent Coordinates (X,Y)

Case 1: The MDP may refer to the remarks “MLOC” in the shapefiles or in Form 1, if there is any. The remarks “MLOC” means that the EN moves the location of geopoints after adding Form 2 during field enumeration. In this case, the coordinates in the shapefile should prevail.

Case 2: If there is no other information on the remarks field of shapefile or in Form 1 on the reason for inconsistent coordinates, the coordinates in shapefile should prevail.

Case 3: If there are remarks stating that the coordinates in the data file are the correct coordinates, the MDP should update the coordinates of the shapefile using QGIS.


Cases 1 and 2:

To update / copy / match the X,Y coordinates of data file from shapefile, click Update Coordinates in the Data File button under the Action column.

placeholder

Case 3:

  • Copy the Geocode of the geotagged building point and Form 2 with Inconsistent Coordinates (X,Y)
    placeholder
  • Go back to QGIS. In the Layers Panel, click the geotagged CBMS Form 2 (ppmmbbbeeeeee) and open the attribute table.
    placeholder
  • Paste the geocode in the filter text box and press Enter to filter the table.
    placeholder
  • Go back to DPS and copy the Longitude and Latitude of the Data File.
    placeholder
  • In the filtered attribute table of ppmmbbbeeeeee layer, click the Toggle editing mode then update the Longitude and Latitude.
    placeholder
    Do not forget to save the edits made by clicking the Save Edits.
    Reminder

Step 2: List of Geotagged Building Points without Form 2

Check the BSN in Form 1 - Listing Record. If the BSN is not listed, the corresponding geotagged building points without Form 2 will be deleted using the QGIS.

  • Copy the GEOCODE of the geotagged building points without Form 2.
    placeholder
  • Go back to QGIS and open the attribute table of ppmmbbbeeeeee layer.
    placeholder
  • Click the Show All Features > Field Filter > Output FileName and paste the copied geocode.
    placeholder
  • Click the Toggle editing mode button, select the row of geotagged Form 2 to be deleted then click the Delete selected features button.
    placeholder

Step 3: List of Form 2 without Geotagged Building Points

The list of Form 2 without geotagged building points will be tagged in the QGIS. The longitude and latitude recorded in Form 2 data file will be used as basis in tagging the list of Form 2 without geotagged building points.

  • Copy the Longitude and Latitude of the geotagged building point without Form 2.
    placeholder
  • Go back to QGIS. In the Layers Panel, click the geotagged CBMS Form 2 (ppmmbbbeeeeee), then click the Toggle Editing button.
    placeholder
  • Click the Vector Menu > Numerical Digitize > Numerical digitize.
    placeholder
  • Encode the copied coordinates (X,Y) from DPS to the X, Y text fields in the Add Feature Panel of QGIS, then click OK.
    placeholder
  • Refer to Form 1 - Listing Record and use the geocode in encoding the necessary field items then click OK.
    placeholder

Step 4: List of Same Geopoints but Different BSN in Data File

The list of same geopoints having different BSNs must be checked in the QGIS. This is to verify which BSN is correct.

Copy the GEOCODE and search the geocode in the attribute table of ppmmbbbeeeeee layer in the QGIS.

placeholder

If the coordinates (X,Y) is the same in the ppmmbbbeeeeee layer in the QGIS, retain the BSN and update the BSN of the other geotagged Form 2.

Reminder

Step 5: Updating of BSN, HUSN, and HSN

  1. In the Consistency Check HTML Viewer, click the Show All Cases button.
    placeholder
  2. The Modify Data File will be displayed. Under the Action column of the list of cases, click the Edit button.
    placeholder
  3. A data entry for updating the BSN, HUSN, and HSN will be shown. Encode the correct BSN, HUSN, and HSN, then click Save Changes.
    placeholder

Step 6: Adding of Remarks

  1. In the Consistency Check HTML Viewer, click the Show All Cases button.
    placeholder
  2. The Modify Data File will be displayed. Under the Action column of the list of cases, click the Add Remarks button.
    placeholder
  3. A data entry for adding remarks will be shown. Encode the correct BSN, HUSN and HSN then click Save Changes.
    placeholder

5.5.5 Consistency Check of CBMS Form 2 Shapefiles and Data File using DPS by Map Focal

The Map focal shall also conduct further consistency checking of GeoID and location data of CBMS Form 2 data file and shapefile using the DPS. This process shall be done after the submission of processed shapefile and data file by the MDP. If the GeoID of submitted shapefile and data file deemed inconsistent, the Map Focal shall return the EA to the MDP.

Step 1: Completeness/Consistency Check of Shapefile

  1. Double-click the CLIENT to launch the CBMS DPS and log in.
    placeholder
  2. Type the Username and Password of the Map Focal, then click LOGIN.
    placeholder
  3. To view the list of submitted workload, click the Data Validation menu.
  4. Go to the Consistency Check tab, right-click on the selected EA, and click Completeness/Consistency SHP File.
    placeholder
  5. A confirmation message Confirm Self Assignment? will prompt, click Yes.
    placeholder
  6. The following Error Lists for Map Consistency Check Module will be displayed:
    • List of Geotagged Building Points and Form 2 with Inconsistent Coordinates (X,Y)
    • List of Geotagged Building Points without Form 2
    • List of Form 2 without Geotagged Building Points
    • List of same Geopoints but different BSN in Data File
    • List of same BSN but different Geopoints in Data File
    placeholder
  7. The submitted workload will be moved on the Assigned EAs tab.
    placeholder

Step 2: ID Validation

  1. For ID Validation, right-click on the EA and click ID Validation.
    placeholder
  2. The Error Lists for ID Validation HTML Viewer will be displayed. Scrutinize the list of errors displayed. If the error displayed is minimal, the Map Focal may opt to do the necessary corrections, otherwise return or reassign the EA to MDP.
    placeholder
    If the error list has content, return/reassign the EA to the MDP using DPS.
    Reminder

Step 3: Reassigning/Returning of Submitted EA to MDP

  1. To reassign EA, click Submit.
    placeholder
  2. Go to the Workload Management menu > Processed tab. Right-click on the workload and click the Accept EA.
    placeholder
  3. A dialog box will appear reflecting the details of the EA, click the SAVE button.
    placeholder
  4. Two dialog boxes will appear confirming the successful return of the workload, click the OK button for both boxes.
    placeholder
  5. Go back to the Workload Assignment menu. Filter the Municipality to easily check the EA. Under Receipt and Control tab, select and right-click on the EA, then click Reassign EA.
    placeholder
  6. A dialog box will appear reflecting the details of the EA to be returned. Select the MDP in the dropdown list, and click the SAVE button.
    placeholder
  7. Two dialog boxes will appear confirming the successful return of the workload, click the OK button for both boxes.
    placeholder

5.6 Merging and Submission of Shapefiles

5.6.1 Checking of Overlapped Geotagged Points from other EA

  1. Download all the EA shapefiles in the Map Data Validation module of DPS.
  2. Open the QGIS v3.22.x application. Click on Layer menu > Add Layer > Add Vector Layer.
    placeholder
  3. Browse and locate the ppmmbbbeeeeee.shp from the C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES\*HPQppmmbbbeeeeee*, then click Open.
    placeholder
  4. From the Data Source Manager | Vector window, click Add button.
    placeholder
  5. The vector layer has been added to the map viewer.
    placeholder
  6. In the Menu toolbar, click the Processing menu > Toolbox.
    placeholder
  7. The Processing Toolbox panel will appear on the right side. Type GMD in the search box, then choose 2. CBMS Check_Overlap_Query[PFP] modeler to open the processing panel.
    placeholder
  8. In the 2. CBMS Check_Overlap_Query[PFP] panel, ensure that the selected shapefile under CBMS HPQ Shapefile is the ppmmbbbeeeeee layer which was previously loaded. For EA2022, browse and locate the ppmm_ea2019.shp in C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\*CITY/MUN_NAME*. Then click Run.
    placeholder
  9. A new layer Overlap_bldgs_to_another_EA will be added to the layers panel as shown below:
    placeholder
  10. Open the attribute table and check the features. Remove the layer after reviewing.
    placeholder
  11. Repeat the process until all the EAs in each Barangay are reviewed.
    If the generated layer has content, reassign the EA to the MDP.
    Reminder

5.6.2 Merging and Submission of Shapefiles to Barangay Level [PSO]

Step 1: Downloading of EA shapefiles from DPS

  • Log in to the CBMS DPS and click the Data Validation module.
    placeholder
  • In the Data Validation module, click the Barangay Level tab. The list of barangays will be shown. To download the shapefiles of all processed and submitted shapefiles per barangay, right click on the selected barangay and click Get shapefile per bgy.
    placeholder
  • The downloaded shapefiles will be saved in the following directory: C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES

Step 2: Merging of EA shapefiles per Barangay using QGIS

  1. Open the QGIS v3.22.x application. Click on Layer menu > Add Layer > Add Vector Layer.
    placeholder
  2. Browse and locate all the EA shapefiles of the Barangay from the directory C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES, then click Open.
    placeholder
  3. In the Menu toolbar, click the Processing menu > Toolbox. The Processing Toolbox will appear on the right side. Type GMD in the search box, then choose 3. CBMS EA_Merge_to_Bgy[PSO].
    placeholder
  4. In the 3. CBMS EA_Merge_to_Bgy[PSO] panel, click the elipsis button.
    placeholder
    placeholder
  5. A new temporary layer will be added in the layers panel. Right-click on the ppmmbbb_bldgpoint_CBMSF2(Bgy) > Export > Save Feature As… to export the shapefile.
    placeholder
  6. In the Save Vector Layer as… dialog box, click Browse and find in the directory C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022 folder.
    placeholder
  7. Create a folder and name it as HPQmergebgy, then click Open.
    placeholder
  8. In the File name:, type in the ppmmbbb_bldgpoint_CBMSF2(Bgy), where pp = Province code, mm = City/Municipality code, and bbb = Barangay code. Then click the Save button.
    placeholder

Step 3: Uploading of Barangay Shapefiles to Cloud Storage

  1. Go to the PhilSys Cloud Storage (https://cloud.philsys.gov.ph) dedicated for CBMS operations. Type in the user credentials provided per province.
    placeholder
  2. Open the folder DPS[PSO], then *MUN*. Click the plus '+' button, then choose Upload File.
    placeholder
  3. In the Open dialog box, browse and locate the HPQmergebgy folder in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022. Highlight all file types with the same geocode ppmmbbb_bldgpoint_CBMSF2(Bgy), then click Open.
    placeholder
  4. New files will be uploaded in the DPS[PSO]/*MUN* cloud storage.
    placeholder

5.6.3 Merging and Submission of Shapefiles to City/Mun Level [RSSO]

The Regional Map Focal shall also conduct further consistency checking of GeoID and location data of CBMS Form 2 data file and shapefile using the DPS. This process shall be done after the submission of processed shapefile and data file by the MDP. If the GeoID of submitted shapefile and data file deemed inconsistent, the Map Focal shall return the EA to the MDP.

Step 1: Downloading and Merging of Barangay Shapefiles

  1. Go to the PhilSys Cloud Storage (https://cloud.philsys.gov.ph) dedicated for CBMS operations. Type in the user credentials provided per province.
    placeholder
  2. Open the folder DPS[PSO], then *MUN*. Select the tick box to highlight all the files, then click Actions > Download.
    placeholder
    The file will be downloaded automatically in the Downloads folder if this is the default folder location of downloaded files.
    Reminder
  3. Extract the file in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\HPQmergebgy folder.
    placeholder
  4. Open the QGIS v3.22.x application. In the Menu toolbar, click the Processing menu > Toolbox.
    placeholder
  5. The Processing Toolbox panel will appear on the right side. Type CityMun in the search box, then choose 4. CBMS Bgy_Merge_to_CityMun[RSSO] modeler.
    placeholder
  6. In the 4. CBMS Bgy_Merge_to_CityMun[RSSO] panel, click Browse > Add File(s)... and select the barangay shapefiles.
    placeholder
    placeholder
  7. Browse and locate the shapefiles in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022\HPQmergebgy\*CITY/MUN_NAME*. Click the dropdown beside Type column, then select SHP File.
    placeholder
  8. Select all the SHP File, then click Open.
    placeholder
  9. Ensure that the selected shapefiles have been added. Click Run.
    placeholder
  10. A new temporary layer will be added to the layers panel. Right-click on the ppmm_bldgpoint_CBMSF2(Mun) > Export > Save Feature As… to export the shapefile.
    placeholder
  11. In the Save Vector Layer as… dialog box, click Browse and find the directory C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022 folder.
    placeholder
  12. Create a folder and name it as HPQmergeMun, then click Open.
    placeholder
    placeholder
  13. In the File name:, type in the ppmm_bldgpoint_CBMSF2(Mun), where pp = Province code and mm = City/Municipality code. Then click the Save button.
    placeholder

Step 2: Uploading of City/Municipality Shapefiles

  1. Go to the PhilSys Cloud Storage (https://cloud.philsys.gov.ph) dedicated for CBMS operations. Type in the user credentials provided per province.
    placeholder
  2. Open the folder DPS[RSSO] folder, click the '+' button, and choose Upload File.
    placeholder
  3. In the Open dialog box, browse and locate the HPQmergeMun folder in the C:\PSA-GIS\*PROVINCE_NAME*\CBMS\2022. Highlight all file types with the same geocode ppmm_bldgpoint_CBMSF2(Mun), then click Open.
    placeholder
  4. New files will be uploaded in the DPS[RSSO] cloud storage.
    placeholder

5.6.4 Completeness and Consistency Check Report on Form 2 Shapefiles and Data File

The Completeness and Consistency Check Report is generated upon running the Completeness/Consistency Check function of Map Data Validation module. The report shows the consistency in counts of BSNs in shapefile and data file. The report can be located on the following directory:

C:\CBMS-LOCAL-FILES\FORM2SHP\EXCEL

Copy the ppmmbbbeeeeee.xlsx accomplishment report from the above directory

placeholder
to C:\CBMS-LOCAL-FILES\FORM2SHP\SHPFILES\*HPQppmmbbbeeeeee* folder
placeholder

The Completeness and Consistency Check Report has 17 columns:

  1. Region - region name
  2. Region Code - PSGC of the region
  3. Province - province name
  4. Province Code - PSGC of the province
  5. City/Municipality - city/municipality name
  6. City/Municipality Code - PSGC of the city/municipality
  7. Barangay - barangay name
  8. Barangay code - PSGC of the barangay
  9. EA - enumeration area number
  10. Total Count of Unique BSN in the Data File
  11. Total Count of Unique Special BSNs (4000 series and 6000 series) in Data File
  12. Total Count of Special BSNs (5555) with unique X,Y Coordinates in Data File
  13. Total Count of BSNs in CaseIDs (Col 10 + Col 11 + Col 12)
  14. With Uploaded Shapefile (1 = Yes, 0 = No)
  15. Total Count of BSNs in Shapefile
  16. Total Count of Unmatched BSNs in Shapefile and Data File
  17. Remarks
placeholder
Last Updated: