Using technology to enhance USAID’s Monitoring, Evaluation and Learning activities
How Exinsight can help USAID and its partners with data collection
As per the USAID’s Monitoring Toolkit, we recognise six methods for data collection:
- Recording Data Through Administrative Actions
- Electronic Data Harvesting
- In-depth Interview (IDI)
- Focus Group Discussion (FGD)
"Our system can assist in recording data derived from administrative actions during project delivery by digitising the processes and workflows that emerge from the activities of staff."
Our system is designed to enhance each of those methods, as described below:
Recording Data Through Administrative Actions
Our system can assist in recording data derived from administrative actions during project delivery by digitising the processes and workflows that emerge from the activities of staff.
This includes recording things like work patterns and by georeferencing, timestamping and attributing the activities, to individuals or teams, during the life-cycle of the project. This information and analysis can then be used to drive efficiencies and effectiveness and shared with other stakeholders to demonstrate progress in an accountable manner.
Electronic Data Harvesting
The harvesting of third-party and open source data provides valuable contextual data. We recognise this and have built ways to visualize data inside our platform, alongside the proprietary data sets generated by a project’s stakeholders. We can visualize this data if it is available in a structured format as mapping tiles, heatmaps, tables or graphs once structured appropriately.
Our web and mobile apps make structured collection easy for both qualitative and quantitative survey data. Managers can build surveys, respondents can answer them remotely or with assistance, and other managers can see the aggregated or analysed data. We support multiple question types, languages, and methods- including the Pictorial Evaluation Tool method- useful for conducting surveys where literacy is low.
In-depth Interview (IDI)
Although In-depth Interviews discussions tend to be often qualitative in nature, our software can assist with both collection and analysis. Meta-data, such as the location of the interview can be captured automatically, and the attribute data can be databased for further analysis and knowledge transfer. In the case of an agricultural project, for example, the interviewee might have knowledge that is valuable to other stakeholders. Digitising such information, and making it available in a secure but shareable format, via integration, can help add value at marginal cost. Audio recordings of interviews can also be uploaded into the system for further analysis.
Focus Group Discussion (FGD)
Our software is designed to be usable and rapid-to-deploy. That means that we can support Focus Group Discussions in a way that other systems can’t. A fluid and subjective meeting can be minuted into a structured format which can be created during the course of a meeting. This means that previously unstructured, text-based summaries, can now become machine readable, searchable and linkable via the extraction of meta and attribute data. The results can be interpreted in near real-time via charts and graphs and auditable logs.
Our software is built to aid both Participant Observation and Remote Observation. In the former, researchers can use our phone app to capture location, time-series, attribute or meta-data as well as images, sound and video. In the latter, third party or remote data such as custom satellite imagery and security data can be viewed inside our system, facilitating the overlaying of both participant and observed data sets.
How ExInsight can be used to manage the ART’s Reports and Deliverables
This section highlights some of the areas where ExInsight can be deployed to assist the ART Reports and Deliverables.
ART Report or Deliverable
Description of how ExInsight can help MEL partners
Geospatial tools within ExInsight can be used to present graphics on the geographic nature of the plan.
Monitoring, Evaluation, Research, Analysis and Reporting Plan
Geospatial tools within ExInsight can be used to present graphics on the geographic nature of the plan including the security situation and logistical context.
In terms of the requirement for a Monitoring Plan, ExInsight can be used to assist the following requirements as described by USAID:
- Recording Indicators
- Providing Data Quality Assurance
- Ensuring confidentiality
- Qualitative protocol
- Quantitative protocol
- Integrated Data Management
- Integration of GIS Mapping
- Capacity Building and Training Plan
ExInsight’s survey tools can be used to demonstrate to USAID that the MEL partner has the ability to conduct a survey to ascertain training requirements via the collection of a baseline sample, and progress surveys thereafter.
Weekly Activities Reports
ExInsight can facilitate weekly reporting by structuring and visualizing contextual data, programmatic highlights, financial, procurement and logistics, staffing and travel summaries. Much of this can be automated if a collection plan is in place.
Special and Ad-hoc Reports
The interactive nature of ExInsight lends itself to special or occasional reporting. Data can be queried over space, time and attribute, meaning that a custom report could be generated just for BRICC projects, for example, with short notice, and with no additional support required.
Geospatial tools are built into our software. This includes the ability to import custom mapping tiles via ArcGIS or Mapbox Studio, which means we can produce high-quality, high impact mapping via third party base maps overlaid with proprietary data sets derived from the data of USAID and its Implementing Partners. Features include:
Satellite or Aerial Imagery, both off-the-shelf and third-party if available.
Custom cartography in terms of control over styling of colours and typography via Mapbox Studio
Heat-mapping, Clustering, Hex Binning, Choropleth, Polygon, Line and Point Data
The ability to display spatial, time-series and attributed data on-the-fly
Maps can be delivered in an interactive format, or can be exported to PNG for use in conventional presentations.
Monthly Performance Reports
ExInsight’s conditional formatting lets users set baselines and standards which can then be highlighted against a set of rules. For example, data that is less than a value of X might be coloured red, while data that is equal to or greater than X might be coloured green. The structured nature of ExInsight’s approach lends itself to the reporting of KPIs.
The power of a database is that the data can be queried, which in itself can be automated to present snapshots and analysis of the data. This includes four primary components:
- Graphs and Charts
In each of these cases, the data can be queried by space, time, meta and attribute data.
Monthly atmospheric reports
Contextual data can be visualized on maps to help understand atmospheric data - such as the location of security threats. Conditional formatting can also be used on tabular data to highlight if data is above, on or below a baseline.
Quarterly work plans
The data collection module can be used to record deliverables for each activity and expected outcomes for each activity. Such a format can be used as the reporting standard thereafter.
Quarterly Program Reports
The database can provide the raw and filtered data to help inform the Quarterly Program Reports. The graphical, geospatial and tabular elements can be used to help visualize the data as part of a dynamic report, or as an exported image.
Quarterly Special Assessments/Reports
The database and graphical tools in ExInsight can inform special assessments or reports and can be used to visualize data.
Quarterly Opinion Polling
ExInsight’s data collection tools can record concurrent activities relating to stability, essential services, governance, and livelihoods. Our web-based, Android and iOS forms allow remote or supervised data collection, in the field, and can be conducted in near-real time. Data can be exported for use in SPSS, R, or other systems.
Quarterly Financial Reports
Financial data, such as dollar values can be reported although more specialist accounting software is preferable for this task.
Annual Training and Learning Plans
ExInsight’s data collection tools can be used to record training and learning records in the form of questionnaires that establish baselines and then performance thereafter.
Inventory can be digitised via a web or mobile-based form.
Annual Program Report
The data collected and analyzed using ExInsight can inform and help report annually.
Data Quality Assessments for USAID projects
Data quality is a recurring issue of concern across our customer base. USAID offers useful guidance on how Implementing Partners can help the agency make better decisions by using the highest quality data available.
Our software can help meet and exceed the five data quality standards that all data from performance monitoring indicators must meet.
Data recorded in ExInsight’s system can be made available for future use and can be exported in a machine readable format.
1. Validity - Data clearly and adequately represent the intended result.
Asking quantitative questions in a structured way reduces uncertainty. A common example we see when consulting with clients is the use of free-text, open-ended questions which lead to subjective responses.
An example question might be 'describe the health of livestock'. When asked this way, the response will be varied, difficult to compare, and not always valid.
An iteration of that question might be a true or false type question. For example, 'Are the livestock healthy?' which would be followed by yes or no responses. The answers might still lack validity, though as the question remains subjective.
A more developed question might make use of options, where the respondent is guided to make a choice based on a fixed number of options. A Pictorial Evaluation Tool is a good example of a question set that promotes validity.
2. Integrity - Data have safeguards to minimize the risk of transcription error or data manipulation.
The first step is to move away from paper-based surveys. The next is to use systems that reduce the margin for error. Avoiding free-text responses increases integrity by avoiding mistakes such as spelling errors, mis-use of acronyms, duplication or mistakes in data entry.
Finally, a permission based system means that data cannot be manipulated and changed without record. The geo-tagging and time-stamping of information also adds certainty to who said what, where and when. The ability to audit information via logs is of crucial importance in providing oversight of the data processed in ExInsight.
3. Precision - Data have sufficient level of detail to permit management decision making.
While a question and answer based system is useful, if structured correctly, there are occasions where the information manager needs to know more. In the worst cases, this can mean redeploying a team to a difficult environment to collect more data. Collecting imagery, video, and GPS data are examples where precision can be enhanced via ExInsight, and where the probability of having to re-collect data is reduced. Survey forms can be updated in real-time and distributed via the internet allowing information requirements to be changed or added to.
4. Reliability - Data reflect consistent collection processes and analysis methods over time.
A common question we are asked is 'how do we know if the respondent was there?' The answer is meta-data. ExInsight can record meta-data such as GPS data, the phone's IMEI, time-stamping of responses, and more, with permission. This builds trust in the data collected.
5. Timeliness - Data are available at a useful frequency, are current, and timely enough to influence management decision making.
ExInsight is developed with the latest technology stack to realise the timely transfer of information, even in areas of low or disturbed connectivity. It is a networked, databased system, built with the latest best practice which facilitates and real-time sharing of information.