Browsing by Subject "Cross validation"

Now showing 1 - 1 of 1

Improving accuracy, stability, and practicability of poverty-targeting tools in the context of vietnamese ethnic minorities
(2025) Duong, Be Thanh; Zeller, Manfred
Poverty-targeting tools (PTTs) are essential tools used globally to identify households below established poverty benchmarks, such as food consumption or monetary standards, through simple and verifiable indicators. These tools enable social programs, ranging from cash transfers to healthcare and educational supports, to effectively and quickly deliver urgent support to the households most in need. However, despite the significant improvements made over time, current PTTs face some critical limitations that undermine their accuracy performance in both laboratory and field conditions. First, current efforts to enhance PTT accuracy through algorithmic improvements appear to be reaching a plateau. The Weight of Evidence (WOE) transformation technique has demonstrated success in optimizing the predictive power of input variables and potentially improving the accuracy of the prediction model across other domains, but this methodology remains underexplored in the poverty targeting field. Additionally, most existing PTTs lack transparent and user-friendly scoring systems. The score-scaling technique, doubling-the-odds, commonly paired with WOE in credit scoring, provides a logical and interpretable relationship between scores and outcome probabilities, yet it is rarely applied to the development of PTTs. Second, although the performance stability of PTTs is critical for ensuring their reliability and robustness in different datasets and real-world applications, this aspect remains insufficiently addressed in current research. Most existing studies rely on hold-out validation, which assesses the model’s performance based on a single random draw of a training-testing split. This approach offers limited insight into performance variability and restricts developers from making timely model adjustments. While k-fold cross-validation, a technique widely adopted in machine learning, offers a more comprehensive assessment of model stability and facilitates the fine-tuning of prediction models, this technique remains underutilized in poverty-targeting research. Third, the predictive accuracy of PTTs in the real world is significantly impacted by their practicability, namely the simplicity and verifiability of their indicators, but this aspect is often assumed rather than empirically validated. Currently, developers select indicators based on theoretical assumptions about simplicity and verifiability, without systematically validating these assumptions with frontline users. In addition, various contextual factors may influence the practicability of PTTs, ultimately affecting their overall predictive accuracy in practice. Despite this, no studies have thoroughly explored this dimension. To address these critical gaps, this dissertation aims to enhance the predictive performance of PPTs in both controlled research environments and real-world implementation settings. The research focuses on advancing fundamental dimensions of PTT effectiveness: technical accuracy, performance stability, transparency, and practicability. The research is situated in the context of ethnic minority communities in Vietnam, which experience the highest poverty rates and are a major concern for national poverty reduction programs. To achieve this overarching goal, this dissertation pursues the following interconnected sub-objectives: (1) Examine the efficacy of applying the WOE transformation method to improve the accuracy of PPTs; (2) Explore the combination of k-fold cross-validation with the WOE technique to ensure high and consistent performance PTTs with transparent and interpretable scoring systems; (3) Evaluate the practicability (simplicity and verifiability) of the PPTs and identify the key determinants impacting their practicability in field conditions, particularly in the context of Khmer communities in Southern Vietnam. These objectives are addressed through three scientific articles, presented respectively in Chapters 2 through 4 in this dissertation. The first research component of this dissertation is presented in Chapter 2, which examines the adaptation of the WOE technique in constructing high-accuracy PTTs. This research introduces a comprehensive approach, called the WOE (Logit) method, to develop PTTs based on international and national poverty lines. The WOE (Logit) method integrates the WOE technique with other techniques such as logistic regression, the indicator selection approach based on c-statistic value, and a score-scaling technique of doubling the odds. The resulting PTTs are evaluated against well-known methods such as the Simple Poverty Scorecard, Proxy Means Test, and Poverty Assessment Tool. The results show that the WOE (Logit) method yields higher accuracy, specifically improving identification rates by 1.9–5.9 percentage points for households and 1.5–3.3 percentage points for individuals. This finding contributes a valuable alternative methodology for more accurate poverty targeting. Chapter 3 explores the effectiveness of integrating k-fold cross-validation into the WOE (Logit) method in constructing high-accuracy and stable-performance PTTs with transparent scoring mechanisms for ethnic minorities in Vietnam (EMP tools). This research integrates a 5-fold cross-validation technique into the WOE (Logit) method to assess predictive performance and its variation. Two indicator selection strategies are compared based on the results of 5-fold cross-validation to find out the best prediction models, high-predictive power, and low performance variation, to develop EMP tools: one based on the highest c-statistic value (MaxC) and the other on the smallest Akaike Information Criterion (MinAIC). The results indicate that both indicator selection approaches yield high predictive power with minimal differences between them. However, the MinAIC approach shows a significantly lower performance variation in small urban datasets and remains slightly more stable in large rural datasets. Based on these results, MinAIC-based models are selected to design the EMP tools by applying the score-scaling technique of doubling-the-odds. Independent testing further validates this choice: the MinAIC models outperform their MaxC counterparts in terms of predictive power and standard error across rural and urban settings. When tested out-of-sample, EMP tools outperform the Vietnamese government’s current models while using one-third the number of indicators. Crucially, the EMP tools also feature a transparent and user-friendly scoring mechanism, which the government's tools lack. These results underscore the value of incorporating k-fold cross-validation to assess both accuracy and stability during model selection. Moreover, this study demonstrates that this integrated strategy effectively develops transparent PTTs with high accuracy and stable performance. The third article in Chapter 4 evaluates the practicability of the EMP tools developed in Chapter 3 through a field study in Khmer communities in the Vietnamese Mekong Delta. Employing both quantitative and qualitative methods, this study examines enumerators’ perceptions of indicator simplicity and verifiability. Quantitative data are analyzed using Likert scales, Factor Analysis, and Ordinary Least Squares regression, while qualitative data are analyzed using a thematic approach. The findings reveal a complex landscape of practicability that challenges conventional assumptions about indicator simplicity and verifiability. While EMP tools are generally perceived as practical, with over half of enumerators affirming the straightforwardness and verifiability of most indicators, significant variations exist across different types of indicators. Some commonly assumed simple indicators (e.g., cellphone ownership, job type) are rated low in verifiability, while more complex indicators (e.g., land ownership) are seen as more practical than anticipated. Qualitative insights highlight the vital role of local knowledge, particularly from village managers, in verifying high-risk indicators. The characteristics of enumerators (such as ethnicity, age, authority level of enumerators, and prior poverty-targeting experience) are significant determinants of overall tool practicability. These findings contribute a mixed-methods framework for pre-assessing PTT practicability and suggest solutions for improving tool practicability through more targeted enumerator selection. In sum, the dissertation achieves its three sub-objectives by demonstrating that: (1) the WOE (Logit) method enhances the predictive accuracy of PTTs; (2) Incorporating the k-fold cross-validation alongside the WOE (Logit) method reduces performance variation, enabling the development of high-accuracy and stable PTTs with transparent and use-friendly scoring systems; (3) Practical field assessments confirm that while the EMP tools are generally practical, conventional assumptions about indicator simplicity may require re-examination. Furthermore, enumerator characteristics significantly impact the practicability of the tools, suggesting their consideration in enumerator recruitment processes to optimize the tool’s practicability during implementation. This dissertation makes substantial theoretical and empirical contributions to poverty targeting research and practice through three key aspects. First, it provides compelling empirical evidence demonstrating how techniques so far have been underutilized in poverty identification; that is, the WOE technique and the k-fold cross-validation can enhance both the accuracy, transparency, and stability of poverty targeting, while highlighting the critical importance of field-based practicability assessments before implementing PPTs. Second, the research contributes an integrated methodological framework that integrates the WOE (Logit) method with the k-fold cross-validation and a mixed-methods approach of practicability assessment. This framework serves as a robust foundation for developing transparent PTTs that consistently deliver high accuracy and stable performance across diverse contexts. Third, the study not only delivers high-accuracy, stable-performance, and practical PTTs with a transparent and user-friendly scoring mechanism for ethnic minorities in Vietnam, but also offers actionable insights and strategies to maximize these tools’ practicability in real-world settings, ultimately supporting their accuracy performance during implementation. While focused on ethnic minorities in Vietnam, the broader implications of this research extend beyond this context, offering a replicable framework for constructing context-appropriate PTTs that perform accurately and reliably across diverse conditions while remaining practically feasible for frontline implementation.