How do we know what works?

Photo credit: Humboldt Gardens, source: Home Forward

by Lisa K. Bates

When thinking about assessing the impact of Humboldt Gardens’ GOALS program, which is the project‘s version of HUD‘s Family Self-Sufficiency Program (FSS), it is useful to know the program‘s context. The concept of FSS is straightforward — parents participate in programming designed to promote employment and financial stability, working with a case manager to set goals. During the five-year program, any additional rent the participants would be paying due to increased earnings are placed in an Individual Development Account as savings, which are accessed upon graduation.[1] Folks living in public housing or using Housing Choice Vouchers may be able to participate in FSS.

Humboldt Gardens, source: WALSH Construction

In the early 2000s, researchers from the Center for Budget and Policy Priorities who evaluated FSS titled their report “HUD”s Best Kept Secret for Promoting Income and Asset Growth.”[2]  Despite that rosy title, results for program participants over time suggest a more nuanced reality — while those who graduate seem to do well with employment and earnings, many don’t complete the program. Moreover, differences in outcomes for participants compared to non-participants are not always statistically significant.

These findings and varied program designs in the field make it complicated to translate learning from experiences into implementation. The many optional components of the program, plus partnerships that a local housing authority might develop, mean that there are a huge number of permutations for how FSS can operate. For example, FSS might include job skills training with or without “soft skills;” could address the entire family unit or just one parent as the participant; or could focus on financial literacy and empowerment or self-esteem.[3] Complexity is further introduced because housing authorities can opt for different approaches to case management, such as how check-ins occur.

For a local housing authority trying to determine how to design and implement FSS, translating research studies into practice can thus be challenging. Studies that employ traditional research designs— such as nationally drawn random samples, pre-and post-tests, and control group comparisons— don’t provide detailed information about which elements of a program are most important, and how the local context and individual family challenges interact with program choices and implementation.

As we examine how residents at Humboldt Gardens make choices about participating in GOALS (the local FSS program) and other resident services and what their experiences, successes, and challenges are, we hope to provide the kind of information that will better illuminate how the program really works for people. For instance, we’re hearing about how residents perceive and respond to different approaches to case management. Some program staff have acted as coaches, others as connectors into a network of ‘DIY’ services, and still others as accountability checkers. The relationship with the case manager, and fit with the participants’ needs, seems to be critical for staying engaged and on track. For some residents, Home Forward programs may be only part of the picture of their work with case managers and social services; they may be also working with school-based programs or seeking culturally-specific agencies. Coordinating goal-setting and tracking in multiple arenas could be enhancing to self-sufficiency progress, or could be overwhelming. Finally, we’re aware of how life’s unanticipated challenges may get in the way of participation and progress, despite intentions on the residents’ part and careful program design on the housing authority’s part. What can we learn about how families bounce back and get on track after a setback that could inform the FSS model?

The interplay of program features, case management styles, and the complications of life for low-income families are complex and explain why research results on FSS programs are so mixed. From our interviews with residents and staff at Home Forward and its partner agencies, we can recognize that there isn’t one FSS program that works the same way for everyone. A clear-cut conclusion on “what works” is further complicated by the change from mandatory to voluntary participation, staffing changes, and the evolution of the community-based organization landscape in a gentrifying neighborhood. However, through our approach of looking at program implementation, context, and participants’ agency, my team is aiming to provide practitioners with some actionable knowledge. This will include findings about which elements might be most important for success, and guidance about how to think through adjustments to program features or approaches based on context and participant feedback.

I am hopeful that this knowledge will help future iterations of FSS implemented by housing authorities across the country be more effective, helping more people to set and achieve their goals for stability and success.