THE HONG KONG COUNCIL OF SOCIAL SERVICE
Sharing Session on How to Assess Social
Capital
( September 26, 2002)
Evaluating Social Capital
Joe Leung
The University of Hong Kong
1.
Purpose of Evaluation
- accountable for program effectiveness.
- program improvement and learning (formative evaluation)
2.
CIIF and social capital
-
The underlying theme of CIIF is to develop
social capital.
- Social capital is a highly contested, multi-dimensional and ambiguous concept. There are no acceptable and comprehensive definitions. Most projects would prefer their own definitions. Features of organizations. It is about the value of social network, bonding similar people and bridging between diverse people, with norms of reciprocity. Participation in civic organizations, in government, in family, in workplace…. informally or formally. (Trust and participation).
For CIIF:
- Programs that link to the increased of social capital; theory/ strategy/ proposition explaining how social capital can be enhanced.
- what social capital is for? Impact? Problem-solving. Improvement in education, poverty alleviation, equity, economic development, family stability and child development, health, employment, crime, civic participation and drug (almost everything!). Not simply increasing membership in soccer club, and residents’ associations, or building social capital for the sake of social capital).
- who needs social capital?
- What levels? (horizontal – vertical; individual, family, organization, community, society, workplace. Individual – groups; groups – organizations; organizations – organizations; organizations – society/ government)
- What types? What network? Bridging (inter-sectors/ groups) or bonding (intra-groups)?
- How to measure? Can trust be measured directly? Use of empirical scale or qualitative measures.
3. Research
Foci of Social Capital
- Traditional studies on social capital put emphasis on political implications of trust – how voluntary participation can translate into political democracy. Singing choirs or soccer clubs may not lead to good governments. Political implications of cooperative attitudes. Whether social trust can translate into political cooperation? Civil liberties and economic growth, sustainable development. Relationship between independent variables (dimensions of social capital) and dependent variables.
- Others projects showed social capital relate to improvement in poverty alleviation, health, education, etc. Cross-country studies – social trust and civic engagement correlated with the density of associational membership in a society. Most community projects emphasize on community capacity building/ community building.
4.
Difficulties in measuring social
capital
- Measuring is difficult, as definitions vary. There is so far no single correct and precise approach. Even on the same concept, these tools differ substantially.
- Specific dimensions – political trust, economic development, educational achievement, family stability, quality of life, (surveys). Indicators of social capital:
- E.g. Horizontal associations (number and types of associations, membership, participation); civil and political society (civil liberties, corruption, voter turnout)l social integration (suicide rates, homicide rates, crime rates, divorce rates); legal and governance (independence of court, contract enforceability).
- What types of groups/ network that social trust would promote collaboration for mutual benefits (public-regarding networks, not anti-social groups; clearly defined public purposes and values).
- HKFYG: simplified dimensions of social capital to include: volunteering, donations, social participation. Behaviours, not attitudes and relationships.
- CIIF: network, trust, mutual assistance, civil engagement and reciprocity.
- Putnam: networks of civic engagement; civic virtue. Put over-emphasis on civic association participation.
- Difficult to use “experimental design” of pre and post measurement, and control groups. Long-term and short-term results.
· horizontal complexity – each project has unique features
· vertical complexity – individual, family, organizational and community level changes
· contextual issues – socio-economic factors affecting outcomes
· flexible and evolving intervention – program design flexible and subject to improvements
· broad range of outcomes
· absence of comparison groups.
- Would a tight evaluation affect innovations? Balance of accountability and need for providing information for program learning.
- Operators’ responsibility in making self-evaluation and in facilitating external evaluator for assessment. Record and information requirements.
5.
Outcome-based measurement
- The need to develop an information system to monitor programs and enable program learning. Basic information system on program activities and users and perceived impacts.
- Listing out some common outcome indicators (performance indicators) is controversial. What can be considered as successful? Grouping projects according to modes and objectives. Surveys on participants, rating by participants. Qualitative study (focus groups); sharing of good and best practices.
Basic Questions
- What the programs are for (objectives – public purposes, not solely benefiting limited members only)? Addressing what needs or problems? Evaluation of needs (assessment). Program description. Importance and priorities.
- What social capital (types and means/ intra- or inter- group/ capacity, relationship and attitudes) is used to achieve the outcomes (problem-solving, meeting needs, capacity building, multiplier effect, resource generating).
- What are provided?
- What are the results/ outcome?
- Which program produces the best results? Good/ best practices?
- Is the outcome justified the cost? (compare program efficiency)
- How to improve service quality and outcome?
- Any unplanned side effects
- public purposes, not for self-interests (social capital may have negative effects on other people, or benefit individuals only.
- Organization (membership and participants) – relationship change (trust) leading to attainment of objectives. Not only trust among members, but trust across sectors (inter-group trust).
- what for? Why social capital matters? Application of social capital. Better outcomes for what?. Community capacity for collective problem-solving, developing resources (volunteering, financing, action, leadership, utilizing outside assistance, expertise).
- Measurement of social capital (community capacity)
- What account for the building of social capital? (strategy) what types of social capital? – vertical network, horizontal network (preferred), cross-sector (members and participants include other sectors, heterogenous).
Performance
Indicators:
- number of members, frequency of meetings, dimensions of membership (cross sectors).
- Impact: use of resources by the poor; schooling, poverty reduction, health, pooling of risks, quality of education, lower dropout rates.
methods
- Multiple criteria (program focus)
- ratings by users/ participants.
- Random and representative sample of users. surveys
- Focus groups
- Observations
- Case study
- Role of a third party evaluator should not be threatening, but providing consultation.
World Bank Social Capital Assessment Tool:[1]
1. Community profile: integrates
participatory qualitative methods with a community survey instrument to assess
various dimension of community-level social capital, including community assets
identification, collective action, solidarity, conflict resolution, community
governance and decision making, institutional networks, and organizational
density.
-
collect
through interviews with key informers, focus groups
-
history,
location, services, strengths and weakness, formal and informal organizations,
developmental strategies, impacts and results.
-
Community
questionnaires
a) definition of community and identification of community assets
l
how do you
define this neighborhood (boundaries)
l
where are
the schools, elderly centers……
l
What are
the changes in recent years? who move in? who move out?
b) Collective action, solidarity, conflict resolution and
sustainability of efforts
l
Do you
think that everyone in this neighborhood has equal access to
l
Have there
been any efforts by the community to improve the quality of life or overcome a
problem.
l
Has the
neighborhood ever attempted to make improvements but failied.
c) Community governance and decision making
l
who are the
main leaders in this community
l
How do they
become leaders? How are they elected?
l
How are
decisions made within this community? What is the role of community leaders?
d) identification of community institutions
l
what are the
groups in this neighbourhood. Which ones are more active?
e) community-institutional relationship
l
which
organizations are most important? Most accessible?
f) Institutional networks and organizational density?
l
which
organizations work together? How do they work together?
l
Are they
any organizations that work against each other (competition)? Which one and
why?
l
Which
organizations have same or similar memberships
l
Are there
organizations that share resources?
2. Household survey: survey on cognitive
social capital (trust, values, norms, behaviours towards participation),
consisting of 39 items on structural social capital and 21 items on cognitive
social capital.
The questionnaire covers community characteristics (basic interviewees
information), household characteristics, genogram (family tree and household
members), structural social capital (network of support) and cognitive social
capital (trust).
3. Organizational profile: delineate the
relationships and networks that exist among formal and informal institutions,
integrating semi-structured interview data with a scoring system for assessing
organizational capacity and sustainity. relationships and network of formal
organizations/groups (leadership, participation, culture, organizational
capacity and sustainability).
l
semi-structured
interviews with organizational leadership, members and non-members. Assess on
organizational origins and development, quality of membership, institutional
capacity and institutional linkages.