Stephen W. Raudenbush & Robert J. Sampson
Raudenbush, S. W. & Sampson, R. J. (1). 1. Ecometrics: Toward a Science of Assessing Ecological Settings, with Application to the Systematic Social Observation of Neighborhoods. Sociological Methodology, 29(1), 1–41. http://dx.doi.org/10.1111/0081-1750.00059
This paper considers the quantitative assessment of ecological settings such as neighborhoods and schools. Available administrative data typically provide useful but limited information on such settings. We demonstrate how more complete information can be reliably obtained from surveys and observational studies. Survey-based assessments are constructed by aggregating over multiple item responses of multiple informants within each setting. Item and rater inconsistency produce uncertainty about the setting being assessed, with definite implications for research design. Observation-based assessments also have a multilevel error structure. The paper describes measures constructed from interviews, direct observations, and videotapes of Chicago neighborhoods and illustrates an “ecometric” analysis—a study of bias and random error in neighborhood assessments. Using the observation data as an illustrative example, we present a three-level hierarchical statistical model that identifies sources of error in aggregating across items within face-blocks and in aggregating across face-blocks to larger geographic units such as census tracts. Convergent and divergent validity are evaluated by studying associations between the observational measures and theoretically related measures obtained from the U.S. Census, and a citywide survey of neighborhood residents.
Five scales were created to examine the perceived social and physical properties of Chicago neighborhoods. The five scales were social disorder, perceived violence, social cohesion, social control, and neighborhood decline. An intraneighborhood correlation coefficient (or ICC) was calculated as the ratio of between-neighborhood variance to the sum of between- and within-neighborhood variance. The ICCs ranged from 0.13 to 0.36, for informal social control to social disorder, respectively. These modest ICCs are similar to others found in different studies. The observational strategies used (videotaping and direct observation from a vehicle) show the potential applications of such strategies for the study of neighborhoods. Physical and social disorder behaved differently enough to show how confirmatory factor analysis, generalizability theory, and item response modeling can help better understand the measurement of ecological units.
Description of method used in the article
Chicago’s 865 census tracts were combined into 343 neighborhood clusters (NCs). Twenty to fifty households per NC were selected using a multistage probability sample. The total sample size was 8,782 and the response rate was recorded to be 75%. A randomly chosen adult from each household was interviewed regarding the social relationships and conditions in the surrounding neighborhood. For the systematic social observation, eighty of the 343 NCs were sampled from strata representing socioeconomic status and ethnic mix. Observers drove a car at five miles per hour down each of the streets in 80 sample neighborhood clusters. Cameras attached to the vehicle recorded physical features and social activities on the blocks. Two observers on either side of the vehicle also noted observations on 14 variables, emphasizing land use, traffic, building conditions, and physical disorder. The teams observed 23,816 face-blocks and recorded an average of 298 face-blocks per NC. Video content analysis of a random subsample of 15,141 face-blocks resulted in 126 variables detailing physical conditions, housing characteristics, businesses, and social interactions.
Of some practical use if combined with other research