SEARCH THE DATABASE

A number of recent studies have used surveys of neighborhood informants and direct observation of city streets to assess aspects of community life such as collective efficacy, the density of kin networks, and social disorder. Raudenbush and Sampson (1999a) have coined the term “ecometrics” to denote the study of the reliability and validity of such assessments. Random errors of measurement will attenuate the associations between these assessments and key outcomes. To address this problem, some studies have used empirical Bayes methods to reduce such biases, while assuming that neighborhood random effects are statistically independent. In this paper we show that the precision and validity of ecometric measures can be considerably improved by exploiting the spatial dependence of neighborhood social processes within the framework of empirical Bayes shrinkage. We compare three estimators of a neighborhood social process: the ordinary least squares estimator (OLS), an empirical Bayes estimator based on the independence assumption (EBE), and an empirical Bayes estimator that exploits spatial dependence (EBS). Under our model assumptions, EBS performs better than EBE and OLS in terms of expected mean squared error loss. The benefits of EBS relative to EBE and OLS depend on the magnitude of spatial dependence, the degree of neighborhood heterogeneity, as well as neighborhood's sample size. A cross-validation study using the original 1995 data from the Project on Human Development in Chicago Neighborhoods and a replication of that survey in 2002 show that the empirical benefits of EBS approximate those expected under our model assumptions; EBS is more internally consistent and temporally stable and demonstrates higher concurrent and predictive validity. A fully Bayes approach has the same properties as does the empirical Bayes approach, but it is preferable when the number of neighborhoods is small.

The collection of large-scale administrative records in electronic form by many cities provides a new opportunity for the measurement and longitudinal tracking of neighborhood characteristics, but one that will require novel methodologies that convert such data into research-relevant measures. The authors illustrate these challenges by developing measures of “broken windows” from Boston’s constituent relationship management (CRM) system (aka 311 hotline). A 16-month archive of the CRM database contains more than 300,000 address-based requests for city services, many of which reference physical incivilities (e.g., graffiti removal). The authors carry out three ecometric analyses, each building on the previous one. Analysis 1 examines the content of the measure, identifying 28 items that constitute two independent constructs, private neglect and public denigration. Analysis 2 assesses the validity of the measure by using investigator-initiated neighborhood audits to examine the “civic response rate” across neighborhoods. Indicators of civic response were then extracted from the CRM database so that measurement adjustments could be automated. These adjustments were calibrated against measures of litter from the objective audits. Analysis 3 examines the reliability of the composite measure of physical disorder at different spatiotemporal windows, finding that census tracts can be measured at two-month intervals and census block groups at six-month intervals. The final measures are highly detailed, can be tracked longitudinally, and are virtually costless. This framework thus provides an example of how new forms of large-scale administrative data can yield ecometric measurement for urban science while illustrating the methodological challenges that must be addressed.

This paper assesses and synthesizes the cumulative results of a new “neighborhood-effects” literature that examines social processes related to problem behaviors and health-related outcomes. Our review identified over 40 relevant studies published in peer-reviewed journals from the mid-1990s to 2001, the take-off point for an increasing level of interest in neighborhood effects. Moving beyond traditional characteristics such as concentrated poverty, we evaluate the salience of social-interactional and institutional mechanisms hypothesized to account for neighborhood-level variations in a variety of phenomena (e.g., delinquency, violence, depression, high-risk behavior), especially among adolescents. We highlight neighborhood ties, social control, mutual trust, institutional resources, disorder, and routine activity patterns. We also discuss a set of thorny methodological problems that plague the study of neighborhood effects, with special attention to selection bias. We conclude with promising strategies and directions for future research, including experimental designs, taking spatial and temporal dynamics seriously, systematic observational approaches, and benchmark data on neighborhood social processes.

This paper considers the quantitative assessment of ecological settings such as neighborhoods and schools. Available administrative data typically provide useful but limited information on such settings. We demonstrate how more complete information can be reliably obtained from surveys and observational studies. Survey-based assessments are constructed by aggregating over multiple item responses of multiple informants within each setting. Item and rater inconsistency produce uncertainty about the setting being assessed, with definite implications for research design. Observation-based assessments also have a multilevel error structure. The paper describes measures constructed from interviews, direct observations, and videotapes of Chicago neighborhoods and illustrates an “ecometric” analysis—a study of bias and random error in neighborhood assessments. Using the observation data as an illustrative example, we present a three-level hierarchical statistical model that identifies sources of error in aggregating across items within face-blocks and in aggregating across face-blocks to larger geographic units such as census tracts. Convergent and divergent validity are evaluated by studying associations between the observational measures and theoretically related measures obtained from the U.S. Census, and a citywide survey of neighborhood residents.