Examples of how environmental susceptibility has a direct impact on the social vulnerability of a population, by affecting both the social and cultural life quality are discussed for the Northeast of Brazil, which is considered to be the poorest region of the country. Several direct and indirect mechanisms associated to soil degradation/desertification are addressed, mainly in relation to the impact they cause to the livelihood such as income, immigration/emigration rate, and mortality. Our purpose is to provide, based on a social vulnerability index, a spatial-temporal analysis of the population dynamics in response to the effects of degradation/desertification and extended periods of drought. The results of this study demonstrated that social vulnerability is mostly dictated by social factors but marginally by environmental factors. This conclusion has an impact on federal public policies designed to reduce social inequality in the region.