As humans increasingly settle in dense urban areas, localized natural and anthropogenic shocks become more likely to impact larger numbers of individuals. Research suggests that resilience to shocks is a function of physical fortifications and social processes including critical infrastructure, social networks, and trust. Although physical fortifications are relatively easy to identify and catalog, social processes elude simple measurement due to data limitations and geographic constraints. Recent work has shown that certain types of infrastructure may correlate with social processes that enhance community resilience; however, the ability to assess where and to what extent that infrastructure exists depends on a complete representation of the built environment. OpenStreetMap (OSM) and Google Places are two sources of data commonly used to locate and characterize infrastructure, but they are often incomplete. We address this limitation by applying a convolutional neural network (CNN) to remote sensing data from Sentinel-2 to estimate the density and type of infrastructure. We compare the classification results to known infrastructure locations from OSM data. Our results show that the CNN classifier performs well and may be used to augment incomplete datasets for a deeper understanding of the prevalence of infrastructure associated with social processes that enhance community resilience.