Many factors (physical, chemical, natural, and human activities) contribute to the degradation of historic buildings. Preventive conservation is a cost-effective approach international preservation bodies recommend to mitigate risks to built cultural heritage. A substantial challenge is bio-colonization, especially by microalgae, which affects brick-facing masonry surfaces due to environmental factors (i.e., temperature and moisture), leading to progressively increasing deterioration. Early detection systems could be useful to reduce damage from these organisms. Advances in computer vision and machine learning, such as convolutional neural networks, offer promising solutions for automating the identification of building pathologies using image collection. This research focuses on developing predictive models using convolutional neural networks to monitor bio-deterioration on historic facades, specifically targeting early-stage microalgae colonization. After a training phase using laboratory-induced bio-colonization on brick samples, the method was applied to real case studies of architecturally significant buildings affected by bio-colonization. In fact, a substantial number of digital images of these buildings, even if taken for other purposes, are available. The work shows that analyzing these images with the trained network facilitates the early detection of bio-colonization, providing a contribution to the field of built cultural heritage conservation.