We consider the binary perceptron model, a simple model of neural networks that has gathered significant attention in the statistical physics, information theory and probability theory communities. We show that at low constraint density (m=n^{1-epsilon}), the model exhibits a strong freezing phenomenon with high probability, i.e. most solutions are isolated. We prove it by a refined analysis of the log partition function. Our proof technique relies on a second moment method and cluster expansions. This is based on joint work with Allan Sly.