We will discuss recent advances in the compression of pre-trained neural networks using both novel and existing computationally efficient algorithms. The approaches we consider leverage sparsity, low-rank approximations of weight matrices, and weight quantization to achieve significant reductions in model size, while maintaining performance. We provide rigorous theoretical error guarantees as well as numerical experiments.