A "critical" metric on a manifold is a metric which is critical for some natural geometric variational problem. Some important examples of critical metrics are Einstein metrics and extremal Kahler metrics, and such metrics typically come in families. I will discuss some aspects of the local theory of moduli spaces of critical metrics, and present some compactness results for critical metrics which say that, under certain geometric assumptions, a sequence of critical metrics has a subsequence which converges, in the Gromov-Hausdorff sense, to a singular space with orbifold singularities. I will also discuss some results regarding the reverse problem of desingularizing critical orbifolds to produce new examples of critical metrics on smooth manifolds.