Two r-by-n matrices will be called equivalent if they differ by invertible row operations and rescaling of columns. The set of all matrices equivalent to your favorite matrix is called a matrix orbit, and a matrix orbit closure consists of a matrix orbit and those matrices close to the orbit. I this talk I will describe some algebra and geometry of these objects. In particular, I will describe polynomial equations that describe a given matrix orbit closure. I will describe evidence for two fascinating conjectures, one of which says that a powerful distinguishing gadget for varieties (their Hilbert series) can only see basic linear algebraic properties of a matrix orbit closure. My goal is make most of this talk accessible to undergraduates who have taken linear algebra and multivariable calculus. The material is based on joint work with Alex Fink (Queen Mary University of London).