Slips are coloured liquid clay and must be applied before work is fired, when it still has some moisture content. They are best applied when the clay has reached the 'leather-hard' stage, otherwise the clay can go mushy. Slips have the same unglazed finish as clay, so you will probably want to glaze.
Underglaze is similar to slip, but with added frit (glass forming ingredients found in glazes) so they can be applied on green or bisque fired ware. The frit means underglazes fire to a smoother slightly glassier surface than slips and so can be left unglazed or glazed, heavy underglaze application can react with glaze and result in a slightlu pitted surface.