Short Version
Texture units (aka TMUs or texture mapping units) map textures onto 3D geometry. (see Texture Fill Rate )
Long Version
3D scenes are generally composed of two things: 3D geometry, and the textures that cover that geometry. Texture units in a video card take a texture and 'map' it to a piece of geometry. That is, they wrap the texture around the geometry and produce textured pixels which can then be written to the screen.
Textures can be an actual image, a light map, or even bump mapping.
