Slavery was common among the Men of Darkness and Orcs as well as many peoples of the Wild Men. Slavery was also common among the late Númenóreans and their black númenórean descendants. Although Slavery was officially not practiced among the Dúnedain and Northrons there still were dependent and virtually unfree bondsmen and serfs.
In Variag (and some Easterling) society large parts of the common populace were considered inferior slaves by the ruling castes. Slavery was generally unknown among Elves, Hobbits, and Ents. Dwarves usually didn't practice slavery but some fallen and outcast tribes had adopted such practices.