Non-fighting members following a noble character around are given the summary term entourage.
Most of these people expand the options and available actions in the game for you. Like Soldiers, entourage is trained at Settlements and can be assigned to others as well.
Scouts and Camp Followers do not give you new options, but simply increase a property, which means that recruiting several of them also increases the benefit you gain.
Merchants and Heralds sometimes stay behind to secure their benefits for you on a continual basis. You should usually have as many with you as you expect to use before you can recruit more.
For other entourage members, you typically want a low number, basically one to secure you the option and one or two as reserve.
Entourage can be killed during battle, if your forces are routed from the battlefield, but it does not happen very often. Like soldiers, they also need food and will starve if there is nothing to eat for an extended time.