It probably won't hurt the health of a breeder buck to leave him with the doe year round. If you want him to grow the nicest set of antlers his genetics enable you should consider in what environment on your farm will he be stressed the least. Consider the bachelor groups you see in the wild. Yearling bucks hang with other yearlings and the mature bucks with other mature. Mother Nature is very wise. If you have plenty of pen space I would put your breeder with a few of the smaller bodied bucks you have in his age group during growing season to get the best set of horns on his head. Don't put him all by himself! The biggest bodied buck is almost always king of the pen during growing season. Once in hard antler throw all this out the window. The clean thick mass, narrow, long tined,big bodied 8point will destroy everybody in the pen in pecking order.We as herdsman dictate the stress level of the animals behind our fences.The whitetail is an amazingly adaptable creature but remember forcing him to adapt creates a level of stress. Stress is an antler growth killer. Flees Bucky grew his best set of antlers surprisingly in the doe pen I believe.
