As with all things Mike, I think one must always weigh ''risk against gain'', as in why is there a need to leave them ''out'' for awhile.
I will be speaking here on my experience using 100mg/ml Xylazine mixed with Telazol(100mg/ml) of various mixes of mls of Xylazine per bottle of Telazol(5ml). I've never had any experience using BAM or any other sedative drugs for deer.
Touching on the risk factor - Whenever sedating, it is always best to fast the deer the day before sedation. Xlazine is the culprit drug when using the mix I speak of above. Xylazine takes a deer's ability to use their muscles away, hence the reason to fast your deer the day prior. If the deer has stomach contents enter the throat during the process of the sedation, and then does not have the ability to swallow due to the Xylazine, the deer is apt or can then breath those contents that are in their throat into their lungs, and they will typically die 10 days to 2 weeks or so later of asperated pneumonia.
Fasting or not fasting your deer the day before, you should always keep the deer's head, neck and chest above the rest of the body...the head higher than everything, the neck higher than the chest and the chest higher than the rest of the body would be the perfect scenario. We always used leafs of straw to accomplish this to prop them up accordingly. Sternal angle is good here.
Before I started using Tolazine as a reversal, or reversal at all for that matter(cause I was under the direction of my ''old'' vet not to use reversal, there's a good idea for ya, NOT), I've seen them be out for hours and as long as they were propped properly and fasted the day before, I never had any problems. Before Tolazine, I used Yohimbine as a reversal, which I personally think is next to worthless. I've seen deer wake up on there own with no reversal as quick or quicker than when given Yohimbine, and have seen them lay for HOURS after an injection of Yohimbine...this of course being dictated by each and every individual deer as it relates how they react to the drugs we give them.
So Mike, Yes it's best I believe to bring them back as quick as you can, but I personally would not be real concerned over how long they were under, as long as you have taken all the necessary precautions above, ie fasting and proper positioning of their body.
Best to you Mike!