You would have had to start up at least 8-10 years ago if you wanted to have a chance at doing it for a living, and even if you succeeded you would have felt the decline by now and started to get a little nervous. Times have changed, and it is now a labor of love. Just a little better off financially than cattle farming cause deer don't eat as much. The coolest part is bucks have Antlers and are always different and interesting, cattle aren't. That's why I raise deer.
I guess I should never say never, but your really going to have to put ALOT of time and money in to succeed now. My opinion is it's easier to get a job you can rely on a weekly paycheck, and do the deer as a hobby, because with deer there is no guaranteed income. They can die in more ways we can list,
I don't want to scare you off your dream, just wanted to throw my opinion since you ask.
I always dreamed of raising deer. When I finally did it 8 years ago, my dreams came true. Sure I wish I could make more money than I actually do, but I love to walk out and feed my herd. These animals are amazing. Even when I "quit" someday, I will still have a few for pets.
Good luck Iowaboy, whatever direction you go with the deer.