I think you pretty much stated your problem. Please don't take this as me being an a$$ - a simple solution is SLOW DOWN. Archery, especially archery hunting, is all about patience, self control, and keeping your head in the game. Mind over matter. \
When I began archery, I rushed my shots because I was pulling too much weight to hold for a long time. So I had to build up to the weight. After a few years of hunting, I hit a funk where I was wounding everything I shot at. I was aiming a the vitals, instead of a specific pinpoint in the vitals. Now I imagine the small spot where I want to hit and aim there.
Don't over think or over complicate it. just my .02