I think I heard the same podcast or read the same article from Randy Ulmer. I can shoot decent, but definitely not good enough to see much, if any difference (like Randy Ulmer). I still bring a target and check my zero at some distance with broadheads when I get to my base camp just like you said. So far I havnt had to make any adjustments, but you never know if something gets bumped or loosens up while traveling.You will find a bunch of answers on this......in the end, you dont need to do anything but recheck it at camp.
If you want to really geek out about it Randy Ulmer ran some math and produced a pretty indepth artilcle on it that proves you need to actually lighten up the draw weight a little when climbing up in altitude to keep POI the same. Like a 1/2 quarter turn for 1 -2 lbs less. If I remember his wording correctly.
I go from 8500 at the house down to a 6000 to shoot a 3D range, back up to 8500 and up to 11,800 at camp with no adjustment needed on my Hog Father single pin.
Id be interested in Dan Ps thoughts on this.