My wife and I did the Panorama Trail back before we had kids and man it was a fun hike. Couple things to mention, it’s 11 miles back to Yosemite Lodge if you take the shuttle bus from Yosemite Lodge to Glacier Point in the morning. It’s NOT all downhill. There’s an uphill section right after you pass over Illilouette Creek and downhill on the Mist Trail is worse than going uphill. This video is great about showing you all the reasons why you want to put yourself through the pain.
Video by John Galbraith.