I did the same thing in Cades cove. We went with some friends, and the guy happened to be a Minister, They wanted to check out the old churches in the cove, ...
Do these old churches count as federal buildings? If they're empty and no rangers are there...
I did OC at Glacier and surprisingly my wife was happy about that. The few federal buildings (not privately run gift shops) have federal stickers on the doors, but the restrooms were in a detached building. Gun stayed in the car when in the visitors center.