In my house with permission, you have the benefit of my hospitality and the goods and protections that I openly provide.
In my house without permission, you are a criminal invader and will be subject to any actions I make to protect who and what belongs to me. No hospitality is guaranteed.
Is the government the owner of the house? Because if the government gives people permission to exercise their rights, then government can take them away too.