Sanctuary Fairy Guard

So long as you have spoken to first, during the quest, you have permission to enter and may walk past the Fairy Guards without fear... they will not attack you.