Hi Sara! Thanks for writing the Panel with the great question!
The theme parks calendars are posted six months in advance, which helps with planning. Not knowing your dates, I checked several random ones and when I found the Hollywood Studios closing at 8:30 pm with Extra Magic Hours 8:30-10:30 pm,
Fantasmic! was being held at 8:30 pm. This is usually the norm, with
Fantasmic! being showed at closing time for all guests to see.
You are totally right that Extra Magic Hours are for guests of Disney resorts only, just one of the great perks. You will be asked to swipe your Magic Band before entering attractions during Extra Magic Hours to ensure you are a Disney resort guest.
Keep the questions coming Sara as you plan your magical vacation!