I definitely think there is enough to do at a Moderate Resort on your off day!
Last year my family stayed at the Port Orleans French Quarter and we had a blast there. We had the benefit of the Port Orleans Riverside being just a lovely walk away and the boat to take you to Downtown Disney which is always a treat for the whole family.
And of course, I can't forget to mention the Moderate Resort's pools and playgrounds! Once the kids see the pools they won't even think about the beach again!
All of the Moderate Resorts are landscaped beautifully in connection to the Resort's theme and my kids always have fun with critter sightings. For example, if you are at the Caribbean Beach Resort don't miss Parrot Cay Island with its playground and live parrots in the trees. You can also rent different types of watercraft at Old Port Royale to explore Caribbean Beach's 45 acre lake!
One of the perks of staying at a Disney World Resort property is that you have the benefit of the extensive transit system. You can take a bus to any of WDW's other properties to explore them. My family loves to visit the other Resorts and take in the scenery.
I think it would be really hard to get bored at any of Disney World's Resorts, and you are sure to be able to entertain your entire group with no troubles.
I hope you have a fantastic trip!