Hi Amy,
I think the
Port Orleans - French Quarter resort may be just what you and your family are looking for. It is by far my favorite moderate resort because it is smaller and much quieter than many of the others.
Quiet doesn't translate to boring, however. The pool with a sea serpent slide is always a hit, and the resort offers access to Downtown Disney by water taxi.
Port Orleans -- French Quarter is adjacent to
Port Orleans -- Riverside, which is bigger and has more amenities that might be appealing to your younger child, such as a playground and fishing hole. It also is fairly quiet, but would have more hustle and bustle than French Quarter.
It's an easy stroll from the French Quarter to Riverside to take advantage of its pools and other fun.
The other resort you might look at is
Coronado Springs. Because it has a convention center, my experience has been that a greater number of the guests there tend to traveling without small children.
I hope that helps. Please let me know if I can answer any more of your planning questions. Enjoy your time with your family at Walt Disney World!
Wilma