Hi Jennifer,
It seems to me that when you are planning to go is neither a "low attendance" period, nor a "high attendance" period. You are kind of right in the middle of the pack. Late April through early June
(except for Memorial Day weekend), would be an okay time to go, especially weather wise, it should be quite comfortable. Keep in mind that special events and promotions can influence the park attendance and so far the only things I notice are "Mothers Day" on May 11th, which could get a little crowded if you happen to be here on this particular day, and
Epcot International Flower & Garden Festival:
March 19, 2008 - June 01, 2008
Have a field day as Epcot celebrates spring with a colorful array of fragrant flowers, gorgeous gardens and "toon"-ful topiaries.
If you are not too worried about Epcot being busy at this time and you miss May 23-27 (which is considered the "highest attendance" period), you should be in good order! I hope this helps and that you have a fantastic vacation!
Best wishes,
Brigitta