Hi Lisa! This is such a tough question to answer as there are really no "down" times at Walt Disney World anymore. There are festivals and special events held throughout the year so finding a time that is slower then others can be difficult.
Disney does designate certain dates as peak, others as normal, and some at value. You can also check out the
Disney offers for vacation packages, resort booking, or special ticket pricing. Basically the Value season at Disney will be the end of January thru the beginning of February, most of May, the end of August, most of September, and November until Thanksgiving time. These dates will also find the lowest crowds in the parks although never expect no lines as that won't happen.
As far as weather is concerned the heat and humidity start towards the end of May as well as the daily thunderstorms that pass over. This continues through September. January/February can be quite cool for those who want to do a lot of swimming in the resort pools. My favorite time is probably November as the parks gear up for the holiday season. The weather is warm but not hot and the humidity takes a drop. If you can only go at a holiday time I suggest Thanksgiving as your first choice.