Yikes, that's a toughie Kristin! Of the two, I would have to go with Port Orleans Riverside. I prefer the layout of the resort, as it's not as spread out as Caribbean Beach Resort. The main pool at Old Man Island is awesome too, and, in addition to having all the smaller pools throughout the resort, you can also use the pool at Port Orlean French Quarter. We love going to Downtown Disney, so the boat ride there from Port Orleans is also a huge plus for us.
Now, to confuse you further, another great moderate resort is
Coronado Springs. The rooms are beautiful, and, since it's a conference resort, it also has a salon and fitness facilities, something the other two don't. That being said, I'd still choose Port Orleans, and as a matter of fact, we're booked there for the Princess Race Weekend!
Hope this helps Kristin! Write back with any other questions!