Hi Amanda! Thank you for visiting the Disney Parks Moms Panel!
Disney's Caribbean Beach Resort is one of my son's favorite Disney Resort hotels and I know you will have a magical time when you are there! Just pardon the pixie-dust while the Resort hotel is being reimagined.
I spoke with a Cast Member and he confirmed you will be able to get fresh milk while you are there. There are still several dining options available for breakfast, lunch and dinner. Grab-and-go meals will be available for purchase in 3 Island Markets located in the areas of Aruba, Jamaica and Martinique. The Island Markets feature items such as yogurt parfaits, pastries, sandwiches and specifically directed to your question, an assortment of chilled beverages.
If you don't find what you are looking for at the Island Markets, you can also contact in-room dining. I know the Cast Members will do everything they can to accommodate you during your vacation.
You have just a few short days before your trip to your "island paradise!" Thank you so much for sharing your question with the Disney Parks Moms Panel and please let us know if you need any other last minute tips or advice before you begin your magical Walt Disney World Resort vacation!