Welcome to the Disney Parks Moms Panel, Erica!
I hope your vacation is off to a magical start! I also hope my answer gets to you before your festive night at Mickey's Very Merry Christmas Party!
I have done exactly what you are looking to do: not bring my personal stroller, but rent one upon arrival at Magic Kingdom Park for the party. I'm so glad I did!
Stroller Rentals at Magic Kingdom do not require advanced reservations, they are on a first come, first served basis. Magic Kingdom's rental location is right under the Main Street Train Station at the main entrance. Rentals are $15 a day with length-of-stay options available, too. Just select the highlighted link for more information!
Erica, did you know guests entering Magic Kingdom for Mickey's Very Merry Christmas Party are often allowed "early entry" beginning at 4:00pm? If you can, plan to end to Magic Kingdom earlier than the 7:00pm official start time to maximize your visit.
I hope this is helpful! Please let me know if you have any additional questions. You can reach out to me directly.
Wishing you magical merriment!
Faith