Hi, Andrea!
I am not sure I am understanding your question, but I believe you want to use the pool at
Disney's Polynesian Village Resort for your afternoon break from the parks to avoid heading back to your own resort. Unfortunately, you are unable to "pool hop" as pools are for the exclusive use of that resort's guests.
You are welcome to visit any resort, rest in the lobby, explore the grounds, do some shopping, or enjoy a restaurant. You can even hit the beach and relax in a lounge chair. However, swimming is not something you can do except at your own resort.
You have a few options. You can hop on the bus back to your resort and enjoy the beautiful pools at Disney's Caribbean Beach Resort. To save time bussing it back to the Magic Kingdom and getting the monorail to the Polynesian, you can hire a taxi at your resort that will afford you door to door service. The taxi fare will be around $20, but it will save you time and waiting for Disney transportation.
Another option is to do some resort hopping. Get on the monorail, and head to the Contemporary. Stop by the Contempo Café for a snack and watch the monorail zoom by overhead. Then head back to the monorail and exit at
Disney's Grand Floridian Resort and Spa, where you can enjoy some wonderful shopping, and sometimes even live music in the lobby. It is the perfect place for some down time in the afternoon. For a change of pace, take the boat
transportation back to the Polynesian for your dinner reservation.
I hope you find a way to get some afternoon relaxation without spending too much time commuting. Good luck!
Cathy