Hi Glorian:
No worries. You likely will not be sacrificing much by jumping aboard the Wild Life Express Train. For my family (two boys, ages 5 and 4), we make it part of our Animal Kingdom visits. The five-minute Train takes you to Rafiki Planet Watch. The Trains leave about every 5-7 minutes, so if you feel Rafiki's Planet Watch is not for you once you get there, you can easily head back.
The ride is short but fun, and you'll likely grab some glimpses of animal holding areas during the ride - sort of like the "backlot" of Animal Kingdom. Once at Rafiki's Planet Watch area, highlights for my family are
Affection Section and
Conservation Station. Affection Section is an animal petting zoo and Conservation Station is a fun and educational behind-the-scenes look at the animals of Disney's Animal Kingdom theme park. During our last visit, there was a live animal presentation, and my boys got to ask questions and pet a snake.
While Rafiki's Planet Watch Area is fun for all ages, it's possible it may be more popular with your three year old than your older children. It would say it's a must do if your family enjoys animals. I don't think a trip over will be a waste since it's pretty easy to get there and back. However, if this doesn't sound interesting to your family, there are plenty other attractions at Animal Kingdom to keep you entertained.
Enjoy your vacation.