I have a rat terrier, is it okay to get some gerbils?

It should be fine. As long as you get an aquarium-style cage your gerbils should be perfectly safe. However, I would recommend putting the tank on the ground rather than up high. Higher up, they're susceptible to drafts and sunlight might shine on them. Their tank is like a greenhouse and any direct sunlight could result in them overheating.