I agree with Marlee the lamb/goat teats are by far and away the easiest type to use.
I routinely feed new foals unless they are up and at the world in record time and these are the only teats I have ever used...I cannot imagine using the human ones!! (This is not to say the people that use them do not have success, only that I have never had it, only been doing it 30 years so I may yet change my mind
)
You do need to make quite a large hole at first, so the milk drips out under gravity, but you can make a smaller hole in the teat you use permanently, once baby has got the idea.
If you truly think it is the taste of the stuff (and if you have a "nurse mare" filling in it may well be) milk the foster mare off and feed that from the bottle the first few goes.
I never do raise a foal on a bottle, I will always get it into a bowl or a bucket once the danger period is over.
If you do decide to keep it on a bottle make sure you wean it well and get it acclimatised to horses early, bottle babies are far more dependant than bucket babies.
Good Luck with this, it is a real commitment, not one, thank God I have had to do for a few years.
You are right not to stress the foster mare, though, if they do not take to it voluntarily you should abandon it, she could end up rejecting her own foal, and I have seen real tragedies when people have pushed it too far.
It's OK when the mare is willing, although even then, I am not really convinced.