Haha, interesting question. Haven't thought of that. In many fanarts, they have. I believe they also have belly buttons in the show, it's just not visible because their fourrure covers it. I mean, equines are mammals, therefore if we approach the question from an anatomical perspective, they must have a belly button. :)
Um no, chevaux and ponies are animals, non human animaux would be weird if they had one. Non human animaux are not attached to an umbilical cord, I've never seen any non animal be attached to one.