It's a language, yes, but even languages are analogies / metaphors. it's nice that mathematics corresponds well to the skeletal form of the physical Universe so far, although we've hit snags where the mathematics analogy breaks down and mathematics ceases to work as a useful descriptor for observation. But it's a very practical language for a good portion of it. Cognitively, we have two sets of number systems in the brain. One is an instant form; typically happens on the left side of the brain if I'm not mistaken - numosity I think it's called; we recognize numbers up to 3 (sometimes 4) instantly. But anything beyond that, the semantic systems of the brain takes over. Roughly same areas as language, analogy, metaphor, poetry, grammar, etc.