Response from Stephen Maitzen on March 5, 2012

Not really my area, but until someone else responds...

I can see why you'd be tempted to think so. If numbers -- standardly understood as abstract objects -- exist, they're causally inert, and so they can't affect the world in any way. But I'm not sure that implies that their existence is just as compatible as their non-existence is with everything else we believe.

It's highly plausible that numbers are essentially noncontingent: they exist necessarily if they exist at all. The concept of number doesn't seem to be a concept that could be instantiated only contingently. So, given common modal assumptions, it's either necessarily true that numbers exist or else necessarily false that numbers exist. Whichever one of those it is, then, the other one is impossible and hence inconsistent with everything we believe. Now, we might never be able to discover that inconsistency, and so the question whether numbers exist might be undecidable in that sense. But I'd be surprised if it were provably undecidable in the way that the continuum hypothesis and the axiom of choice are provably undecidable in standard set theory.

It also depends on what's meant by "all other things that we believe." Some people believe nominalism or physicalism about ontology. Those views imply the non-existence of numbers, understood as abstract objects, even though they're not views only about the existence of numbers.