Only digits to the RIGHT of the last significant digit can be "any value." So, yes, if you report a pH of 3, you are giving an answer with "zero" significant digits, in the sense that the leading digit in [H+] could be anything -- any value between 1 and 9, e.g. any number between 0.001 and 0.009. What you do NOT mean is that digits to the LEFT of the thousandths place could be anything at all!
Be careful to distinguish between the everyday meaning of "significant" and the scientific meaning of "significant" in this case. That is what may lead to the confusion. You're thinking that "zero significant digits" has some profound meaning, like the number could be anything between -infinity and +infinity. But the number of sig digs just is a shorthand way of specifying the precision in powers of 10. 1 sig dig means your precision is 1 part in 10^1, or +/- 10%, and 2 sig dig means your precision is 1 part in 10^2, or +/-1%, and so 0 sig dig just means your precision is 1 part in 10^0, or +/- 100%.
In theory it's even possible to imagine a measurement that has "negative" significant digits, e.g. a precision of -2 sig dig would mean an answer precise to +/- 100x, meaning you could be off by as much as two orders of magnitude. In short, the scientific meaning of "significant" in this context is a lot less significant than you may think, ha ha.