I've seen an info-graphic circulating around the Web recently, particularly in atheist spaces. It depicts an infant, and then below that, the words: 'you don't become an atheist. You go back to being one.'
That statement is clearly meant to imply that, since babies have no concept of a god, they must be atheists. Hence, you go back to being one.
This is a bullshit claim, and it doesn't take very long to refute. So let's do that right now. Say it with me, "LOW HANGING FRUIT!"
Babies aren't atheists for the same reason that proponents of the above theory claim that they are. I know that sounds weird, and it is, but just bear with me for a minute. This'll all make sense in time. Atheism as a viewpoint is entirely dependent upon theism to exist. That is to say, if there were no theists, there would be no atheists. The only reason that atheists are able to reject theistic claims is because we were introduced to them first. Without the concept of a deity imparted to us, we would never know how to refute said claims, nor would there be a reason for us to do so.
That's why babies don't meet the definition of atheism. Atheists lack a belief in God, but babies do not. It is impossible for them to make any decisions about their theological positions because they have no context on which to base those decisions. It would perhaps be accurate to state that babies are secular, as in wholly nonreligious, but they are not atheistic. They can't be, because they don't know who or what God is yet.
Just something to think about.