Imposition of Will vs Freedom
I am among that latter group.
Religionists like to make the argument that teaching no religion at all is merely teaching a "religion of atheism." This argument is specious at best, and blatantly false at worst.
There is no moral equivalence between forcing people to profess a certain belief system (say, Evangelical Christianity) and preventing people from doing so. While religionists would have you believe (as many of them do) that "atheism" is a religion too, any person with critical thinking skills can see that this is not the case. Atheism is not the embracing of a particular belief system, it is the rejection of one. Atheists are as diverse a lot as can be grouped together, tied together not by shared belief but by a shared skepticism of unprovable claims based on questionable sources.
Even in this there is a polarity of thought, with some tending more towards "I do not believe in religion," and others tending towards, "I believe religion is wrong and/or incorrect." The difference is subtle but very real.
With this in mind, we come to the question of freedom and the imposition of will. While I can mouth the platitudes and participate in the rituals of a religion, this does not mean I share the underlying beliefs that formed those things. If I am forced, either implicitly or explicitly, to claim beliefs that I do not truly have, society suffers from my deception. When a person to claims to be Christian but does not truly believe in the central tenets of their particular brand of that faith, it introduces a layer of deception into every interaction they have with society. It also creates a situation where their actions, dictated by their true beliefs, do not match their stated intents, dictated by their need to present themselves as part of a religion. Good examples of this are the various Christians who regularly attend church and even refer to themselves as Christians, but when called on to oppose war or support programs to feed the hungry, shelter the homeless, or heal the sick use their secret ballot votes to oppose those basic pillars of that belief system.
The end result of this kind of deception (both self-deception and outward deception) is the superficial society which Americans find themselves in today, where the appearance of propriety and goodness is much more important than the truth of it, and where a token gesture is well regarded but a lifetime dedication to good works is derided. This kind of "style over substance" society eventually finds itself at odds with an intrusive reality, often in the form of a large scale natural disaster or a vicious terrorist attack.
While this is not the only factor, by making public display of religion increasingly important, the tyranny of the majority eventually becomes an albatross around their neck, one that they find harder and harder to remove. Once secularized education is corrupted and the majority religion is allowed to impose its will on the young, the core of free society begins to crumble. When children are taught what to think instead of how to think, the superficial "style over substance" culture becomes endemic in the population, and eventually the majority that originally thought itself wise in bringing the minority under its belief system finds itself face to face with a cold, hard reality that respects no books or borders, no laws be they handed down by humans or Gods or any other agency one cares to name.
Imposition of beliefs on others is thus not morally equivalent to preventing people from imposing beliefs on others. In the first case, great harm comes not just to the impose but the imposer, and all of society suffers. In the second case, a healthy diversity of ideas allows society to remain adaptive to intrusive processes, be they natural or artificial, and thus reduces the chances of negative progress due to those intrusive processes.
This is why the theocratization of society must be opposed by all people of good conscience. To fail to do so is to hand the fate of society into the capricious grip of nature, to risk actual loss for an illusion of gain. This must not be allowed to happen.