It was created during two religious upsurges - on coins in 1864, and on bills in 1957, and is, in fact, unsconstitutional based on the US Constitution's First Ammendment Free Exercise Clause. Or, it would be, had it been actually by the government.

It is issued by the Federal Reserve, which is not a government institution, but a private one, despite on popular belief in otherwise. It has its own non-government regulatory body, and is composed of twelve private banks and several trust funds.
Perhaps in the context of today it would be unconstitutional to take it off(The "In God We Trust" thing in case everyone forgot what the thread was about), no?