For the longest time I didn't get what the idea was with the Nazis going after the Ark of the Covenant. What on earth made the persecutors of the Jews think that the God of the Jews would be on their side, and why even bother?? It didn't seem satisfactory somehow just to pass it off as them seeing it in a purely talismanic way. Then I realized: of course! Hitler was always convinced that God was on his side in real life: what more fitting way would he have found as a genocide for the Hebrew people than to have them be slaughtered by their own ark? Only that's what ended up happening to them instead! — Ziggy Zag
One could argue that most, if not all, fanatics start from the belief that (Insert Deity Here) is on their side, logic be damned.
Alternately, the Nazis thought the Ark was a powerful supernatural artifact of some mythical Aryan super-race, which the Hebrews had stolen long ago and then claimed to have crafted in honor of their patron deity.
The staff they used was six kadans tall - which, according to Indy, is roughly 72 inches - minus one kadan, which comes out to 60 inches. When Indy uses the staff to find the location of the Ark, it towers over him, which, given Indy's height, would make it well over seven feet tall. Perhaps they realized a kadan is longer than a foot, and found a correct sized staff.
How does Indy know not to look in the Ark? Because it says so in The Bible: 1 Samuel 6:19 to be exact. Obviously Indy did his homework better than Belloq, because he knew he had to close his eyes to be spared by the Ark's power. This isn't too surprising really once you consider Belloq's entire archeological method consisted of follow Indiana Jones and steal the shit he finds.
Belloq was dressed up like an ancient Hebrew priest, which suggests he had read the relevant scripture; he just thought he could fool God by playing the part.
It seems more like Belloq thought it was just "say the magic word and it will work". He does claim the Ark can do things that have never been mentioned anywhere, so it seems unlikely he's done the actual reading.
Indy also reminds his students the importance of research.
Also note that Raiders chronologically comes after Indiana Jones and the Temple of Doom. He may tell Marcus that he doesn't believe in "superstitious nonsense" but he has seen the power of the Sankara Stones himself when he evoked Shiva, the benevolent Hindu god. He isn't willing to believe the stories of the Ark are real but he knows from experience there's still a possibility. That's why he doesn't blow it up with a bazooka when he has the chance. When they open it and all the lamps blow out Indy realizes the power of the Ark is real and the only way to survive is from those instructions in the Bible.
Belloq mentions that he thinks the Nazis are after the Ark because any army that carried it into battle was always victorious. In the Bible, every time the army (God's chosen people included!) carried the Ark into battle without God's specific direction to do so, they were SOUNDLY trounced and usually lost the Ark as well.
Possibly Belloq knew damned well that the Ark wouldn't help the Nazis: he just pretended it'd be the ultimate weapon so they'd fund his expedition to find the single biggest prize in archeology. Deep down, he might not have expected it to have any supernatural powers at all.
As Rifftrax pointed out, "It probably helps to believe in god and not be evil...", which the Nazis certainly were.
When the Ark of the Covenant gets opened and they find nothing but dust in it (at first), people probably wondered what on earth is dust doing inside the Ark. Some think that it was the Ten Commandments Tablets, long since disintegrated over the millennia. However, the dust may actually be the remains of every poor fool who was destroyed by the Ark's terrifying power! It's not a stretch, when you consider what happened to the bad guys at the end...