How does Indy know not to look in the Ark? Because it says so in The Bible: 1 Samuel 6:19 to be exact. Obviously Indy did his homework better than Belloq, because he knew he had to close his eyes to be spared by the Ark's power. Belloq's entire archeological method consisted of follow Indiana Jones and steal the shit he finds. In his meeting with Army Intelligence, he displays his Biblical knowledge and chides the officers, "Didn't you guys ever go to Sunday school?"
Raiders chronologically comes after Indiana Jones and the Temple of Doom. He may tell Marcus that he doesn't believe in "superstitious nonsense" but he has seen the power of the Sankara Stones himself when he evoked Shiva, the benevolent Hindu god. He isn't willing to believe the stories of the Ark are real but he knows from experience there's still a possibility. That's why he doesn't blow it up with a bazooka when he has the chance. When they open it and all the lamps blow out Indy realizes the power of the Ark is real and the only way to survive is from those instructions in the Bible.
The Sankara Stones were also at a fraction of their power (literally, having only three of the five) and more subdued in the way they demonstrated it, with the most overt show being burning the Big Bad. He's aware of the supernatural but he's never seen it outright kill anyone.
It's not necessarily the possibility that it's actually full of deific power that stops him from blowing it up. It's that it's a priceless archaeological find. Irreplaceable, historic, and will make him incredibly wealthy and famous to find it
Belloq mentions that he thinks the Nazis are after the Ark because any army that carried it into battle was always victorious. In the Bible, every time the army (God's chosen people included!) carried the Ark into battle without God's specific direction to do so, they were SOUNDLY trounced and usually lost the Ark as well.
Although it mostly shown God's wrath, there was a blink if you miss moment, when it was all over. Indy and Marion are now free from being tied up with Indy holding up what was left of the burned rope, after seeing if Marion was ok they both look at the Ark with it giving off some kind of gentle glow. God took the time out of his wrath to free two people that respected him enough not to look into the ark.
Indy's father was a quiet, bookish type who alienated Indy's mother while their son was young — so where does Indy's reckless daring come from? In Raiders, Marion drinks Belloq under the table, then pulls a dinner knife to escape his tent, willing to flee a Nazi camp in the desert without even shoes — a textbook Indy Ploy, suggesting both learned this from her father, Abner.
Before opening the Ark, Belloq recites an actual Jewish prayer that is said when opening the Torah ark in synagogue. However, he omits the last part of the prayer, which consists of a blessing to the Jewish people (because he's with the Nazis!).
The Ark burning off the swastika in transit might be a reference to 1 Samuel, when the Philistines captured the Ark and its first and second warning to them was to knock down, then destroy a statue of their god Dagon. Clearly, the Nazis only deserved one warning, but they were warned nonetheless.
Indy's warning to Marion to close her eyes when the Nazis open the Ark doubles as a warning to young or squeamish audience members (and the parents of such) as well, letting them know exactly when to put their hands over their eyes before the most grotesque scene in the movie begins in earnest.
Consider how that after all of the trouble Indy went through to get the Ark, it ends up being placed inside of a box in a warehouse somewhere. Who's saying that there aren't things that are more dangerous than the Ark inside of that warehouse too? This is confirmed in Indiana Jones and the Kingdom of the Crystal Skull.
While they're probably taking precautions to ensure only appropriate people have access, one wonders how long it will be before someone disrespects the Ark and dies. The bureaucrats certainly don't seem to respect it, and a box in a warehouse is hardly the proper place for such an important religious artifact.
Or it's the perfect place for it: out of the way, where it can't be used at all, much less misused.