It is also important to realize that they are not, strictly speaking, universes either, but it is easiest if you don't try to realize that until a little later, after you've realized that everything you've realized up to that moment is not true.
The reason they are not universes is that any given universe is not actually a thing as such, but is just a way of looking at what is technically known as the WSOGMM, or a Whole Sort of General Mish Mash. The Whole Sort of General Mish Mash doesn't actually exist either, but is just the sum total of all the different ways there would be of looking at it if it did.
The reason they are not parallel is the same reason the sea is not parallel. It doesn't mean anything. You can slice the Whole Sort of General Mish Mash any way you like and you will generally come up with something that someone will call home.
Harold: No way. That kind of thing only happens in poorly written children's books whose authors have clearly begun running out of ideas!
My Child, not to speak of Indras, of those Brahmås there is no end. Brahmå follows Brahmå. One sinks, the next arises. Nor can anyone estimate the number of the universes, side by side, at any moment of time, each containing a Brahmå, a Vishnu and a Shiva. Like delicate boats they float upon the fathomless, pure waters of the body of Mahå-Vishnu. And like the pores of the body of that Great Vishnu, those universes are numberless, each harboring no end of gods such as yourself."
Giga: We're the same person.
Chuck: No you're not. You're alternate universe counterparts. That's a huge difference.
Chuck: I know a version of myself who is a Juggalo Crime Boss. Do I look like a Juggalo Crime Boss to you?
Giga: ...No. But I was recognizable enough that people know my name.
Chuck: That's the two most distinguishing features of Others. Name and appearance. Followed by country and planet of Origin and general hobbies. But other than that...
Giga: Even so, it's horrifying.
Chuck: Yeah... I can understand that...
Iron Man: That's right, and no one else does.