"Why leave the Atlantean paradise, or any other unearthly paradise?"
The choice was not exercised because there was no context to make it for most. Some made the choice to carry the essence and root memories for the whole of humanity. Having made that choice some of that group understood the "paradise" within.
"Why seek as a soul to delve into the duality that brings the struggle when our only purpose is then to spend our life trying to get beyond it into the very Unity we left in the first place?"
The unsatisfying answer - "This is the play of consciousness".
Longer answer:
Struggle and suffering on the basest of levels is endemic to life on Earth.
Because a soul forgets (most of the time) upon re-entry to the Earth plane. A Reawakening may happen. So, it is not about delving. We awaken to the context we are in. How to live in it without becoming attached or identified with it is the journey. Suffering will continue to exist, but by releasing our attachment to suffering we are not suffering over our own suffering. (BTW "Why" questions are intellectual-ultimate questions that are not in the realm of human consciousness except maybe for those that are "transcendently enlightened". - IMO)
Also many make the assumption that a Soul's Body is always completed. If that were so reincarnation would not be necessary. The Soul's body needs reincarnation, to make more light needed for its completion. Reincarnation is the fastest way and maybe the only way to complete the process. Duality, suffering, and struggle are the fuels necessary to make light.
"I am not Newtonian--more the Quantum paradigm."
You are neither Newtonian nor Quantum Mechanics (QM) but I think I know what you mean. Because your spirit is in a physical body you are subject to the laws of Newtonian Mechanics (NM) unless you have bent these laws. QM is the backdrop to NM. QM does not make NM invalid. Both are valid and very different. But if you were to vanquish all your illusions about yourself and the world QM would embrace NM and you could bend its laws and pass through walls without damaging your body or the walls, for example.
Souls choose NM because in the 3D/4D worlds the greatest friction between the highest and lowest is created for making light. Its not just the individual soul's journey that's at stake. It's all humanity and trillions of other worlds that long to incarnate here to make light to raise vibrations on trillions (maybe more) worlds. Why here? I don't know why, I just know that it is so.
"If consciousness is all, it seems to me we had that consciousness before we arrived here and then raced into duality, the earth vibration. a lower vibration."
This last question is answered above. Is it not?
Also grasping paradox on a higher vibrational emotional level may be key to understanding all your questions.
A teacher, Bhante Dhamawara Mathahera, once said: "For every person that loves, it nullifies the hate in ten other people." I believe this is the reason why planet Earth and humanity continue to survive.
