Hi again Michael, I do appreciate your ability to draw parallels between different cultural and spiritual traditions and I envy your inclination to believe that the very fact that lots of people, in the past, too, have chosen to view our "journey through pain" (to use your very apt definition) as something "useful", it must all make sense somehow - but this attitude after all boils down to the same thing as NDE-based religions, or any religion or "faith" (even if it is not in an established religion) that purports that our lives, after all, must have a (hidden) meaning that we would approve of, and that there is some kind of "salvation" at the end of earthly life: I just don't have such faith.
I don't have such faith (which would be necessary to manage to downplay all the evidence to the contrary that I see in this material world) that "there is a loving wisdom behind it all", as you say. You interpret what humans and animals go through as tough love, I see it as systemic violence perpetrated on ignorant sentient beings, which gets couched in heroic rhetoric to make it more palatable.
I am 100% in agreement with you (and I guess with everybody here on Skeptiko) that the material world we see and touch is most definitely NOT all there is. But this does not imply for me, as it seems to imply to most people here, that 'whatever is behind the veil' (for lack of a better definition) is necessarily what we would wish to find there.
A couple of friends had a baby a few months ago and just found out that it has a very serious genetic disease. It will become increasingly difficult for it to have a normal life, including to eat and to breathe (so he will need medical interventions and support for everything), and its life expectancy is only about 2 years. Now, if there really is a "mastermind" behind the physical world, this means that it has designed it so as to make it possible to happen (and unfortunately I could have made gazillions of similar examples - it is most often "nature itself" that causes pain and suffering, and such suffering is most definitely not caused by the evil actions of human beings, or "all in our mind"), I cannot and will not rejoice about the "mysterious and perfect ways" of whatever has given rise to this material dimension where these and even worse things happen on a daily basis (is it metaphysical but still 'blind' forces/archetypes, like numbers or geometrical patterns? the incomprehensible Ein Sof of Khabbalists? Strings "vibrating" (and whence comes such "vibration") ? A god? Many gods? I have no idea, and this is not the point. The point is the nature of such Source, ie: what it is doing with us).
Thing is, I doubt there is "a MasterPlan" that we would endorse, if we knew it (and that's why we are not told - we are kept guessing, and very probably misled, so that there's a big cacophony of 'spiritual experiences' and it's impossible to really know what is going on behind the scenes), or even that there is some kind of "wisdom" whereby our trials will take us to some "higher level" (during this life? After death?) as you seem to suggest. And if there was such a Plan, I would disagree with a system that enforces such horrors on us to "make us better". They are too extreme to be a means to a benevolent end, constituting a contradiction in terms to any loving wisdom supposedly driving this process.