The thing is, the Istari were expressly forbidden from ever retaking their form's to directly contest Sauron. There is a reason that they were in the form of men, it was to ensure that they could never do as Sauron had done and don a fair face and seduce men and elves. Likewise, the limits to their powers were intentional. In fact it could be reasonable to believe that the greatest of the Eldar were mightier in direct power than the Wizards, because fundamentally, the Istari were meant to not be able to contest with the Children of Eru. It could be summed up with there being an incompatibility with Hroa and Fea, Elves such as Galadrial and Elrond would be more in tune with their powers as those powers are a consequence of their soul and body. Gandalf and Saruman in contrast would be more limited, their souls being forced into an unnatural body, in a similar way to how Sauron and Morgoth became intrinsically linked to the forms they wore as they invested more of themselves, however in the process their power became increasingly less, Morgoth once mightier than his kin was left less than the weakest by the time of his final fall, as he had become more tightly bound to his form. Likewise the Istari should probably be much weaker than they are in game, they are not supposed to fight a war, they are supposed to advise others who do!