The Fanalis tribe's lack of ability when it comes to wielding magic makes sense with The Reveal that they are actually a non-human race of beast-men that were somehow granted human-like forms. The flashbacks of Alma Torran show that magic was a power granted only to humans to give them a much needed edge over the other more physically powerful races. The Fanalis tribe can barely use magic because they are barely human.
Ja'far is appalled at Sinbad for trying to recruit children (specifically Aladdin, who is 10 years old at this point) to help him defeat the Fog Troupe. At the time, it seems like just another instance of Ja'far playing the Straight Man to Sinbad. Then, much, much later, you find out that when Ja'far was 10, he was a hardened child combatant in a fiercely, intensively abusive military subdivision of assassins.
What, exactly, must have been done to Ja'far starting in his very early childhood in order to turn him into the character he's introduced as in Sinbad's Adventures at the Improbable Age of just 10 years old.