Gaby's odd behavior throughout most of the film (her occasional recklessness, cool nature in dangerous situations, desire to get drunk and slap around Illya, and occasionally just plain weird behavior) make perfect sense when you find out that she's a British asset and agent working for Waverly who just had her cover blown by two different versions of The Ace from both the United States and Soviet Union, neither of whom knows about her actual affiliation, and dragged into someone else's operation. Considering just now chaotic, dangerous, and tenuous that situation is, it's no wonder she doesn't give a damn.
As for Gaby's reasons for betraying Kuryakin and Solo part of it could be because she was obeying orders by Waverly but also think about it. Earlier in the film, Solo tells Kuryakin that in order for him to maintain his cover, he must act like a normal man instead of a KGB agent trained to fight. Ultimately, Kuryakin was unable to do this as evidenced when he lost his temper at the Vinciguerra party and beat up that trio of Italian aristocrats. It was here that he blew his cover. But in compromising his cover, he also compromised Gaby's as well, since she is meant to be masquerading as his fiancee. Once Team Fascist realized he was not what he seemed, suspicion would probably fall on anyone associated, such as his "fiancee". Waverly probably realized this, contacted Gaby and pretty much told her look, your Russian boyfriend screwed the pooch and in all likelihood, once the Vinciguerras get their hands on you, you're in for a night of torture and interrogation. Her only choice at that point was to pretend that she was playing Kuryakin and Solo the entire time if only to avoid getting imprisoned outright and interrogated outright.
When Waverly mentions that Kuryakin was already exposed by the time Gaby betrayed him and Solo, he was referring to the Count Lippi incident at the party. So Gaby decided to give him a head start so he would have time to escape, and a chance to save Napoleon and possibly herself. And how did she know he would be able to pull this off? Because he had told her the night before that he kept bugs and trackers on Napoleon. And she somehow deduced that he must have done the same to her as well.
We're told that Uncle Rudi is Gaby's maternal uncle, making him and her father brothers-in-law. And it's blink-and-you-miss-it, but Rudi's last name has a von in it, which indicates that side of the family is aristocratic. No wonder Uncle Rudi makes the comment about "the blood of the racehorse" - it's not even about the Aryan ideal (since the 6'5" blond Illya fits that bill) but that he doesn't come from the right kind of family.
It could still be a race issue; Illya is Slavic, not Germanic.
Meta-example here in the characterization of Napoleon Solo: yes, it's true that this Napoleon is a good deal more amoral and cynical than TV!Napoleon. However, in his last-minute action of giving Illya his father's watch and then quickly making up with him, we can see a bit of the idealism and optimism from Robert Vaughn's portrayal of the character. Perhaps he'll either get a chance to develop a bit more of it assuming there are more movies, or it's already there, just buried a bit deeper under the surface.
Victoria Vinciguerra practically relishes the idea of watching Napoleon suffer, and is disappointed that she can't stay and watch. It's clear that she's worked with Rudi before and admires his work (saying that he's "never in a rush" and describing his torture as "the work of a tireless artist"). Just how many people has Rudi tortured for Victoria? And was it to extract information, or just to satisfy Victoria's sadism?