Big Pig actually works with William the Kid
He stole the lucky charm because he knew that somebody was going to use Captain Drydock to find Kid's fortress. For this reason, he works as hard as possible to make sure nobody can beat him at Go Fish, but tries to look innocent.- There is no such evidence that proves that they are in cahoots with each other. The captain may very well have decided to challenge Big Pig to break his streak but got unlucky and lost his lucky charm when he lost as a result.
