To be fair, it looks as though the directors tried to get around that, as most scenes with Tom and Bernie feature either one standing while the other sits or both standing, and in the Miller's Crossing scene, Bernie is groveling pathetically, disguising his height.
And when the characters find the body, it's mentioned to have been picked over by scavengers, even aside from the normal decay AND the fact that Mink was shot in the face. While we don't see it, it's clearly a disgusting sight. Tom knows it's not Bernie, but the two mooks with him were the idiots who let him alone in the first place. They went looking for a body, found one, and didn't feel a compelling reason to look for signs it was someone else.
Also the body had the same clothes Bernie was wearing.