When you find yourself trying to remember a show (or any works) that's on the tip of your tongue but just out of reach, come here - the collective brain of the TVTropes community can probably help. Post all the details you can remember (examples help). If you're looking for a trope, head over to Trope Finder. Have general questions about tropes? Visit Ask The Tropers!
Could it be Big O? Anyway, you should check all the anime in this list: http://en.wikipedia.org/wiki/List_of_Adult_Swim_programs
I think it might be Code Geass, which had bright colors and Zero's outfit was blue. The robots were also fairly small and the Gurren was red.
@Metal Gear Chick, Yes to me an you its recent, however its really 5 years old now and a current College student could have seen it when in junior high. It seemed to fit so I suggested it.
I believe you may be talking about s-CRY-ed. It was one of the first shows that Adult Swim showed and the "blue guy with a robot" is named Ryuhou and the "red mecha" Kazuma, well he wasn't exactly a mecha but his arm was kinda. That is the only show I remember them airing with a description like that. And the "feared group" is the HOLY organization.

there was this anime I saw on adult swim one night when I was a wee lad. I remember it had robots and lots of seizure inducing rainbows which surprised me because it was a really dark show.
One scene that stands out from my memory is two people; one was this guy dressed in this blue outfit (I can't remember if it was a uniform or not) and the other person whose features and gender I fail to recall. the two are arguing over something next to the body of water and the guy turns away and this robot appears behind him (it appears between an arch thing that just shows up out of nowhere if I remember right), it's a smallish blue robot not a giant mecha and everyone around them starts to panic because the robot signifies the guy is a part of a feared group or something. There was also this bit about this guy in a red mech fighting something which I also fail to recall.
Edited by blahhaha