More-or-less all the worlds this takes place on were conquered by the Rith at one point in time, right? So...where are they? One would assume they'd want to colonize their conquered worlds, that generally being the point of conquest. It doesn't seem likely that they'd move away just because the Hegemony doesn't rule their planet anymore, so where are they?
* It's been 400 years since the Empire pushed the Rish out of those captured worlds. All of the colonists have probably been convinced to relocate by now.