What's a realistic rate of growth for my town?

I'm trying to come up with a population size for an American city I invented. The backstory is that it was founded in the 1780s alongside a river. The local geography made the area ideal for an inland harbor and the settlement became of stop off point for boats traveling through the river. This benefitted the local economy and the town grew into a commercial center. Another event I was thinking of happening would be for a local farmer to have discovered a vein of gold at one point, resulting in another population boom.

For the population numbers, I was thinking the initial settlement would be less than one hundred. Within ten years, I was thinking the population would grow to about one thousand. On the eve of the Civil War, I figured it'd the population would be over 10,000. In present day, I was thinking the population would be over 100,000 people. Are my numbers realistic? Are they too big or too small for an inland port?

Not so far off. Compare the historical growth rate for Detroit, Michigan

Interesting. Another aspect of the city is that it's controlled by a political machine. My idea was that this machine would develop in the early 19th-century. Would a town with a population of 10,000 have a political machine or city boss?

Population depends on several factors-how enticing the place is to migrate to, and also what economic incentives there are to have children. Part of why people in the pre-industrial age had so many kids was in part because the kids died pretty often, but mostly because kids were a very safe financial investment (you can put them to work for a profit for you).

Be mindful of that Gold Rush era, many mining towns in the American West in Colorado, California and elsewhere experienced huge shifts in population from the initial boom and then going bust. Some of those towns never recovered despite at their height being hundreds or even thousands of people, they're now ghost towns.

Depends on what you mean by "political machine." The original RL political machines in the US got their start in response to large scale immigration, which itself was a result of the labor shortage caused by the industrial revolution. Such a large influx of second generation citizens needed access to jobs and other services, and rising politicians needed a way to organize the new voters, so local bosses from the ethnic groups in question filled the gap.

Previous to that, politics was primarily the vocation of wealthy landowners and their friends. There weren't enough civil service jobs to hand around, so patronage systems hadnt arisen yet. If you follow the career of someone like, say Andrew Jackson, he got his start by becoming a lawyer and a landowner, and becoming known in that capacity to a lot of people. In other words, it was based on personal connections, not some organization. Abraham Lincoln was much the same.