William Crimsworth, an orphan, defies his aristocratic relatives by refusing to become a clergyman and marry his cousin. This causes them to disown him. He applies to his successful older brother, Edward, for work. Edward uses him as cheap labour in his manufacturing business and refuses to own him as his brother. William is depressed and cannot fit in into the society of that industrial town. A wealthy manufacturer, Hunsden, notices him, and makes overtures of friendship towards him but William shuns them. After Hunsden denounces Edward's ill-treatment of William at a town hall meeting, Edward confronts William and threatens to whip him. William stops work and on Hunsden's recommendation goes to teach at a Brussels boy's school owned by M. Pelet.
He is also hired by the girl's school headmistress next door, Mademoiselle Reuter, a shrewd manipulative young woman. At first he is attracted to her, but over time loses his attraction to her. He finds the girls noisy, ill-bred and inferior, except for a girl called Sylvie, and expresses his dislike of Catholicism which he thinks is curbing Sylvie's independence. The lace-mending teacher also takes English lessons with him, and he finds that she speaks and writes English well. This makes him interested in her. After class he questions her, and finds that she is from Switzerland and her mother was English. She came to Brussels to earn a living and wishes to teach English for a living. Without education, money or connections however, her prospects are limited. Crimsowrth becomes attracted to Frances Henri, the lace-mender. At this point Mdelle Reuter dismisses Frances and flirts with Crimsworth, but he is disgusted by her cruelty. He resigns from her school.
By accident he finds Frances at the grave of her aunt and he gets to know her better. He reads English classics with her and longs to marry her. But in the meantime M. Pelet, who is engaged to Mdelle Reuter accuses him of stealing his fiancee, and has less income now. He also resigns from Pelet's school, knowing that his wife will want to commit adultery with him. All this while Crimsworth struggles with depression and loneliness, which haunts him, even though he has the love of Frances.
Frances in the meantime secures a job as teacher at an English school in Brussels. Crimsworth approaches the father of a former pupil whose life he saved for a job recommendation, and eventually gets a job at another school, with a good income. With his increased income he can marry Frances. He proposes to her and tells her to resign from her job, as she doesn't need her job now, but she insists on being equal and working along with him.
During their courtship Hunsden comes to Brussels to see William, who introduces him to Frances. Hunsden and Frances argue fiercely on politics, economics and England, and Hunsden is impressed by her fire. Hunsden and Crimsworth have a tussle, where Hunsden says Frances is too good for Crimsworth, but not his type. Hunsden appears as a cynic, Frances a Romantic.
Crimsworth and Frances marry. Some years later they start a school with many well-connected clients, and prosper. Frances gains confidence and affection from her pupils. After they have realised sufficient capital they wind up the business and move to England near Hunsden. They have a son called Victor, a melancholy boy spoiled by his mother. The ending is ambiguous, as Crimsworth shoots Victor's rabid dog, and the boy bursts into tears. He will go to Eton, where he will suffer.