Margaret Carey, a widow, and her three children, Nancy, Gilly, and Peter are having financial problems. Nancy inquires about a yellow house in Beulah, Maine, and Osh Popham, that house's caretaker, offers to rent it to them. When they move in, they find the house in a state of neglect, and with help from Mr. Popham, they begin to make repairs.
As the Careys begin to adapt to their new town, they receive a letter asking for their orphaned Cousin Julia to stay with them due to the financial failings of her adoptive parents, the Fergusons. The Careys agree, although Nancy and Gilly are not very glad to have their stuck-up, disagreeable cousin live with them. Margaret forbids her children to tell Julia that the Fergusons had dumped her on them after gambling away their money. When Julia arrives, she is shocked when she is jumped on by a dog, and finds Beulah to be too primitive for her tastes.
While continuing to make improvements to the house, the Careys realize that they don't have enough money to fix the house while continuing to rent it. When Margaret says this to Mr. Popham, he writes a letter to Mr. Hamilton, the owner of the house. He tells Margaret and Nancy that because their repairs added value to the house, Mr. Hamilton had agreed to waive the rent, on condition that they have a ceremony for his mother on Halloween and hang her picture in a place of honor in the house. Mr. Popham chooses a painting from his attic to present as Mr. Hamilton's mother.
The following week, after church, Nancy and Julia introduce themselves to Charles Bryant, a schoolteacher. They invite him to an impromptu lawn party, where both Nancy and Julia attempt to woo him. Nancy talks about literature and sports, and she invites him to play a game of croquet. Julia dresses in her finest gown and acts in a flirtatious manner. By the end of the lawn party, it is clear that Julia has won Charles' affection.
In the evening after the lawn party, when Nancy and Julia are going to bed, they have a fight about Charles and about Julia's condescending attitude toward the Careys and Beulah. In the heat of the moment, Nancy reveals the secret that she was forbidden to tell. In disbelief, Julia runs downstairs in her nightgown to ask Margaret if it is true. Nancy follows her and apologizes for telling the secret.
Margaret confirms that it is true, and she reveals that the Fergusons' situation had improved, and that they are ready to take Julia back. Suddenly, Nancy realizes that she loves Julia and wants her to stay, and she begs Julia not to go back to the Fergusons. Julia realizes that she is happy with the Careys and decides to stay, promising never to mention the Fergusons again.
Everyone gets ready for the Halloween party in which the ceremony honoring Mr. Hamilton's mother will take place. Mr. Popham's daughter, Lallie Joy, who has a crush on Gilly Carey, is embarrassed of her ugly dress. When she shows it to Nancy and Julia, they offer to alter it for her, and they reassure her that it's what's inside that counts. They instruct her on how to act around boys, telling her that the secret to getting a beau is to act feminine, compliment his masculinity, and hide her real self.
When Nancy goes outside on the day of the party, she bumps into a handsome stranger at the door. She tells him about how they came to live in the house and invites him to the party for Mr. Hamilton's mother. The stranger, who is actually Mr. Hamilton, goes to Osh Popham's store and asks him about the people in the yellow house. Mr. Popham nervously tells him about the arrangement he made with the Careys behind Mr. Hamilton's back.
At the party, Lallie Joy is together with Gilly, and Julia is with Charles, making Nancy the only one without a partner. She reluctantly goes to the party alone, and on the way down the stairs, she runs into Mr. Hamilton. When she and her mother reveal the painting in the ceremony, Mr. Hamilton is offended, because the picture is of an ugly woman. He reveals his identity to Nancy, who becomes afraid that he might kick her family out of the house, but he asks her to dance instead.
