Mike Rose's theory of Writer's Block: Writers get blocked when they subconsciously adhere to rigid rules that are actually detrimental to their creative process. Going with this theory, when you intentionally try to create your magnum opus, the self-imposed pressure causes you to impose rigid rules on yourself, but when you couldn't care less about your creation, it follows that you don't care what rules you follow and leave your creative process unfettered. The harder you try and the more invested you are in the project, the more self-conscious you are, and the more rules you try to follow, but you don't bother with rigid rules when you don't care about the result; ironically, the latter conditions produce better results. Now why can't artists turn off their subconscious rigid rules at will? If an answer to that could be found, we could eliminate Magnum Opus Dissonance and cure Writer's Block.
I think that there's no real rule for this. There are many series that are well crafted that go on to become big successes. There are many series made where They Just Didn't Care that came out bad. All that can really be said is that it's hard to anticipate who will like your work and why they would. Individual works have individual circumstances, too.
What original works are hits and what are misses is more or less random from the point of view of the author. Thus, Magnum Opus Dissonance.
When you're writing to get something over with, you get to the memorable stuff quickly. When you write something you love, you linger and get into all the little details. It doesn't mean that the Magnum Opus is bad exactly, but that the throw-away work is easier to gush over. This is especially true when its being written by a genuinely good writer. See also the contrast of Brevity Is Wit vs. Wall of Text.
God blesses works neglected by their creator: "The stone which the builders rejected has become the chief cornerstone." (Psalm 118:22 NKJV).
People are trying to write the Magnum Opus because it's a challenge for them, whereas the work written with less effort isn't so much of a challenge. When the author challenges him- or herself, the writing doesn't match up to the author's capability; on the other hand, when the author writes something easy, it succeeds in matching the author's capabilities. It's like taking a test that requires calculus versus a test that requires just arithmetic: the calculus test will be more satisfying to complete, yet it'll be harder and you'll get a lower grade than the less rewarding, easier arithmetic test. Your personal Magnum Opus is the calculus test, but the evaluator's is the arithmetic test.
Writers are weird people, with very different views on what makes a story good.
A self-described Magnum Opus may relate to or require knowledge of the artist's own experiences and issues, while a throwaway work probably won't. Thus, the throwaway work is more approachable to a mass audience.
The throwaway work was written to appeal to a certain audience, but the artist underestimated how much cross-audience appeal it had.
Wired once had a story about the science of sports chokes and the Centipede's Dilemma. The theory is that most of the time, an athlete is working on pure muscle memory. When the game really really matters, they start actually paying attention to what they're doing, and end up interfering with that muscle memory, causing them to perform at less than peak performance. Maybe writers undergo a similar process - the harder they try, the worse they interfere with their more natural, unforced creativity.
When you're intentionally trying to write your Magnum Opus, you're writing for you. When you write that fluff piece, even if you aren't putting that much effort into it, you're writing for your audience.
What becomes popular and what doesn't can be down to luck always. If the author is extremely unlucky all time, screw writing-abandon all hope.