You mean those plastic things NES games were stored in? Yes and no. The plastic cases were just a covering, although that is why they are called cartridges (ink, bullets, and 8 track music in such cases are also called cartridges). The actual game is stored on a chip board, most of which is stored safely in the case, save for the connectors to the system.
Cartridges have some advantages over Optical Discs. First is durability. The cartridges only have the outer connectors exposed to wear, and the systems themselves have no moving parts to wear out or overheat. Second, the games run faster; load-times are very short since data doesn't have to be physically sought on a disc or hard drive. But most importantly, the game itself only takes up a small physical part of the cartridge. The rest of the space can be used for extra chips, which can be used to increase the power of a system. Let's say the system doesn't have enough memory to run the game on the chip, but can accept extra memory from the cart. So an extra RAM chip is included. Early games with save points (like the NES Zelda games) used this trick, and included a watch battery inside the case to keep the RAM from erasing itself. (Flash Memory, which doesn't need a backup battery, was several years off at this point).
The NES, SNES, and Sega Genesis all used this kind of modular system to get more out of their games than the main specs allowed; the SNES was particularly known for these "expansion systems", as some of the cartridges had the Super FX chip (a dedicated GPU that added 3D capability), an add-on DSP for physics calculations, or (in the case of Super Mario RPG and a few others) an entire second CPU clocked much faster than the main CPU.
The disadvantage was the cost. With enough chips, games could cost around $100. That's about how much Chrono Trigger and Phantasy Star IV cost when they came out. Even at the bare minimum, a cartridge generally cost around $14 to manufacture as opposed to about a few cents for discs, which was getting lower and lower. This, combined with the much higher storage capacity of CDs at the time, is why developers left the N64 for the PlayStation. Carts were just too expensive to sell games at a decent profit margin. Nintendo switched to discs with the next system.
But what about handhelds? The Game Boy's simple monochrome-screen design and emphasis on battery life meant that expensive coprocessors weren't of much value; aside from a few "memory mapper" chips (which allowed bigger games than usual via bank switching), add-on chips were barely used in Game Boy carts. This meant that the games could sell for half as much as home console games, and still have a decent profit margin. Also, cartridges' advantages of fast loading times, energy efficiency, and reliability resulting from the lack of moving parts make them ideal for portable electronics. Sony tried using discs for their first portable console, the PSP, but most users found that the discs' long loading times and high power consumption weren't ideal for portable gaming, so Sony switched to carts for the PSP's successor, the PlayStation Vita. Nintendo kept cartridges through all their handhelds, even the DS and 3DS. The technology was also quite different: the Nintendo DS, for example, used flash ROM later in its run, the same kind you'd find in a USB drive.
So nowadays carts seem to be back on track. As the prices of Flash Memory continue to fall and the data they hold rises while optical discs are slowing down a bit, there's a real chance that solid state distribution media could become economical again, and rather soon. In addition to the physical advantages mentioned above, they make Digital Piracy significantly more complicated since you need both a compatible blank cartridge to write on, as well as the hardware to actually do the writing, and as the gaming cartridges are usually proprietary and specific to the system, it's rather more difficult to do than with the open standard disk media (unless the next generation of consoles switches to using USB drives). Time will tell if solid-state media will make a comeback to console gaming.
Initially, predictions that The Eighth Generation Of Console Video Games would use cartridges proved to be incorrect, with the Xbox One, PlayStation 4, and Wii U all using either Blu-rays or Blu-ray derivatives to distribute their games (to say nothing of Digital Distribution). However, 2017 finally saw the introduction of a new cartridge based console, the handheld hybrid Nintendo Switch, which continues the trend from the DS and 3DS of using small "game cards" to store its games. While the Switch has unfortunately exhibited some of the disadvantages that caused the decline of cartridge-based systems in the past (namely high cost and storage capacity, with many Multi-Platform games charging more their physical releases on Switch than on other consoles and/or requiring large downloads to supply content that won't fit on the cartridge), the strong sales of the machine and its games suggest that cartridges may once again become relevant in the console space.