Roguelikes are a subgenre of Role Playing Games, so named for being like Rogue, a very early computer game.
Rogue was a dungeon simulator originally played on text terminals in the early 1980s, which used ASCII characters to represent a tile-based game world. For example, your character is an @, walls are represented by lines of | and —, ! is a potion, and the various letters of the alphabet represent different monsters (H is a hobgoblin, while D is a dragon). You're effectively looking at a top-down view of a dungeon made of text characters.
Every game, the dungeon would be arranged differently, with different items to find, and the various monsters would appear in different places. All of this meant that the game was never the same twice, making it incredibly replayable. The game was turn-based, with everything in the world moving on each turn, which meant that no quick decisions were required - you could play it like chess, thinking carefully about your options when you needed to.
Adding to the addictive nature of the game was the thrill of permadeath - the fact that the death of your character would end the game, forcing you to start again from the beginning no matter how far into the dungeon you got.
This combination of random generation, turn-based combat, and permadeath is the defining characteristic of Rogue. Players relished the risky, rewarding challenge offered by the game, and it wasn't long before copycat games began to follow, giving birth to the roguelike genre.
One factor that likely contributed to the rise of roguelikes was the fact that they have no graphical requirements. Any coder can create one without having to worry about graphical or audio resources - the only requirement of a roguelike is the ability to manipulate a grid of text characters, which any computer system can do trivially.
Because of the lack of reliance on graphics, roguelikes tend to focus far more on game mechanics instead, with the result that they are often extremely intricate, and allow for complex strategies and interactions.
Today, a truly enormous family of roguelikes exists. Many are written as labours of love, or as experiments to try out new and interesting game mechanics. (The 'experimental roguelike' is practically a genre in itself.)
The most traditional roguelikes have the following characteristics:
Roguelikes are centered around Dungeon Crawling through randomly-generated environmentsrandomly stocked from a huge list of monsters and items. This was arguably the main attraction of Rogue - that it could offer you a different adventure every time you played. Some (such as ADOM) also have a static overworld and/or special levels, but even those games rely on random content in other places. This means that memorization is not enough to win a roguelike, and walkthroughs as such cannot be made for them, but they have high replay value. On the downside, it is possible to get very unlucky in a roguelike and lose through no fault of your own.
Roguelikes take Final Death to the extreme. When your character dies, that's it - he's dead for good. Saving the game is often possible, but it is only used for having a pause from playing, and when your character dies, the save file is deleted. Save Scumming is thus flatly disallowed (even if it may be possible through outside means). The result of this is that roguelike players are very invested in their characters, and are forced to learn the essential skills for survival.
Roguelikes have only a single controllable character, with a turn-based engine in which everything moves at the same time. Some allow you to have allies or pets, but they can't be directly controlled, only given general orders.
Roguelikes generally feature an enormous menagerie of monsters and enemies, which will have various abilities, resistances, weaknesses, and defenses.
Roguelikes generally have a 'food clock' - characters will hunger over time and have to eat, which means they cannot stay in one place forever - they have to push on to get food at the very least. This forces them to confront the increasingly difficult parts of the game. This is usually a measure to attempt to prevent the player from level grinding. The original Rogue, for example, required you to eat food every so often or starve to death, and it was nearly impossible to find more food on a dungeon level once you'd cleaned it out — but going down to the next dungeon level meant fighting tougher monsters.
Most roguelikes have randomized appearances for items that do not persist from one game to the next. In one game, for example, 'a green potion' might be a potion of healing, but in the next game, it might be a potion of hallucination. Items must be identified either by blindly using them and see what happens (beware of Poison Mushrooms), by careful observation (for example, seeing a monster drink a potion and noting what happens), or by using something which grants identification (traditionally, a magic scroll or an identify spell). It's typical, after dying, to be revealed that you had an item which could've saved you, but was unidentified at the time. They might also up the ante by using Randomly Generated Loot.
Roguelikes, especially the well-known or popular ones, have often been under continual development for many years (sometimes a few decades), making them extraordinarily large and complex. Many have to use both capital and lowercase letters to have enough inputs for their commands, and some go even further. Interactions between gameplay elements are also often very intricate; The Dev Team Thinks of Everything is named for a catchphrase among the NetHack community.
As a result of the above points, roguelikes are mostly very hard. Death is expected to be fairly frequent, enough so that the community has developed the acronym "YASD," for Yet Another Stupid Death. It is easily possible to play many roguelikes for years without even coming close to victory.
Roguelikes can be roughly classified into a few different Subgenres that occasionally overlap:
Hacklikes, influenced mostly by NetHack. They mostly focus on Dungeon Crawling, and have mostly finite resources to force the player to manage them well.
Bands, influenced by Angband. Bands usually feature a non-permanent dungeon, infinite resources and very tough bosses, so the games are focused on taking levels in badass until the player is ready to punch dragons to death.
Coffeebreak roguelikes - simple roguelikes with few controls that are easy to pick up and play. Sometimes these are deliberately short, rather than the sprawling affairs that traditional roguelikes tend to be. These are also known as Roguelites.
Open-sandbox type hybrids of the genera popularized by Dwarf Fortress, though not all Dwarflikes are necessarily roguelikes.
Though they have a very steep learning curve, many roguelikes are incredible time sinks, which is only exacerbated by the fact that most of them are entirely free.
In the west, roguelikes are mostly a niche thing, but their influence can be seen in some of the most popular games of today, especially with the rise in popularity of open sandbox style games. Minecraft developer Notch has admitted to being a huge roguelike fan, which is the reason that Minecraft has a Hardcore difficulty mode (to reproduce the roguelike ideal of permadeath).
In recent years, a quasi-genre jokingly referred to as the 'roguelike-like' has arisen, describing a game that, although not a roguelike, makes use of roguelike gameplay traits (usually random procedural generation and permadeath). Space adventure game FTL, and platform game Spelunky often receive the 'roguelike-like' label.
There are a few roguelikes that have managed to creep into the Western mainstream; the best known is probably Diablo, which was inspired by NetHack, and more recently, Dungeons of Dredmor. The genre is much less niche in Japan, and there are quite a few Eastern roguelikes; the most well-known in the West is probably Pokémon Mystery Dungeon.
See also Multi-User Dungeon for a related genre of RPG with its roots in Text Adventure games.
Notable games in this genre:
3059, 3069, 3079 and 3089 sit in a grey area between action RPG & roguelike while continuing the theme of random quests, enemies, items (that can be customized with random parts), terrain & more.
Diablo and itssequels, which take the Roguelike formula into real time. It's also more lenient-rather than being permanently killed, you're teleported back to town with no equipment when you die, but with your level and everything in your personal chest intact. it also spawns an entity called "your corpse" on the spot where you died that has all your goodies on it. They have arguably become a Genre-Killer in that almost all new post-Diablo roguelikes take inspiration from it instead of Rogue itself. Its own clones include:
Dungeons of Dredmor not only has sprite graphics, but also animations, sound effects, background music, Difficulty Levels and the option to turn off Permadeath, all of which are very rare for roguelikes.
Monster Gate 1 and 2, two GBA games that function very much like the Mysterious Dungeon games, but only had a Japanese release.
The arcade game that these are based on, where you put in real currency to get game money which is used to pay the dungeon fee for each dungeon (and to cast spells). Each dungeon you start at 0 XP, but can usually take up to 10 spells with you. The game also featured a non-interactive multiplayer where you could beat dungeons to take them over, and the ability to customize your own dungeons (set the number of levels, type of enimies, and specials) and challenge other players to try and beat it.
Mysterious Dungeon (Fushigi no Dungeon) games, all but one of which are licensed spinoffs of other franchises:
RedRogue: A Homage to the Trope Namer involving the now widowed lover of @ guided by his revenant to retrieve the Amulet of Yendor and restore him to life. Unlike the original it is in a side-scrolling platformer format with no jumping. Combat system derives from a rudimentary casting and enchantment system with dual-wielding a main weapon and a throwable weapon.
Risk Of Rain, another hybrid of a roguelike and an action-platformer, this time in space.
The tabletop The SPLINTER, takes the tabletop RPG elements that made Roguelikes Roguelikes and brings them full circle: randomly generated dungeons, a large variety of (very bizzarre) enemies, a focus on (randomly generated) gear for survival, frequent and permanent character death... It feels more like playing a roguelike than playing a tabletop.
Zettai Hero Project - By the Disgaea team. Far more lenient that most in that dying is not only not-permanent, it's encouraged. You still lose your fancy equipment(which becomes more taxing as you go on), but dying provides the same bonuses to base stats and stats per levelup as actually beating a dungeon, in a game where you start each dungeon over at level 1.