Tenchi Muyo!: When Tenchi first sees the Master Key, a physical blade had been affixed to the hilt. This blade was in a state of extreme disrepair, prompting Tenchi to exclaim, "What a rusty old piece of junk!" after breaking it on a rock...which then splits in two.
In the hands of anyone except InuYasha, Tetsusaiga looks like a katana that has seen better days. It's also quite dull, though it's still almost indestructible. When InuYasha wields it, the blade takes its true form. Bonus points for Inuyasha calling it "a piece of junk" constantly in the two episodes that featured him gaining it.
Saito's sword, Derflinger, in Zero no Tsukaima. It is eventually revealed that Derflinger is a "spirit sword". The rusted sword wielded by Saito is simply a vessel for its powers; when the blade is broken, Derflinger can transfer its powers to another completely ordinary sword.
Bleach: Most zanpakutou look utterly ordinary when sealed only revealing a special look when they release their powers. The two zanpakutou that exist in permanent shikai form also fit this trope. Ichigo's zanpakutou looks half-finished, missing both a hilt and a cross-guard, and Kenpachi's is a chipped, pitted mess. Several zanpakutou also look completely ordinary when released, as well, and it's only when the power activates that the true nature and incredible power of the weapons are revealed.
In Slayers, Gourry wields a normal looking sword that suited him just fine. But if he were to take off the hilt he can turn it into The Sword of Light.
Variation in Yu-Gi-Oh!. Rafael's cards are so old that the ink is fading and the edges are frayed. That doesn't change the fact that he has one of the most powerful decks in the series.
Pictured above is the Master Sword, the iconic blade from The Legend of Zelda, as pictured in the last panel of the Nintendo Power comic adaptation. The comic hews to the idea of centuries passing between games: after Link triumphs over Ganon, he returns the sword to await the next hero. This is implied to be a long time coming.
In the 1982 Conan the Barbarian (1982) movie, the Atlantean Sword looks like a solid piece of rust when Conan first finds it in the tomb. That might just be the decayed scabbard, though, since Conan just whacks the sword on stone a few times and the crud comes right off.
Discworld: The absolutely ordinary-looking, non-magical sword wielded by Carrot Ironfoundersson. Does not gleam, has minor chips along the edge, and sharp enough to effortlessly nail someone to a marble pillar. His ancestors had explicitly wanted a sword that was really good at killing people rather than looking good. It is also implied that it has taken on a quasi-magical nature that is far beyond mundane magic weaponry by virtue of being wielded for a good cause by a good man who by all rights should be a king but refuses to be one. One book states that such a sword could cut through a dozen magical words like butter. The sword is so powerful because it is more real than the rest of the Disc, which cannot exist without magic.
Of course the eponymous blade in The Sword of Shannara is this, just a well-made if nondescript weapon actually found and overlooked by the heroes as they ransack the royal armory looking for miracle artifacts. They soon realize their mistake and spend half the book chasing after the luckless serf they'd let take it away.
Dyrnwyn, the Cool Sword in the books, is burned black by the fire that killed its first wielder.
A nonmagical version: Tarin makes a sword with a blacksmith, and initially wants to cast it away because it's ugly and has chips in it. It proves very powerful, and he wields it against Dorath, who is using Tarin's previous beautiful sword, it utterly shatters Dorath's sword.
Fyodor in the Starlight And Shadows trilogy has a two-handed sword, clearly too heavy... and blunt. Far too many people didn't think it was serious until they learned a few little details: first, it's enchanted and works as if was sharp enough for any purpose except "friendly fire". Second, while most people can lift it with a visible effort, the owner is built like a smith (and was one) and always carries the damn thing with him, so he can use the ugly piece of metal as intended. Third, as The Berserker, in rage he swings it with enough strength to crash a minor golem and enough speed to hit a superhumanly fast drow fencer.
In Oathbreakers, of the Heralds of Valdemar series, the long-lost Singing Sword of Rethwellan that is used to identify the country's rightful king is discovered to be none other than a rusty, dirty old sword that Kethry picked up along the wayside.
In The Dresden Files, the Knights of the Cross have three swords that can burn faeries, kill demons, and help dispatch all manner of supernatural nasties. These three swords are fairly plain, save for the nail from the Cross embedded in the hilt. Turns out that all of these swords are actually pretty famous, because they're Excalibur, Kusanagi, and Durendal. So this a very good example of the trope.
Sort of played in Merlin where a plain looking sword is transformed into Excalibur by the dragon's breath.
Commander Kruger carries around an old sword that looks chipped and battered - but when he gets dangerous for the first time, it does too, transforming into a much more impressive blade.
Samurai Sentai Shinkenger: The Kyoryu Disk, when first found, is an inert disk containing only a cryptic message from the founding Shinken Red. Its power was drained to seal the Kusari Ayakashi general Manpuku, Big Bad du jour of the movie who was unsealed by the overall Big Bad's summer power surge. Takeru restores it by slotting it onto his sword and stabbing said sword into the kanji on Manpuku's body left by the seal. Its power restored, the Kyoryu Disk can now be used to transform the Shinkenmaru into the very powerful Kyoryumaru, able to infinitely lengthen its blade to smash into enemies and act as a Whip Sword. On top of that, it can transform Shinken-Oh's Daishinken into the Kyoryu Origami (A mecha version of itself), which combines with Shinken-Oh as a new helmet and a BFS to swing.
Its Power Rangers Samurai counterpart, the Shark Disk (Saban, like some fans, thought the Kyoryu Origami looked more like an ambulatory shark than a dinosaur), was apparently similarly inert though in this case it was offlined by battle damage and said to have been mended by the Sixth Ranger using the same powers and skills he used to repair certain Zords and complete the Black Box.
Dragon magazine #14 article "Interview with a Rust Monster": A PC party unknowingly carries the Sword of Toshio. "[It] looks like an ordinary sword, bent and worn from use. But in the hands of a samurai, it takes on its true appearance as a katana of great power."
The Templars' swords in Deadlands: Hell on Earth are their iconic weapons and often relic or magically empowered. Physically, they are whatever kind of sword you can scavenge in a post-apocalyptic world, most often stolen from ruined museums.
A classic trait of Legacy weapons in Dungeons & Dragons; they always appear to be something much less than what they truly are.
In Secret of Mana, the player acquires a rusty old sword at the beginning of the game, which is none other than the game's legendary Mana Sword. According to the opening scroll, it actually is Excalibur, as well as many other swords of legend.
It also happensthe first Seiken Densetsu game (Final Fantasy Adventure), and the same twist applies: it's the Mana Sword. The remake doesn't let you wield it, but it opens the path to the Dime Tower in both versions of the game.
Pale Justice in Icewind Dale is an extremely powerful sword for paladins, and looks exactly like an ordinary longsword.
A sidequest during the prologue has you retrieving a broken heirloom sword for a villager named Jhonen (no, not thatJhonen). In the last chapter, if you did this quest, you can talk to him again to find out that the sword magically reforged itself sometime while you were off doing the adventury thing. Not being a fighter himself, he's only too happy to hand over the rather nice Reforged Sword of Aihonen to help you out against the Big Bad. And it doesn't stop there, either - if you bring it with you into Heart of Winter, about halfway through you meet an NPC who will upgrade it even further.
Dragon Quest IX features the Rusty equipment, the first piece of which you're likely to find is the Rusty Sword towards the end of the game. It's not much use at first, but add a little glass powder and some Unobtainium (Orichalcum, in this case) and you get the third best sword in the game. And then the rest of the equipment is also the third best of its type in the game. It is very difficult to find anything other than the sword.
Various rusted weapons can be found at a low chance in the Monster Hunter series by mining them or from quest drops. They usually tend to subvert this trope in being generic iron swords, hammers, et al. On the off chance you're lucky, you'll get a "Rusted" or "Worn" weapon which, with a lot of work in getting parts to break the rust, become some powerful weapons. In particular are the Eternal Hate and the Divine Exodus, two Sword and Shield weapons with the highest Dragon attributes in Tri, bar none.
Glory of Heracles often hands you Rusty items. These are useless until you visit a Polisher, who scours the rust off. Usually you get pretty decent gear out of it. Sometimes subverted in that the rust is so far gone all you get is Scrap Iron. The first sword in the DS game gets buffed to an Infinity Minus One Sword before the final dungeon, and one more time by the Blacksmith God, Hapheastus, right before the Big Bad into an Infinity Plus One Sword.
In World of Warcraft one of the most powerful knives in the game when it was introduced looked exactly like a small broken bottle. There are also a few magical maces that just look like unadorned bones.
Many of the Celestial Weapons from Final Fantasy X start off looking this way and being rather useless, but with a little polish, some sharpening, and a number of mini-games, their true worth will be revealed.
In Assassins Creed II, the Sword of Altair (best weapon in the game) is actually the simplest and least-adorned weapon in the entire game, especially when compared with the exceptionally ornate blades and hilts of the other sabers, scimitars, and rapiers Ezio can buy. The only real adornment on the blade is a subtle eagle-head carving in the sword's pommel.
In Lunar: The Silver Star, Althena's Sword (so named for the goddess of Lunar) can only be wielded by her champion, the Dragonmaster—but has been lost since the last known Dragonmaster, Dyne, died. Truth is, it's been hiding in plain sight, crusted over with debris and buried to the hilt in Dyne's monument right outside the (mostly) Silent Protagonist Alex's own village. After proving himself as the next Dragonmaster, Alex is able to draw Althena's Sword from the stone and reveal its true power.
In the sequel, Lunar: Eternal Blue, Althena's Sword is being carried by once-Team Pet Nall, now grown up and in human form. Astute gamers will recognize it right off the bat, even though the distinctive carved golden handle is wrapped up in bandages. Nall later deems not-so-Silent Protagonist Hiro worthy of wielding it, despite him not technically passing the Dragonmaster trials.
Dark Souls starts you out with a broken sword hilt. It deals pitiful damage and you get a better weapon nearly immediately, but if you hang on to it and forge it with a particular boss soul, it becomes one of the most powerful weapons in the game. Demon's Souls featured a similar weapon constructed from an equally useless sword hilt.
In Golden Sun: The Lost Age, you can find rusty weapons in certain patches of shallow water in the ocean as soon as you get the ship. If you take these to Sunshine the Blacksmith, he will agree to fix them up, resulting in weapons that are reasonably good for that part of the game.
Mind you, you find a good majority of the first set of rusted weapons when you get into "Triton's pieces/Quick Sand Box" part of the game, and by the time you actually get around to finding more of them you mostly have weapons that overpower the crafted ones. That's not to say later in the game.
In Fire Emblem: Awakening, Priam's Ragnell is visibly worn, with a dulled blade and cracks all over. While not the infinite-use juggernaut that it was when his ancestorIke wielded it, it's still a highly powerful top-rank weapon.
The fact he uses its ranged attack by throwing it at the enemies probably doesn't help.
Appears in Half Minute Hero's Hero 30 mode late in the game. It becomes the ultimate weapon if you refuse to kill Sasha after she becomes a zombie.
Boktai 2 had a ridiculously weak sword, spear, and mallet free for the taking from treasure chests after the first boss. These items also have a chance of nullifying defense, and when that happens their low attack levels do tons of damage against 0 defense.
You could also craft them into moderately average weapons with the same abilities if you were good enough at sword smithing.
In The Elder Scrolls III: Morrowind it's possible to stumble across a unique rusty dagger in a certain daedric ruin. Said dagger later turns out to be Mehrune's Razor, mythical artifact and signature weapon of the local God of Evil, and if you bring it to his shrine he's happy to restore it for you.