Finding indexed redirects

Ok, I've come across two examples in the last week alone where a redirect was indexed, rather than the target page, so I think it's time to talk about a special effort. With all the renaming involved with TRS and namespacing, I suspect there's a whole lot of "missing" indexes out there at the moment.

The only problem is that I think we'll need Eddie's help with this. I don't know of any way to find indexes pointing to redirects without querying the database, which seems to be an Eddie-only function at this point. I don't think it'll be a difficult query to write (though it might be processor intensive, and thus something we wouldn't want to run very often), so basically, I guess I'm asking if Eddie would be willing to help with something like this?

eta: we have a sandbox now, thanks to Eddie's help and jkbeta's. See post 14.

edited 28th Aug '12 3:20:15 PM by Xtifr

There were plenty of redirects on the Hentai index when it got purged. So, this is a real issue.

Is there any way to get started on this without Fast Eddie's help? I admit it's silly, but I hate P Ming him when he's got so much already on his plate.

Manually walking down every single index on the wiki?

Don't forget the autoindexed subwikies — if a Playing With page is redirected, then you have the same trouble if clicking through the bottom.

I could write a script for that. This script would call all the pages on an index, and check whether the return address differs fromthe original one. However, it would still take very long. There is only a certain number of pages my script(s) can open per day.

On second thought: All I'd need to do would be calling the source of the indexed pages. If the source contains nothing but a redirect, I know I found what I'm looking for.

But so or so: I think that at the moment, there are more important works to do.

Yeah, I thought about something like that, but it would seem to be a bit abusive of the site's bandwidth. A SQL query should be both faster and more efficient. And, probably, even simpler to write. The only problem is that it can only be done by one very busy man.

Just wondering: Does google index the source code for pages as well as the formatted pages? In that case, one might just google for the markup...

Bad idea. When I do search for something, 90% of what is thrown up is noise.

Indeed, google is not helpful here. That means there's two solutions - either walk all indices by bot, or make Eddie do it.

Here is requested list of index paths containing articles which are marked as redirects. Caution: Will contain false positives due to mislabeled articles.

http://216.151.212.47/indexed_redirs.html

edited 26th Aug '12 6:15:04 PM by FastEddie

Wow, that's great! A little intimidating, but I expected it would be. :)

Looks like there's some pages marked as indexes that probably shouldn't be, like Administrivia.How Indexing Works, which has a pagetype of index+index and Administrivia.What Goes Where On The Wiki, which has a pagetype of admin+index. Neither of those seems to have an actual index section.

eta: there's also some false hits for whatever+index pages that refer to redirects but don't actually index them. Which is fine; the majority of those are namespace related and should be cleaned up anyway. Anyway, we should figure out how to divvy these up. Maybe I'll make a sandbox. Tomorrow.

edited 27th Aug '12 2:30:05 AM by Xtifr

These are 5718 items, for the record. I'll try and put it into one or more pages.

Is 256976 too much for one page?

edited 27th Aug '12 3:11:26 AM by SeptimusHeap

Built a list page: Sandbox.Indexed Redirect

I'll take No. 6.

ETA: What to do with Our Ghouls Are Creepier? It's generating a wick to radiohead

ETA ETA: Number 6 is done, except for that Troper Tales page.

edited 27th Aug '12 5:14:55 AM by SeptimusHeap

We may want to discuss some of these. For example, I'm pretty confident that YMMV Redirects should not be cleaned up. Maybe it should be converted from index to list, but since it's probably used for YMMV flagging, I'm hesitant to do so without Eddie's ok. So we should probably leave it alone for now.

But in general, it's worth keeping an eye open for pages that shouldn't be indexes in the first place, like the Administrivia pages I mentioned earlier.

eta: I'm going to take a stab at section 3, as it has a lot of stuff that's relevant to my interests.

edited 27th Aug '12 3:56:23 PM by Xtifr

I had removed YMMV Redirects from the list exactly because it doesn't need fixing. We might try and de-index though after testing.

Quick question: if we fix the redirection, then do we delete it from the Sandbox list?

Yes, please.

All the sandboxes got wiped in the recent sandbox purge. I'm wondering if it would make more sense to ask for a new list rather than just restoring them? It has been a while.

edited 16th Nov '12 3:04:47 PM by Xtifr

Better to ask for a new list, assuming that it's technically possible.