Proboards Huge Real-Time Code Index!

[Wormopolis]

Feb 24, 2009 15:00:39 GMT -8 Webworldx said:

SK-Bundy, Of course, the more the merrier, as long as the codes aren't repeated anywhere else that we're indexing, otherwise it just causes duplicates

Storing codes in:
wormocodes.proboards.com/index.cgi?board=codedb

there shouldnt be any duplicates.. none of my codes ever made it into PBS codebase (<-- noob).

EDIT:
I retract that statement because I just saw Triad added 3 of mine in. The watchdog code is duplicated because it has a different thread title in each codebase. should I just make them the same to eliminate the duplicate?

[Webworldx]

Added m8

[Webworldx]

Feb 8, 2009 9:59:06 GMT -8 Chris said:

I'll take a shot at some suggestions/ideas...

9) Can you add forum.smangii.com/index.cgi?board=pbsupport ? They also provide at least a decent amount of codes. (Guessing like 300 or so)

10) socal.proboards26.com/index.cgi?board=PBCODES Though the database doesn't get updated anymore, it contains at least 100 unique codes to all PB sites. The owner California has passed on, so I'm the one left in charge.

That's about it from a quick glance. Still really like what you've done with it WWX.

Added these two to the index too now

[Chris]

Danke.

[Charles Stover]

How often does it crawl the DBs?

Also, can it crawl non-ProBoards sites?

[Webworldx]

It could, if necessary.

Crawls every couple of days at the moment.

[Charles Stover]

What about the grouping of codes with the same name?
So you don't have:

96 #43 Advanced Hack Manager v1.2 Chris
97 #194 Advanced Hack Manager v1.2 Chris [Busy]
98 #1006 Advanced Hack Manager v1.2 Chris
99 #1470 Advanced Hack Manager v1.2 Chris

[Webworldx]

How would you suggest going about it?

I'd imagine a "check next row" on the while loop would do it, but it seems inefficient to check the next row of each record in case it's a duplicate.

The other option would be to disallow duplicates and remove them from the index completely, but that's the wrong direction I think.

[Charles Stover]

Is it MySQL?

GROUP BY `id`

But I don't use grouping functions much, so you'd have to screw around with it. I'd recommend trying it out in phpMyAdmin before updating the official code.

[Webworldx]

Apologies for anyone currently looking for the code index. My lovely host seems to have encountered a problem they promise they'll solve soon!!

I'll let everyone know when it's back up and searching

[Michael]

dntresources.proboards.com/index.cgi

Can that be indexed too please? It's a small index at the moment, but will continue to grow! Cheers.

[D5K0]

Apr 14, 2009 3:41:48 GMT -8 Michael said:

dntresources.proboards.com/index.cgi

Can that be indexed too please? It's a small index at the moment, but will continue to grow! Cheers.

Lol, "Hello World".

I think this index is a great idea and will decrease the amount of threads asking for codes that are already out there. Thanks.

[Michael]

Huh @ Hello World?

[D5K0]

wimg.co.uk/HQsfwk.png

[Michael]

Ah