Proboards Huge Real-Time Code Index!

[uzi]

We weren't saying linking to the codes is disallowed, we were talking about file_get_contents(), crawlers, and other methods.

[Ryan]

So if he is crawling then that should be ok right?
i mean thats what google does and other search engines so if he's just crawling then he's fine

[uzi]

True true, crawling is acceptable (I'd imagine), but the former (file_get_contents() etc) wouldn't be.

[Webworldx]

It's pretty much what google does, and google cache does, so there's nothing "wrong" with it as such. Visiting the page doesn't cause a hammering of the proboards server, it's all stored on the ifcode side

Hope you like enjoy it anyway!

[Guide]

Very nice, will be helpful to new ones, well done

[Chris]

And this is why James is still considered in the "God" ranking of PB coders.

[Ðravu]

Jan 26, 2008 2:31:16 GMT -8 Webworldx said:

It's pretty much what google does, and google cache does, so there's nothing "wrong" with it as such. Visiting the page doesn't cause a hammering of the proboards server, it's all stored on the ifcode side

Hope you like enjoy it anyway!

But Proboards (or rather Martyn) has repeatedly said no to codes harvesting information like that, even if it's been stored on a different server to reduce resource usage. He even rejected CJ's idea of using a code to grab the replier information for threads, even though there aren't any ads on it and the resource hit for the code would've been extremely low. Allowing you to do stuff like this and nobody else because "you're a great coder" doesn't really seem right.

[Anoop]

Jan 26, 2008 11:02:34 GMT -8 Ðravu said:

Jan 26, 2008 2:31:16 GMT -8 Webworldx said:

It's pretty much what google does, and google cache does, so there's nothing "wrong" with it as such. Visiting the page doesn't cause a hammering of the proboards server, it's all stored on the ifcode side

Hope you like enjoy it anyway!

But Proboards (or rather Martyn) has repeatedly said no to codes harvesting information like that, even if it's been stored on a different server to reduce resource usage. He even rejected CJ's idea of using a code to grab the replier information for threads, even though there aren't any ads on it and the resource hit for the code would've been extremely low. Allowing you to do stuff like this and nobody else because "you're a great coder" doesn't really seem right.

Ah, so this guy might get in trouble .

[Webworldx]

Jan 26, 2008 11:02:34 GMT -8 Ðravu said:

Jan 26, 2008 2:31:16 GMT -8 Webworldx said:

It's pretty much what google does, and google cache does, so there's nothing "wrong" with it as such. Visiting the page doesn't cause a hammering of the proboards server, it's all stored on the ifcode side

Hope you like enjoy it anyway!

But Proboards (or rather Martyn) has repeatedly said no to codes harvesting information like that, even if it's been stored on a different server to reduce resource usage. He even rejected CJ's idea of using a code to grab the replier information for threads, even though there aren't any ads on it and the resource hit for the code would've been extremely low. Allowing you to do stuff like this and nobody else because "you're a great coder" doesn't really seem right.

Dravu, there's no difference between writing each thread URL out individually into a file and doing what I've done. By your explanation, no code index anywhere would be valid, or copying any thread url and pasting it on any web server.

Just be happy it exists and hopefully is helping people!

[Ðravu]

Jan 26, 2008 16:02:13 GMT -8 Webworldx said:

Jan 26, 2008 11:02:34 GMT -8 Ðravu said:

But Proboards (or rather Martyn) has repeatedly said no to codes harvesting information like that, even if it's been stored on a different server to reduce resource usage. He even rejected CJ's idea of using a code to grab the replier information for threads, even though there aren't any ads on it and the resource hit for the code would've been extremely low. Allowing you to do stuff like this and nobody else because "you're a great coder" doesn't really seem right.

Dravu, there's no difference between writing each thread URL out individually into a file and doing what I've done. By your explanation, no code index anywhere would be valid, or copying any thread url and pasting it on any web server.

Just be happy it exists and hopefully is helping people!

Actually, there is. You're viewing the ads when you go to copy and paste which I think is the reason the admins didn't like automatic source grabbing to get the information. Why are you explaining to me anyways? I wasn't the one who said you couldn't do it. I am happy it exists, but am not happy that you can do what you want while all other coders have to go by the unwritten TOS rules.

[uzi]

He could view the ads and then use the crawler. o.o
He doesn't have to manually visit each link I'd assume, just the indexes of each thread.

[Ðravu]

Jan 26, 2008 19:02:40 GMT -8 uzi said:

He could view the ads and then use the crawler. o.o
He doesn't have to manually visit each link I'd assume, just the indexes of each thread.

He could. He could go to each page and use a Javascript code to grab all the links and such since that's client-side and he'd still be seeing the ads and all that stuff. But in what he's doing, he's hitting pages without viewing the ads and that's something the admins have been strongly against for awhile. They won't even let you use those kinds of codes when you have ad-free.

[Webworldx]

If it's a huge problem, I can go recreate the project in AIR/Apollo and do it all client side and still have the same resulting files at the end, but haven't heard anything from Martyn yet saying this isn't allowed - so i'll just leave the post up for now.

Thanks everyone so far who liked it! I fixed a few bugs in it today with the SSD index not showing.

[VS Admin]

Jan 26, 2008 17:14:25 GMT -8 Ðravu said:

Jan 26, 2008 16:02:13 GMT -8 Webworldx said:

Dravu, there's no difference between writing each thread URL out individually into a file and doing what I've done. By your explanation, no code index anywhere would be valid, or copying any thread url and pasting it on any web server.

Just be happy it exists and hopefully is helping people!

Actually, there is. You're viewing the ads when you go to copy and paste which I think is the reason the admins didn't like automatic source grabbing to get the information. Why are you explaining to me anyways? I wasn't the one who said you couldn't do it. I am happy it exists, but am not happy that you can do what you want while all other coders have to go by the unwritten TOS rules.

There's a huge difference between this and putting a code on your board to grab extra data.

If you added a code that grabbed information from another page from your board, you would then essentially be sending double the requests to our servers. Every single pageview becomes two.

In a system like this, he could load maybe 60 pages once a week and have a full database listing available for you to search with.

Patrick

[SwordGrunt]

Nice, but I can't see all 535 PBS codes in there