Topic: Question about tag ripper script

Posted under General

Hi, I've downloaded the site rip that was done on forum #1625, but it would take a tremendous amount of time to sort through all of that. I've since looked at the API, and wrote a bash script to grab the tags off the site, then sort them accordingly. My question is, would my script cause a load spike or other undesired server-side effects. Thanks!
Link to script: http://pastebin.com/ciVdce2S [If you run linux and have the rip, please don't download and run this too, I'll have a modified version that fetches an xz of all of the 'tag' files after I'm done with the first run.]

I'm currently running it, if I'm causing a load spike, just let me know and I'll kill it. It should take about 8 days to complete, sending one request at a time with 2 seconds of cool-down between requests, so if this works the way I think it should, it'll only be the equivalent of one extra search per two seconds.

Updated

  • 1