i am ripping e621 right now and will seed when finished, i will do so once a year

You should probably read this thread:

http://www.e621.net/forum/show/1625

I'd appreciate it if multiple people didn't all rip the site at once (though thank you for letting me know) - all this activity is causing a noticeable increase in resource usage, and that only slows the site down for everyone (not to mention costing more money on the bandwidth bill). CPU usage is up about 20-30% above normal, and bandwidth is about 10Mbit higher overall.

If you could restrict your transfers to single-threaded (ie, one file at a time) and limit your speed to 2Mbit/sec, that'd be greatly appreciated. That'll help keep the site running quickly for everyone =)

Again, I don't mind people ripping the site, just please do so responsibly - and please don't do it if there's another perfectly usable copy available already.

Thanks <3

dont worry im going to make it 50 kb a second max.

Varka, any way to limit the bandwidth per user?... 2Mbits/sec is still pretty high. :(

There are various options open to us if we wish to limit user's bandwidth use; however I only really want to do this if we have to. For now, there's enough CPU and bandwidth to go around.

If individual users start using very large amounts of bandwidth then I'll have to go and rate limit them, but for now - as long as it doesn't make the site load like crap, or use a ton of bandwidth, I'm fine with it.

We'll see, though.

Lol, what have I done!? Hopefully everyone won't want their own personal site rip! I've been running scripts over this holiday weekend so I should have 30k+ items taken down. I'm sorry for the troubles!

i failed and riped 8 gbs of pure coding (hates life) 8(

It's alright, it wasn't an easy rip. I had hoped that the whole thing could have been done in one smooth gesture, running serial operations out of one script. I ended up with 4 different scripts running different tasks at different times, spread across several languages.

To those downloading or thinking about it, I suggest that you stretch your anuss' (forward your ports and check your firewalls) so I can shove this huge zipped file right in there; I don't pay an extra ridiculous sum of money towards bandwidth for nothing. Barely getting above 260Kb/s