Topic: if I were to download everything from e621 how large would the folder be?

Posted under General

bitWolfy

Former Staff

https://e621.net/stats seems to have been nuked at some point. Probably happened during the upgrade.

Crossed the 1TB mark in 2017. Considering the number of posts doubled since then, it's probably somewhere around 2TB right now.

kiranoot said:
Roughly 2.2TB

so would it just be possible to get a 2 tb ssd and download the entire site? Like with being able to search and all that?

bitWolfy

Former Staff

citrus-eel said:
so would it just be possible to get a 2 tb ssd and download the entire site? Like with being able to search and all that?

Sure, assuming neither your ISP nor e621 shuts you down for using up that much bandwidth.
Could probably just run a small script that iterates over post IDs and downloads images.

Pup

Privileged

citrus-eel said:
so would it just be possible to get a 2 tb ssd and download the entire site? Like with being able to search and all that?

You'd have all the images but you wouldn't be able to search as that's more back-end/server side.

The source code for E6 is here:
https://github.com/zwagoth/e621ng

But you'd need to set up a server to have that working along with a copy of the metadata/tags for each post.

Wow.

I was thinking it'd be so much more. At least in the 10's of TB. If not even more.

You might want to think about compression too, at least for extra backups.

crystali_cast said:
Wow.

I was thinking it'd be so much more. At least in the 10's of TB. If not even more.

You might want to think about compression too, at least for extra backups.

It would take me three years to download all of that, so it’s significantly more than you think.

  • 1