Topic: t621 - another e621 downloader (Windows only)

Posted under e621 Tools and Applications

Seeing groundwalker providing his downloader tool for e621, I thought this might be a good opportunity to provide you with mine.

It is called t621, where "t" stands for thin client.

Download-Link

FEATURES
- Can add multiple tags (see the Tagging Help on e621.net.
- Has an automatically updating preview window for the downloaded files of the current session (can also play flash and webm movies).
- Download folder is customizable.
- Makes use of the XML-API web service.
- Detects already downloaded files in the download folder and skips them.

t621 only runs on Windows!

I developed this while having a chat with Varka via email, quite some time back. We agreed to:
- Limiting a download session to 200 files, so it won't be abused to download bachillions of files and take too much server bandwidth. You can simply restart after these 200 files to pick the session up and continue.
- Leaving a fingerprint/user-agent while communicating with the web services.

It has been tested on Windows XP, Windows 7 and Windows 10. It might run on other versions, but since I don't own these, I probably won't be able to provide OS specific support.

If you have any feedback or found any bugs, please drop me an e-mail (see help-button in the app) since I usually don't check in here too often, forgive me for that. If I receive an e-mail and think that it's a big issue, I will make a public statement here.

I could swear I already posted it in this forum, but I couldn't find it using the search.

Oh, one last thing: I am not trying to compete with groundwalker, and I don't want to "steal" his users! I think that we should benefit from each others experiences. Maybe feedback on his app will help me, maybe feedback on mine will help him. Try both out, see which one works best for you. In a perfect world we will have a talk at some point and make a new version together, bigger, better, and all these other awesome shenanigans.

I hope you guys and gals find this useful.

Recent changes:
- Adaptation to the most recent e621.net API changes.
- Added support for webm previews by using VLC components. Up until now the app crashed whenever a webm should've been shown.
- Switched to .NET 4.0 to use VLC components.
- Minor UI improvements.

Updated by jonnyawsom3

No, Its Cool.

I wish I knew about the api before spending time coding my program to do silly stuff to get what i could have gotten from the api. Oh well. I used my downloader and got all of the images off of this site and I just wanted to share.

Updated by anonymous

I started off like that as well, doing a lot of ugly string operations on the returned HTML from the post pages, until I eventually found an API reference in the help section. Boy was I glad. >_>
If I ever release another version of this app, I will use the JSON API instead, since it's smaller data compared to XML.
It's a shame I'm too dumb to find an API on FA. I would really like to extend my program for that page, but I'm really not interested in even more HTML parsing right now.

Updated by anonymous

Yeah, I'm kind of limited on time, I'm going back to college. So I really haven't gotten to structuring my code to limit the amount of requests to the prefered range. (I'm not even sure that the time complexity for my method takes care of that already). But did add the user agent connection.

And I'm not to sure if i want to build of the current code I have. Instead make it use the api instead. But alias, It will have to wait until next year.

Updated by anonymous

Munkelzahn said:
FA doesn't have an API

The thread is from 2011, but AFAIK FA still doesn't have an API.

I've seen that thread. And given the FA server performance, I'm not too eager to download the whole HTML page just to get a few lines of values. At least not now. I too have things to do. ;3

Updated by anonymous

Munkelzahn said:
FA doesn't have an API

The thread is from 2011, but AFAIK FA still doesn't have an API.

FA does'nt have an API!? post #125369 I hafta agreed with Aden, browser requests take more bandwidth (document, scripts, styles, images) than separate requests (only the useful stuff).

Updated by anonymous

//EDIT
I deleted all my previous reasoning about FA vs API, so this won't explode here. I would like to dedicate this thread to t621 and nothing else.

Updated by anonymous

Furaffinity has an eager detection of bandwidth over usage...

Could you tell me the exact specifications of what you agreed on with Varka?
I am currently working on my own implementation of a downloading, uploading and tag changing suit for e621 and don't want to step on any toes.

On a sidenote, to me WPF GUIs have the same touch as Java GUIs. Makes me feel violated after every usage and obviously in the end it's not necessarily a reliable YSWYG technology, it's broken as fuck (folder and tag texts are cut off at the bottom,selection line of tags touches them, scrollbar is squarecut and protrudes over the rounded edges, scrollbar grab thingy looks strange, if you select on line and then go down you still see the last pixel line of the selection) on Windows 8, something I didn't experience with winforms.

Updated by anonymous

@Traubenfuchs: it was practically just that 200 file download limit, and having that fingerprint. But I'd really suggest that you contact the staff yourself, just to be sure.

I got an email yesterday by justavisitor. Since the last e621 update, t621 didn't want to download anymore. I released a bugfix. Just re-download the program from the link above and you'll have it.

Updated by anonymous

This site stays alive because of ad revenue. Is it really the best idea to encourage downloaders like this?

Updated by anonymous

Foobaria said:
This site stays alive because of ad revenue. Is it really the best idea to encourage downloaders like this?

The admin's don't seem to think of it as being a bother. I'm pretty sure that the downloaders ( not one hundred percent sure, someone correct me if I'm wrong ) have to crawl the site very much like a bot, and thanks to that, each page they go through will be popping an ad.

I could be very, very wrong, but we have to assume something like that is in play if they haven't been taken down or requested for removal yet, right?

Updated by anonymous

Shaye said:
The admin's don't seem to think of it as being a bother. I'm pretty sure that the downloaders ( not one hundred percent sure, someone correct me if I'm wrong ) have to crawl the site very much like a bot, and thanks to that, each page they go through will be popping an ad.

I could be very, very wrong, but we have to assume something like that is in play if they haven't been taken down or requested for removal yet, right?

If the downloaders use the e621 API, they can bypass the HTML and thus the ads.

Updated by anonymous

EDFDarkAngel1 said:
Hey everyone. Downloaders are okay, with certain conditions. Varka lays them down here: https://e621.net/forum/show/80071

That was informative.

Munkelzahn said:
If the downloaders use the e621 API, they can bypass the HTML and thus the ads.

I see. Hm. I don't know much about all this stuff, so I have to go on guesses, educated or not.~

Updated by anonymous

Something's wrong with t621. I can't download past first 200 pics of any search. No matter how many times I restart it still immediately tells me that downloading ended without downloading anything new. This could be bypassed by splitting search, but there's another problem: t621 is unreliable even on searches that have less than 200 pics.

I had to restart votedup:Granberia inpool:false -fav:Granberia -rating:S teen_titans 3 times to get all 9 pics. After first time it stopped immediately without downloading anything.

What's even worse it's that unlike while trying to download over 200 pics there is nothing to suggest that it didn't download every pic. It took me a while to realize.

Edit:
Now it doesn't want to download anything past 6 pics of votedup:Granberia inpool:false -fav:Granberia -rating:s how_to_train_your_dragon no matter how many times I restart. I even change dest folder to check whether it causes the issue, but no.

Updated by anonymous

I have updated t621 to support the recent API changes on e621. I've also added support for webm playback and fixed some UI issues. Link in the first post is updated.

Updated by anonymous

So when I try to use this program ever since I moved from Windows 10 back to 7 on a fresh install, I get a .net framework error that dissapears and closes the client before I can see it, then I get the windows problem signature or something.
What I do before I get the error is everything like normal, start it up, add a tag, add where I want my files to go and when I click start, I get a .net framework error.

Updated by anonymous

Does it still work? And is there a way to make it work with proxy (e621 is banned in my country)? I don't know if I'm doing something wrong with proxy or if this app doesn't work anyway.

Updated by anonymous

I love the app (been using for a good chunk of time), except when it comes to downloading pools; small ones aren't so bad, but large pools are... problematic when it comes to numbering. Would it be possible to add some extra naming options for pools and such?

I bring this up here because this is the only reliable image downloader for this site that I've found as others just don't work or have tossed issues at me in the past.

Updated by anonymous

So I gave this a shot but just like Zaryss said, I open the program, set the download location, add a tag and hit start downloading. Immediately after, I get an error. image link below

a network error. I have windows 7.
- I tried running it with admin privileges
- tested it by using a tag that had less than 10 pictures and only jpeg files (in case the program has issues with larger amounts or other file types.
- Updated to the most recent version of Net Framework.

Not sure what else to do. Did the website update again since the last time you replied to this thread?

This seems like a really good program. Hopefully I get a reply.

Thanks.

http://s33.postimg.org/3kwqb0j8v/download_error.png

Updated by anonymous

I got it working fine! (sorta)
I've got a problem with bulk image downloads and this program. it's only able to download 200 images at a time (which I think should be an option in which we can change... {maybe plz? :( })

also having to reset every 200 images is almost an annoyance when you do have those massive downloads you want to do. (if it took a little bit longer in between downloading and hitting the 200 mark i might have been fine with it but it would have been slower lol)

I've gotten to the point where I'm downloading apparently a set of tags too large for the thing to download. it got the first few batches of 200 images done fine! but then after those it just stops downloading 200 and just ends up only downloading about 20 or so. and i continued with that. it's gotten to the point where it won't download any images at all anymore even though there are still a lot to go. i was wondering if there was someway to fix this?

tags: aaron_(artist) rating:e -comic

Updated by anonymous

Okay, so I figured out what was really up. I think I'm just putting too many images in the downloads folder where I put everything when I do download a tag. I tried some different tags and it seems to be having the same problem. maybe moving the images from the temp storage area will fix this. more to come later.

Edit: I did download several thousand pictures with this so sorting through those will take time. results will come much later

Updated by anonymous

So what is that, some weird filesystem-based limitation? Or does t621 have an internal timeout that is passing (takes too long to read the directory)?

Updated by anonymous

Sorry for this stupid question, but what do I do next once I download the zip file? I ask because I assumed that I just needed to open the t621.exe file and get started. But when I try to add tags the program crashes. (I'm using it on Windows 10) I would appreciate any help.

Updated by anonymous

Quasarbooster said:
Sorry for this stupid question, but what do I do next once I download the zip file? I ask because I assumed that I just needed to open the t621.exe file and get started. But when I try to add tags the program crashes. (I'm using it on Windows 10) I would appreciate any help.

I had the same problem upon just opening the downloaded file<exe directory, then extracted the files to an additional folder , opened the exe and then it worked but I'm sure you figured that within those 3 months your comment was ago.

Updated by anonymous

Tested 07/27/2018 and working fine. If you want to blacklist something simply tag -subject to remove it from the search queue. I've had issues with 5 or more tags though so do be aware of that.

It will download in batches of 200 at a time. You need to close the download window after completion then download again with the same tags and directory to continue downloading more than 200.

Updated by anonymous

27/03/2019 and it's working perfectly. just have to re-extract it to be able to download anything after the 200 file limit

Updated by anonymous

IS there any known issue with t621 not finding all posts?
for example when i use it for the tag: mrsafetylion
it downloads 37 files rather then the 64 tagged files

Updated by anonymous

shyvor said:
IS there any known issue with t621 not finding all posts?
for example when i use it for the tag: mrsafetylion
it downloads 37 files rather then the 64 tagged files

1. there's 77 posts by mrsafetylion

2. it's probably just taking ages to load so i'd leave it running and if it isn't done within an hour then something's broken

Updated by anonymous

still working, it may be deprecated but it's still one of the best i've used (apart from the 200 file limit, and when it slows down so the exe needs to be restarted, otherwise it misses posts and just stops working, presumably because of caching or something)

Updated by anonymous

Just curious if this will be revamped with the new updates to E6.

It doesn't seem to work anymore, which is a huge shame. Really made downloading galleries really easy.

Yeah something about the new updates to the site have broken t621...
Any chance in seeing an update?

Until author fixes it, you could try mine. You'll have to visit discord.gg/7ncEzah though, don't have a download link elsewhere yet.

If you only want to download, when it asks to enter api key, just close the window.

Updated

  • 1