Read the rules before proceeding!

Topic: e621 updater - tag local files, download favorites, find posts for saved images!

Posted under e621 Tools and Applications

Just found this tool that does exactly what I was looking for: uploading saved images to favourites. Nice job!

Just one suggestion mentioned earlier in this thread but not addressed - logging of files that aren't found on e621. It'd be really helpful to know which few files out of hundreds haven't been favourited.

Updated by anonymous

Dirtpaw said:
It'd be really helpful to know which few files out of hundreds haven't been favourited.

It's actually not that easy to do because AFAIK server returns nothing after POST request. I'd have to check if amount of favourited pictures has changed after each request or batch check them after process finishes... I'll think about it. :)

Updated by anonymous

I implemented something like that as part of my system.
Basically, it does the fav:USERNAME query, fetching all pages, and then extracting just the URIs. It then reads a list of md5sums corresponding to on-disk files (this is cached, and updated as needed by digup )
It iterates over the list of URIs; for each one it extracts the md5sum. The URI in question is output if its md5sum is not in the list of on-disk files.

So what I implemented is basically the opposite procedure, but is simple to invert. Hope that helps give you some ideas.

Updated by anonymous

We are probably not affected by "Crawlers, Bots, Page Numbers and You." change (unless someone has more than 750 pages of favorites, huh), but feel free to report if something is not working right.

Updated by anonymous

Hi Keito,

Thanks for putting this together, but I don't seem to be able to get tags on my images. Also, it seemed to rename one of my files into oblivion and it disappeared (not the "NotFound" file, but the one starting with hash "00d").

Wednesday, July 11, 2018
14:26:25 : Proccess started
14:26:25 : Tagging mode
14:26:40 : After 0:00:15: 22/22 images processed. 0/22 images updated
Wednesday, July 11, 2018
14:26:57 : Proccess started
14:26:57 : Tagging mode
14:27:36 : After 0:00:39: 22/22 images processed. 1 images not found
Wednesday, July 11, 2018
14:32:14 : Proccess started
14:32:14 : Tagging mode
14:32:28 : After 0:00:14: 21/21 images processed. 0/21 images not found
Renamed A:\Users\Ket\Desktop\new pron copy\00d89c259ac3083408350a08ec5f7c94.jpg to .jpg
Renamed A:\Users\Ket\Desktop\new pron copy\044197f543a763be4a583d75c8cb76a7.png to A:\Users\Ket\Desktop\new pron copy\044197F543A763BE4A583D75C8CB76A7.png
Renamed A:\Users\Ket\Desktop\new pron copy\0a6d72a5a68f3cb67556f2ebfb59b290.jpg to A:\Users\Ket\Desktop\new pron copy\0A6D72A5A68F3CB67556F2EBFB59B290.jpg
[...]
Renamed A:\Users\Ket\Desktop\new pron copy\ed4d8102d520504513630014d8565479.png to A:\Users\Ket\Desktop\new pron copy\ED4D8102D520504513630014D8565479.png
Renamed A:\Users\Ket\Desktop\new pron copy\f1c586633a553be461851e818436fe42.jpg to A:\Users\Ket\Desktop\new pron copy\F1C586633A553BE461851E818436FE42.jpg

I notice you have a GitHub for this now. How do I compile these AHK files? I've never heard of this extension.

Thanks! =}

Updated by anonymous

Better late, than never!

Ket-Ralus said:
Thanks for putting this together, but I don't seem to be able to get tags on my images. Also, it seemed to rename one of my files into oblivion and it disappeared (not the "NotFound" file, but the one starting with hash "00d").

For me the tagger still works fine, are you sure your connection to e621 is direct? Check windows proxy settings and if you use proxy\vpn to access e621. It should be working if there's nothing wrong. Make sure you're not searching for tags in PNG images - only few programs support them (but they are actually there). Check if Windows sees tags in JPGs. And regarding your "renamed" file - i can't reproduce it with the same file. It was only one?

Ket-Ralus said:
I notice you have a GitHub for this now. How do I compile these AHK files? I've never heard of this extension.

You can get a compiler and read more about it here: https://autohotkey.com/

-----------------

Sync mode not working fixed in version 9.3.
I don't login often anymore, so PM me if you have any problems - i'll get a email then.

Updated by anonymous

i still cant get sync mode working, tried on multiple accounts and multiple folders.

Updated by anonymous

If sync mode is not working, try checking your API key field: it should only contain your API key and nothing else.

Updated by anonymous

It seems it IS possible to store EXIF data in PNG. ftp-osl.osuosl.org/pub/libpng/documents/pngext-1.5.0.html#C.eXIf

Updated by anonymous

milanise7en said:
It seems it IS possible to store EXIF data in PNG. ftp-osl.osuosl.org/pub/libpng/documents/pngext-1.5.0.html#C.eXIf

EXIF is already added by my tool: the problem is that not all programs can understand that. Many just don't even try to search for tags, even when they are still there. Windows Explorer is a good example. It won't even show "tags" field for PNGs, while Picasa and Photoshop know that they are there.
You can tag PNGs, you just have to use proper software to make use of them.

Updated by anonymous

Hydrus seems to be my replacement of choice for this, along with it replacing Picasa.

(I know I'm necroing a 2 month old thread but it's worth looking into for anyone needing tag sync / subscription puller solution)

Updated by anonymous

Can this download pictures based on tags I choose and not based on someone's favorites list?

Updated by anonymous

Serkan said:
Can this download pictures based on tags I choose and not based on someone's favorites list?

Nope. I've used e621dl when i needed that functionality. I feel like coding it into my program is excessive, since there is quite a lot of programs out there that can do it for you.

LibertarianHorseFukr said:
Hydrus seems to be my replacement of choice for this, along with it replacing Picasa.

Looking good, i'll stick to using mine and image viewer of choice to avoid locking myself into one specific program. It seems that it's using some kind of internal database, while my program is tagging files themselves. Thanks for the bump! ;)

Updated by anonymous

I am getting an error when I try to do anything. Everything was working perfectly a couple months ago.

Error:Expecting JSON value (string,number,true,false,null,object or array)

Line: 1
Col: 1
Char: 1

specifically: <

line#
134: Throw,Exception(msg, offset, SubStr(text, pos, len))

Updated by anonymous

9.4 IE Doomsday

  • Requests from old IE are now denied by e621.net (it seems like this)
    • Had to change all requests to use curl and send custom useragent to get API data.
  • Here's what this means for you:
    • MD5 mode is disabled (can't get MD5 from files properly, program only works with MD5 in filenames, at least for now). It was turned off by default, but if you've used it, that's bad news for you.
    • Tagger tested on limited amount of picture and works OK
    • Updater tested on even less amount of pictures and works OK (but that's not 100%)
    • Sync and Downloader modes are completely disabled for now. There's a lot of code to rewrite to make them work, i'm not ready to do that now.
    • Overall speed is reduced.

Downloads are in OP post.

Updated by anonymous

Tool is awesome! What app would you recomend for keeping viewing your stash/gallery?

Updated by anonymous

Keito said:
Nope. I've used e621dl when i needed that functionality. I feel like coding it into my program is excessive, since there is quite a lot of programs out there that can do it for you.

Looking good, i'll stick to using mine and image viewer of choice to avoid locking myself into one specific program. It seems that it's using some kind of internal database, while my program is tagging files themselves. Thanks for the bump! ;)

Yeah, it's using SQLite for it's database files to match file hashes with associated tags. All readable if you really want to go rooting around in them.

The API system is super primitive still. Mainly gets used for pushing URLs to the client software from browser extensions.

Updated by anonymous

notaroundhere said:
Tool is awesome! What app would you recomend for keeping viewing your stash/gallery?

Picasa or Lightroom.

Updated by anonymous

Alert caution warning, incoming noob question:
Downloader mode and Sync mode appear greyed out, what do i do?
I tried manually adding Username and API to the .ini
MEGA 9p4

Updated by anonymous

Keito said:
Picasa or Lightroom.

Picasa has been killed by Google and Lightroom cannot be purchased on certain countries.

Updated by anonymous

Fedfed said:
Alert caution warning, incoming noob question:
Downloader mode and Sync mode appear greyed out, what do i do?
I tried manually adding Username and API to the .ini
MEGA 9p4

Unfortunately, they are not working now. I've had to go back from using IE to all-CURL, so half of the tool should be rewritten. I don't have time for that now.

milanise7en said:
Picasa has been killed by Google and Lightroom cannot be purchased on certain countries.

You can still download it if you really want.

Updated by anonymous

Hope this gets updated for the new site at some point, very useful program for organization. Just curious if there is any progress on that. I can wait though, I'm patient.

adnf2012 said:
Hope this gets updated for the new site at some point, very useful program for organization. Just curious if there is any progress on that. I can wait though, I'm patient.

I'm working on a similar tool because this doesn't work anyore, mine is still in progress tho. I finished it shortly before the api rework lol..

I fixed tagger mode through some dirty haks. https://mega.nz/folder/IMYQkQyT#kR-iboAlVIp4WIQYo99waw
My version write tags from artist, general, species, character and copyright groups. Other modes probably still broken. Maybe will look into it later.
-----edit-----
Probably fixed downloader mode.
-----edit2-----
Probably fixed all modes. Tagger, downloader and sync modes was tested on small amount of files. Updater mode not yet tested.

Updated

  • 1
  • 2