Topic: e621 Batch Reverse Search (updated for new e621)

Posted under e621 Tools and Applications

Tool for people who 'collect' images from all over the internet (or Telegram chats) and now want to find bigger versions of them or just e621 post links.
Uploading them one by one to iqdb.harry.lu would take so long so I created a tool right for this job!
It basically scans a directory, uploads images one by one and collects e621 post links.

Download

(script has autoupdate build-in, in case of future updates and fixes)

Github: https://github.com/jacklul/e621-Batch-Reverse-Search/

Requirements:

- Windows: Visual C++ 2012 Redistributable (x86) - download and install: https://www.microsoft.com/en-us/download/details.aspx?id=30679
- Linux: PHP library - "sudo apt-get install php-cli"

Basic usage:

- unzip it
- put all your images in 'images' folder
- run it with run.bat file (run.sh file on Linux)
- images with matched posts will be moved to 'found' folder, images that did not match will be moved to 'not found' folder
- after all is done check 'found' folder for 'links.html' file - open it with web browser, it will contain all the found links

Small screenshot gallery to show what it does visually: https://imgur.com/a/65Qff

Updated

linicks@tux-linux:~/Downloads/Mass$ sh run.sh
         __ ___  __     ____        _       _
        / /|__ \/_ |   |  _ \      | |     | |    v1.4.0
   ___ / /_   ) || |   | |_) | __ _| |_ ___| |__
  / _ \ '_ \ / / | |   |  _ < / _` | __/ __| '_ \    Created by Jack'lul
 |  __/ (_) / /_ | |   | |_) | (_| | || (__| | | |       jacklul.com
  \___|\___/____||_|   |____/ \__,_|\__\___|_| |_|
  _____                                   _____                     _
 |  __ \                                 / ____|                   | |
 | |__) |_____   _____ _ __ ___  ___    | (___   ___  __ _ _ __ ___| |__
 |  _  // _ \ \ / / _ \ '__/ __|/ _ \    \___ \ / _ \/ _` | '__/ __| '_ \
 | | \ \  __/\ V /  __/ |  \__ \  __/    ____) |  __/ (_| | | | (__| | | |
 |_|  \_\___| \_/ \___|_|  |___/\___|   |_____/ \___|\__,_|_|  \___|_| |_|

Checking for updates... up to date!

Scanning for images... 26 found!

[1/26] Searching '8ecf44162f7f42e21d6a1abb49da703a.png':
 Trying md5 sum...Press ENTER key to continue...run.sh: 10: read: arg count

I'm not exactly sure what this means, but there's no kind of error log I can see telling me what's wrong. I'm using Lubuntu 16.04-LTS with

PHP 7.0.8-0ubuntu0.16.04.2 (cli) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.8-0ubuntu0.16.04.2, Copyright (c) 1999-2016, by Zend Technologies

Also, sorry the error code got all squished... Damn dText formatting...

Updated by anonymous

Linicks said:
I'm not exactly sure what this means, but there's no kind of error log I can see telling me what's wrong.
(...)
Also, sorry the error code got all squished... Damn dText formatting...

Tracked down this issue, package 'php-curl' is also required for this tool to work! Update pushed!

Updated by anonymous

So I got it working, but I do wonder... is there a way that instead of having the shell script append the links to an HTML file so the user can manually click them, you can just have the script use curl to download the pictures from E621 directly? Would this break the e621 API rules?

Updated by anonymous

Linicks said:
So I got it working, but I do wonder... is there a way that instead of having the shell script append the links to an HTML file so the user can manually click them, you can just have the script use curl to download the pictures from E621 directly? Would this break the e621 API rules?

Definitely doable, I doubt there is a rule about batch downloading, considering there are mass downloader tools there. :P

Updated by anonymous

Apparently line 10 of run.sh is throwing a argument error... It doesn't break the script, but I removed it, and it just drops me back to linicks@tux-linux: ~$ when it's done doing it's thing.

Have you considered starting a GitHub repo for this project? This is actually pretty cool.

Updated by anonymous

Linicks said:
Apparently line 10 of run.sh is throwing a argument error... It doesn't break the script, but I removed it, and it just drops me back to linicks@tux-linux: ~$ when it's done doing it's thing.

Have you considered starting a GitHub repo for this project? This is actually pretty cool.

Try changing that line to

read -p "Press ENTER key to continue..." key

and see if that works.

Not sure about how legal is bundling packages with PHP runtime on Github...

Updated by anonymous

Interestingly enough, you're free to do with PHP as you will, as long as you express in the readme.md file that it uses PHP technologies.

You can take a look at the Creative Commons Attribution License here: http://php.net/manual/en/cc.license.php

3. License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

1. to Reproduce the Work, to incorporate the Work into one or more Collections, and to Reproduce the Work as incorporated in the Collections;

2. to create and Reproduce Adaptations provided that any such Adaptation, including any translation in any medium, takes reasonable steps to clearly label, demarcate or otherwise identify that changes were made to the original Work. For example, a translation could be marked "The original work was translated from English to Spanish," or a modification could indicate "The original work has been modified.";

3. to Distribute and Publicly Perform the Work including as incorporated in Collections; and,

4. to Distribute and Publicly Perform Adaptations.

5. For the avoidance of doubt:

Non-waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License;

Waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and,

Voluntary License Schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License.

Updated by anonymous

You can save an enormous amount of time by batching your md5 searches into groups of 100. the format is md5:abc123,483bbc,433,34 Basically it allows up to 100 items split by commas.

kiranoot said:
You can save an enormous amount of time by batching your md5 searches into groups of 100. the format is md5:abc123,483bbc,433,34 Basically it allows up to 100 items split by commas.

This is great, thanks.

  • 1