This is something I've been working on and tweaking for the past few days that I thought might be of some interest to people here. Basically similar to other reverse image searching tools such as Tineye, but should be much more accurate for e621 - among some other site-specific features. Should be very useful for finding dupe images, as well as finding higher-res copies of images you might have from other sites, and many other uses you may come up with.

Current features/info:
- Image/thumbnail/modified-image searching; should return the e621 page of the image you input.
- Upload directly from an image URL without needing to save to your disk.
- "mask explicit images" button, hiding images marked as explicit until you hover your mouse over them.
- Fast searching; typical image takes less than a second to query my database running on a mediocre server.
- Not heavy-weight; only does a few .xml request on e621 every few hours, and downloading a few thumbs from that at the same time (usually totaling less than a couple hundred kbs of traffic)
- idk some other cool/boring nerd shit.

Planned features:
- Its own favicon+theme (I just copied some of iqdb's and e621's css since I am lazy and bad at designing - so no one would be eyeraped by a black-on-white page)
- Your suggestions?

Link: http://iqdb.harry.lu/

Please post or PM me any comments/ideas/suggestions/bug reports/etc, thanks!

Besides everything looking like it's zoomed out, i've been waiting for this. You should add a feature that lets us input an image's url from another site, so we don't have to save the image first.

1) Make it run for every upload, the uploader must then decide whether to save the image or delete it, if a very high similarity warning pops up. We will get fewer duplicates that way.

2) Make it compare every existing image on e621 with every other existing image with idle time, mark potential duplicates, let users fix it. How many years would it take?

51823611904 comparisons would be necessary to compare all existing images with each other.
If one comparison takes exactly 1 second (including writing the suspicous combinations in the database) and there is only one comparison at a time possible and there is no wait time between comparisons that would add up to:
599810 days =
1643 years

It would be 102 years with 8 parallel comparisons and 0.5 seconds for each comparison.

Time for a server upgrade. Or rent a scientific super computer.

Akkira said:
51823611904 comparisons would be necessary to compare all existing images with each other.
...
It would be 102 years with 8 parallel comparisons and 0.5 seconds for each comparison.

Time for a server upgrade. Or rent a scientific super computer.

It could just compare hashes to find exact matches, and a similar thing to a hash to find similar images. None of that would take 1 second per comparison.

Works pretty reliably. Does super zoom in things though

Raiden_Gekkou said:
everything looking like it's zoomed out

You should add a feature that lets us input an image's url from another site, so we don't have to save the image first.

I don't have that issue. What browser are you using?

And yep URL upload is planned. For now, in most browsers, you can just put a URL into the file field and press enter, then after a second or so Windows downloads and caches it and rehosts it locally from your temp files.

Akkira said:
1) Make it run for every upload, the uploader must then decide whether to save the image or delete it, if a very high similarity warning pops up. We will get fewer duplicates that way.

2) Make it compare every existing image on e621 with every other existing image with idle time, mark potential duplicates, let users fix it. How many years would it take?

1) I am unsure what you mean. It's not explicitly designed for finding dupe images, nor does it access e621's API. There are also many images that have almost the exact same look as other images, but with slight mods (see https://e621.net/data/de/44/de44ec3ac080c4875f5c381f277545c5.png for example).

2) It wouldn't actually take that long (it usually takes less than a second for 32 different db queries on a server from 2003), but see point #1 :)

I think Akkira is suggesting e621 implement this sort of thing on-site, to run every time someone uploads. If there is a super similar image going up, the uploader can either be like "oh yeah, totally the same image from a different place, nvm," or "It's an edit, so they're different, upload away," or "Oh, this one's higher res, I'll be sure to go flag the older post."

yesthebest said:
I don't have that issue. What browser are you using?

I'm using Firefox, but the problem fixed itself.

Thank you OP. This be useful.
A search bar on the result page will come in handy.

RedOctober said:
I think Akkira is suggesting e621 implement this sort of thing on-site, to run every time someone uploads. If there is a super similar image going up, the uploader can either be like "oh yeah, totally the same image from a different place, nvm," or "It's an edit, so they're different, upload away," or "Oh, this one's higher res, I'll be sure to go flag the older post."

Ah. That is a pretty useful idea. Though that would require the user still upload the file somewhere, since e621's upload preview just displays the image sourced from the user's local disk... so it wouldn't exactly be possible to automate like that without prompting every uploader every time. Could add it to e621's uploading-readme page, but it's not like most people read that :P.

Raiden_Gekkou said:
I'm using Firefox, but the problem fixed itself.

Yay! I'll just assume the css failed to load or something then.

ktkr said:
Thank you OP. This be useful.
A search bar on the result page will come in handy.

Can you elaborate what you mean on this?

yesthebest said:
Can you elaborate what you mean on this?

No idea, but it probably have to deal with putting forth more results if the image search didn't really find the exact image of what the user submitted in.

In other words, put a search bar just in case if the image search didn't get the image or find the image on e621 and the image is there. I hope this makes sense.

Derp.

yesthebest said:
Can you elaborate what you mean on this?

Having the search bar from main page also on the result page so one doesn't have to keep clicking back to search for something new.

RedOctober said:
I think Akkira is suggesting e621 implement this sort of thing on-site, to run every time someone uploads. If there is a super similar image going up, the uploader can either be like "oh yeah, totally the same image from a different place, nvm," or "It's an edit, so they're different, upload away," or "Oh, this one's higher res, I'll be sure to go flag the older post."

Believe me, that is something I've been badly wanting ever since I started administrating the site 2 years ago.

I mentioned this is the feature request thread long ago, but I guess nothing came of it. Just glad to see it's happening now, even though I hardly upload anymore.

speak to me in pms please

ktkr said:
Having the search bar from main page also on the result page so one doesn't have to keep clicking back to search for something new.

Aha, gotcha. Should be there now.

This is incredibly useful. Thank you for this.

No, no, no, no!
I am not pleased at all.

oh noes, can not connect to database at Wed, 24 Oct 2012 14:19:35 +0000 ;_;

Akkira said:
No, no, no, no!
I am not pleased at all.

oh noes, can not connect to database at Wed, 24 Oct 2012 14:19:35 +0000 ;_;

Unlucky! I was running some high-intensity stuff on the server for like an hour at that time without modifying its priority, so the searching probably took too long so it timed out, hah. It should work fine normally (and works now; just tested it).

Is it just me, or something has broken and search never finds relevant pictures? I downloaded and tried to search for several random pictures, and it didn't find any.

Update!111111
- Implemented URL searching; paste in direct URL to image and have search.

Gilda_The_Gryphon said:
Is it just me, or something has broken and search never finds relevant pictures? I downloaded and tried to search for several random pictures, and it didn't find any.

Link what pictures don't work for you if you're sure they are on e621 and gives no results. Also sometimes my database updater got hung up a few times, probably what happened at that time.

I think this should be promoted much more.

Der_Traubenfuchs said:
I think this should be promoted much more.

I agree!

Also people are free to lightly use http://iqdb.harry.lu/?url=<ImgThumb> with some URL to see results that way (maybe a bit easier than automating a POST), but I don't plan to make some json API or anything like that.

yesthebest said:
I agree!

Also people are free to lightly use http://iqdb.harry.lu/?url=<ImgThumb> with some URL to see results that way (maybe a bit easier than automating a POST), but I don't plan to make some json API or anything like that.

You are making me all hot and sweaty.

Loading (? KB)... failed

Der_Traubenfuchs said:
You are making me all hot and sweaty.

Loading (? KB)... failed

Should work, what's the URL you're loading? Make sure you don't have the <>'s around the URL as well.

yesthebest said:
Should work, what's the URL you're loading? Make sure you don't have the <>'s around the URL as well.

Don't worry, I bet he just clicked the sample URL you provided, without replacing the placeholder.

Also, I've already wanted to post this one but I kept forgetting it. There's this bookmarklet that can be used for easy image uploading from another website. So I made one for myself to be used with your iqdb.
Simply put this in a bookmark:

javascript:location.href='http://iqdb.harry.lu/?url='+encodeURIComponent(location.href)

It can be used the exact same way.

blackest_vulture said (forum #136151):
I'm having trouble using the Reverse harry.lu Search since yesterday. It takes minutes to search for an image instead of seconds.

The strange thing is that the engine still says it searched the image in half a second.

EsalRider said:
blackest_vulture said (forum #136151):
I'm having trouble using the Reverse harry.lu Search since yesterday. It takes minutes to search for an image instead of seconds.

The strange thing is that the engine still says it searched the image in half a second.

Thanks for the quoting :) Too bad we can't vote on forum posts. Please consider my symbolical +1.

Working again. Yay!

Has anyone else had success using a bot to construct a POST request using file upload instead of URL? I'm looking for a way to mass reverse search files from my computer. URL works just fine, but it keeps giving me an erroneous page when I try to upload the file.

Given url="http://iqdb.harry.lu/" and using the python requests library, this is what I have so far:

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, data={"url": "http://i.imgur.com/3NEe6.jpg"})

This responds with code 200 and Thumbnail error code - Not an image. Perhaps your file is corrupt? in the webpage.

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, files={"file": ("q6vtycjh.jpg", file, "image/jpeg")})

This responds with code 200 and Database error: param_error Format: query &lt;dbid&gt; &lt;flags&gt; &lt;numres&gt; &lt;filename&gt; in the webpage.

HELP

Assuming "q6vtycjh.jpg" is a valid jpg file in present working directory

comealongpond said:
Given url="http://iqdb.harry.lu/" and using the python requests library, this is what I have so far:

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, data={"url": "http://i.imgur.com/3NEe6.jpg"})

This responds with code 200 and Thumbnail error code - Not an image. Perhaps your file is corrupt? in the webpage.

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, files={"file": ("q6vtycjh.jpg", file, "image/jpeg")})

This responds with code 200 and Database error: param_error Format: query &lt;dbid&gt; &lt;flags&gt; &lt;numres&gt; &lt;filename&gt; in the webpage.

HELP

Assuming "q6vtycjh.jpg" is a valid jpg file in present working directory

The first is just doing http://iqdb.harry.lu/?url=http://i.imgur.com/3NEe6.jpg`, so I don't know why that wouldn't work. However I know if you do that in a normal browser with the cache disabled, hit back, then hit forward you get the same error. It might be that your request is getting redirected somehow (I'm not sure).

The second looks like you aren't giving it the correct form-data. I don't know much about python, but you may have better luck if you grab a cURL/HAR output from the browser to base the request on. http://iqdb.org/code/README might help explain some of the output regarding the error.

Disclaimer: I have very little experience with this sort of thing.

parasprite said:
The first is just doing http://iqdb.harry.lu/?url=http://i.imgur.com/3NEe6.jpg`, so I don't know why that wouldn't work. However I know if you do that in a normal browser with the cache disabled, hit back, then hit forward you get the same error. It might be that your request is getting redirected somehow (I'm not sure).

The second looks like you aren't giving it the correct form-data. I don't know much about python, but you may have better luck if you grab a cURL/HAR output from the browser to base the request on. http://iqdb.org/code/README might help explain some of the output regarding the error.

Disclaimer: I have very little experience with this sort of thing.

Thanks for the insight on the first point, it's quite interesting.

I've tried using Chrome Dev Tools to intercept the post, copy as cURL, and run from the command line, and it gives me the same Thumbnail error code - Not an image. Perhaps your file is corrupt? content. Which makes absolutely no sense to me, I am not modifying it, and I am running the curl command from the directory of the file.

comealongpond said:
Given url="http://iqdb.harry.lu/" and using the python requests library, this is what I have so far:

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, data={"url": "http://i.imgur.com/3NEe6.jpg"})

This responds with code 200 and Thumbnail error code - Not an image. Perhaps your file is corrupt? in the webpage.

with open("q6vtycjh.jpg", mode="rb") as file:
    res = requests.post(url, files={"file": ("q6vtycjh.jpg", file, "image/jpeg")})

This responds with code 200 and Database error: param_error Format: query &lt;dbid&gt; &lt;flags&gt; &lt;numres&gt; &lt;filename&gt; in the webpage.

HELP

Assuming "q6vtycjh.jpg" is a valid jpg file in present working directory

Also, in the second request, if the files parameter dictionary key is anything other than "file" the response contains the same "Thumbnail error code - blah blah", so I'm quite certain it's locking onto the correct form input.

Getting a Host Error when trying to connect, fingers crossed this service isn't dead, i've been using it constantly for the past few years

MrGaoth said:
Getting a Host Error when trying to connect, fingers crossed this service isn't dead, i've been using it constantly for the past few years

Seems to be alright from my side, no problems here.

Also, try to use the most recent thread, forum #256957, than necro-ing an old one.