Topic: Is there an automated system that tells when an image is a duplicate?

Posted under General

I posted an image that was a duplicate without knowing it the image was flagged as such however the image was zoomed in and rotated so that made me ask is it automated or is it a person who does it?

Updated by savageorange

Its a bit of both really, the system only checks for duplicates with the same MD5 hash, otherwise, it is just a person who flags them. It sounds like in your case a person did, if the image was modified in any way.

Updated by anonymous

Imanton1 said:
Its a bit of both really, the system only checks for duplicates with the same MD5 hash, otherwise, it is just a person who flags them. It sounds like in your case a person did, if the image was modified in any way.

Thank you.

Updated by anonymous

iqdb.harry.lu lets you search for duplicate images on e6.

Updated by anonymous

Jackalfag said:
iqdb.harry.lu lets you search for duplicate images on e6.

Does that work for reversed or rotated images?

Updated by anonymous

Furrin_Gok said:
Does that work for reversed or rotated images?

No, but GIMP would work great to reverse any changes to pictures.

Updated by anonymous

Furrin_Gok said:
Does that work for reversed or rotated images?

I would imagine not.

I was gonna try with one of my favorites, but it spat out an error code after reuploading the horizontally flipped version.

Updated by anonymous

Jackalfag said:
I would imagine not.

I was gonna try with one of my favorites, but it spat out an error code after reuploading the horizontally flipped version.

Rather than intentionally uploading a duplicate, why not just use harry.lu to directly check if it existed? Or, upload to photobucket for a short bit.

Updated by anonymous

Furrin_Gok said:
Rather than intentionally uploading a duplicate, why not just use harry.lu to directly check if it existed? Or, upload to photobucket for a short bit.

That's what I was doing. I had no intent to upload the duplicate to e6.

Updated by anonymous

Most duplicate detectors don't detect flipped or rotated images, even though there are statistical methods to normalize orientation.

(for example, take the 1st derivative along a 1px horizontal line of the image. Divide this array of derivative values into two parts, one for the left half and one for the right. Sum each part. Now do the same for a vertical line; you've now divided the images into 4 quadrants. Calculate the score of each quadrant by multiplying the score for the horizontal half with the score for the vertical half. Now rotate/flip the image such that the quadrant with the highest score ends up with one of its corner pixels at (0,0) (top left of the final image))

Updated by anonymous

Reason to if there are video which gets automatically deleted on youtube, like music video, you can sometimes still find it flipped.

They are also really bad to detect cropped stuff. Google will sometimes get things correctly when you put few keywords in. That is why tagging your stuff is really important because reverse search has more data to go trough and when it completely fails, you can still do the searching manually.

Updated by anonymous

Some algorithms do detect cropped versions with some effectiveness -- findimagedupes, for example. It does this by cutting off the border areas of the image before fingerprinting it.

But it's true that detecting crops really well takes a lot more CPU/GPU power than the 'fingerprint comparison' method used by all large-scale image searches.
(Imagine taking your image and using a panorama stitcher like Hugin to auto-align it with each of the images in the database, one at a time.)

Updated by anonymous

  • 1