Pasiphaë said:
On a side not here I would like to point out:

post #790770

post #775873

File Size is listed as 486.8kb for the top version, the image source is exactly the same as the original post and the resolution is the same. However the original post is 294.5kb. When you go to the source link and download it from the source it is the 294.5kb version. Additionally this post is not a PNG vs. JPEG, downloading 790770 from e621 nets me a JPEG image.

So I would really be curious to know how the file nearly doubled in size yet the cited source does not lead to that image at all.

strange...i feel as if i've run across this very problem myself at some point. to my eyes it looks 100% exactly the same as the source pic so i doubt its quality related.

Pasiphaë said:
On a side not here I would like to point out:

post #790770

post #775873

File Size is listed as 486.8kb for the top version, the image source is exactly the same as the original post and the resolution is the same. However the original post is 294.5kb. When you go to the source link and download it from the source it is the 294.5kb version. Additionally this post is not a PNG vs. JPEG, downloading 790770 from e621 nets me a JPEG image.

So I would really be curious to know how the file nearly doubled in size yet the cited source does not lead to that image at all.

There is some issue with inkbunny where the link to the original picture isn't being shown for some reason. You can get it by getting the URL for the picture:

https://us.ib.metapix.net/files/screen/1340/1340635_Mirapony_dj%5Bedit%5D11.jpg

Then replacing the "screen" part with "full":

https://us.ib.metapix.net/files/full/1340/1340635_Mirapony_dj%5Bedit%5D11.jpg

treos said:
strange...i feel as if i've run across this very problem myself at some point. to my eyes it looks 100% exactly the same as the source pic so i doubt its quality related.

There is some artiftacting pretty easily visible if you zoom in 10x and look around the edges of shapes, especially just above the eye on the left.

SyldraTheCat said:
There is some issue with inkbunny where the link to the original picture isn't being shown for some reason. You can get it by getting the URL for the picture:

https://us.ib.metapix.net/files/screen/1340/1340635_Mirapony_dj%5Bedit%5D11.jpg

Then replacing the "screen" part with "full":

https://us.ib.metapix.net/files/full/1340/1340635_Mirapony_dj%5Bedit%5D11.jpg

There is some artiftacting pretty easily visible if you zoom in 10x and look around the edges of shapes, especially just above the eye on the left.

All well and good, except it is 403 Forbidden. So I am not sure why this would work. Additionally this doesn't explain the image discrepency. The resolutions are exactly the same, changing "screen" and "full size" would NOT affect the image's file size, it would affect the image's resolution.

Pasiphaë said:
All well and good, except it is 403 Forbidden. So I am not sure why this would work. Additionally this doesn't explain the image discrepency. The resolutions are exactly the same, changing "screen" and "full size" would NOT affect the image's file size, it would affect the image's resolution.

The "screen" version is a lower quality version generated by inkbunny for faster download when viewing. "full" is the image that's normally shown when you click the "max. preview" link, I think, which is the original source image. You're getting a 403 error because inkbunny forbids hotlinking, try copy and pasting the URLs instead.

SyldraTheCat said:
The "screen" version is a lower quality version generated by inkbunny for faster download when viewing. "full" is the image that's normally shown when you click the "max. preview" link, I think, which is the original source image. You're getting a 403 error because inkbunny forbids hotlinking, try copy and pasting the URLs instead.

Okay...strange and kind of dumb. But whatever. It seems kind of pointless. I visually don't see a quality difference, yet the image file is nearly double in size.

SyldraTheCat said:
The "screen" version is a lower quality version generated by inkbunny for faster download when viewing. "full" is the image that's normally shown when you click the "max. preview" link, I think, which is the original source image. You're getting a 403 error because inkbunny forbids hotlinking, try copy and pasting the URLs instead.

Pretty much this. except sometimes the screen and full versions gets flipped. I'm still looking into this.

@Pasiphaë i have responded on the images you commented on, explaining how i did it. I hope you find it helpful/useful.

Pasiphaë said:
All well and good, except it is 403 Forbidden.

They don't allow links like this to be clicked from elsewhere (several sites are known to do this, I'm assuming it's to prevent hotlinking). However if you go to the url bar and hit enter it should load the image just fine.

I would think the problem here is how Slyroon is using that very obscure trick that, let's be honest, no-freaking-one knew about it, to simply mass-upload pictures then mass-flag these inferior ones just for the sake of "Must have better versions" While there's absolutely nothing wrong with it and he's absolutely on his rights here, the fact that pretty much everything he uploads here is pictures from InkBunny, used to mass-flag these lower quality pictures that he would have otherwise have no interests to begin with, is, frankly enough, enough for people to see that as kind of very douchey from his part.

Personally, I don't care or mind when I got pictures deleted for higher quality, but seeing that he seems to go around and look for Inkbunny sources until he can find one he can flag because this person had absolutely no idea about this URL tweak just so he can proudly upload the so much higher quality, is enough just to annoy me a little, again, not because I got the picture deleted, that I don't care, I'm not here to have my post count being the highest, I'm just a tad annoyed because of the behavior behind that. Feels like people have to place a post-it about this InkBunny URL tweak so they don't forget about it, otherwise the famous SlyBot will come to the rescue and quickly take care of it

Neitsuke said:
I would think the problem here is how Slyroon is using that very obscure trick that, let's be honest, no-freaking-one knew about it, to simply mass-upload pictures then mass-flag these inferior ones just for the sake of "Must have better versions" While there's absolutely nothing wrong with it and he's absolutely on his rights here, the fact that pretty much everything he uploads here is pictures from InkBunny, used to mass-flag these lower quality pictures that he would have otherwise have no interests to begin with, is, frankly enough, enough for people to see that as kind of very douchey from his part.

Personally, I don't care or mind when I got pictures deleted for higher quality, but seeing that he seems to go around and look for Inkbunny sources until he can find one he can flag because this person had absolutely no idea about this URL tweak just so he can proudly upload the so much higher quality, is enough just to annoy me a little, again, not because I got the picture deleted, that I don't care, I'm not here to have my post count being the highest, I'm just a tad annoyed because of the behavior behind that. Feels like people have to place a post-it about this InkBunny URL tweak so they don't forget about it, otherwise the famous SlyBot will come to the rescue and quickly take care of it

The inkbunny "trick" to get the original images. Instead of the inferior previews. Is something i only learned recently. so of cause I'm gonna have a bigger focus on that. Besides i haven't been shy in telling people how to do it or refused anyone. Also it's far from only inkbunny i upload from. I even have a list on my userpage on how to find bvats

I don't know why it's such big deal, that i care about image quality. Sure i can understand people when they upload stuff . they expect it to stay. But if something better is out their, shouldn't we uploaded it. whether it being higher resolution, superior image format or a larger filesize. E621 is an image archive after all.

I'm not trying to make this into some holy quest or play the victim here. I understand and sympathies with people who get offended. when i or some else posted something "better" than them and theirs gets deleted. but i don't really see a better alternative.

slyroon said:
The inkbunny "trick" to get the original images. Instead of the inferior previews.

Assuming you already know how to grab md5, sure you are grabbing the "initial" rather than the IB optimized* "large" or "full". I've seen some posts where the large/full actually had a bigger file size due to IB changing the DPI.

That being said, we're finding that the effort needed to micromanage these posts have been taking up a disproportionate amount of mod time, so we may need to start giving the benefit of the doubt to older posts when the difference is basically negligible.

*I'm actually not 100% sure of what IB does to files beyond what I can pull out of the EXIF data.

parasprite said:
Assuming you already know how to grab md5, sure you are grabbing the "initial" rather than the IB optimized* "large" or "full". I've seen some posts where the large/full actually had a bigger file size due to IB changing the DPI.

*I'm actually not 100% sure of what IB does to files beyond what I can pull out of the EXIF data.

I upload those that are biggest when full. i don't really trust those inkbunny post. with a bigger filesize when they are previews. as i hasn't yet figured what Inkbunny is doing when the creating previews. Only that it affects jpg files. also their preview images have a lot of compression artifacts around dark lines. While the full version (with a bigger filesize) has none.

slyroon said:
I upload those that are biggest when full. i don't really trust those inkbunny post. with a bigger filesize when they are previews. as i hasn't yet figured what Inkbunny is doing when the creating previews. Only that it affects jpg files. also their preview images have a lot of compression artifacts around dark lines. While the full version (with a bigger filesize) has none.

Compare DPI (you should be able to see them even in Windows properties). I've only looked at a couple but they seem to be setting them all at 100 (even if it was lower, usually 72 or 96, on upload).

parasprite said:
Compare DPI (you should be able to see them even in Windows properties). I've only looked at a couple but they seem to be setting them all at 100 (even if it was lower, usually 72 or 96, on upload).

The DPI is the same whether it being higher or lower file size when "full".

Smaller file size when full
171 KB (Screen/Preview) | 137 KB (Full)
DPI 256 (Screen/Preview) | DPI 256 (Full)

Higher file size when full
248 KB (Screen/Preview) | 567 KB (Full)
DPI 350 (Screen/Preview) | DPI 350 (Full)

• I tested this on multiple images. I get a varied changed in file size, but no change in DPI.

parasprite said:
That being said, we're finding that the effort needed to micromanage these posts have been taking up a disproportionate amount of mod time, so we may need to start giving the benefit of the doubt to older posts when the difference is basically negligible.

Solution: Bump slyroon up to Janitor.

If we were able to upload higher quality pictures over existing posts without breaking links, losing all the comments on the old picture and "stealing" the post from other people, that'd probably make it a lot more manageable too. They'd have their own approval queue of course, maybe even with a side-by-side comparison tool for the approver.

I'm sure that's been requested before at some point though. =p

Munkelzahn said:
Solution: Bump slyroon up to Janitor.

I offered that ages ago but slyroon said no. :c

NotMeNotYou said:
I offered that ages ago but slyroon said no. :c

He just wasn't ready for the responsibility.

NotMeNotYou said:
I offered that ages ago but slyroon said no. :c

I was a janitor at one point. Until i got bumped to admin, which i really didn't enjoy. so i quit and went back to being a contributor. i wouldn't mind going back to being a janitor. Since that was rather enjoyable.

slyroon said:
I was a janitor at one point. Until i got bumped to admin, which i really didn't enjoy. so i quit and went back to being a contributor. i wouldn't mind going back to being a janitor. Since that was rather enjoyable.

I said months ago that I would like to aim to become an admin. And if I did, I could see myself focusing on tagging. Though at the moment, I'm just fixing and learning from my mistakes that made me lose my Privileged status and simultaneously growing more experienced with my budding art hobby so that I can eventually draw some truly stellar artwork that would please and motivate anyone, and simultaneously bring me some extra income that I can use to commission others.

I can imagine being Janitor or Mod or Admin to be stressful at times. Hell, TheHuskyK9 briefly resigned because of family issues but came back when they were resolved sooner than expected. Good thing too because TheHuskyK9 is a nice guy to talk to and we even keep in touch on furaffinity.

Of course it might be way too soon for me to think that, and I don't want to give the impression that I "deserve" that rank either. I know what I do and I like what I do. I asked for Privileged back in March and I got Privileged within a week. I won't ask for it again because I don't think that I deserve it right now. But I still want to be here in the long run and I want to help where I can.

If I didn't, then I would've left or perhaps even self-destructed shortly after my demotion.

Help for the sake of helping, not the rewards which may or may not result.

Ratte said:
Help for the sake of helping, not the rewards which may or may not result.

Eeyup.

I've got better idea! WHY DOESN'T SLYROON ASKS PEOPLE TO REPOST THEIR POSTS after he gives explanation for that. Cause I've counted and for les than a mount there are....there we're 22 REPOSTS BY SLYROON. Duuude please use my advice.

P.S. who's with me.

KillingPinkamena9 said:
P.S. who's with me.

the midichlorians(tm)

KillingPinkamena9 said:
I've got better idea! WHY DOESN'T SLYROON ASKS PEOPLE TO REPOST THEIR POSTS after he gives explanation for that. Cause I've counted and for les than a mount there are....there we're 22 REPOSTS BY SLYROON. Duuude please use my advice.

P.S. who's with me.

Actually i "reposted" as of right now, today. 126 images not 22.

But as the matter of asking people to do it them self. I think it's far more complicated, going around and asking people to upload something, they have already uploaded. than doing it my self. Besides the bvats tag ain't a secret. Anyone can "repost" images from there. Even you.

slyroon said:
Besides the bvats tag ain't a secret. Anyone can "repost" images from there. Even you.

heh, bvats, the one size fits all cure for ailing upload limits. only this cure works as intended with the only downside being people complaining about you getting they're inferior uploads wiped. ;)

edit: O.O woah, the bvats tag is actually under the 1k mark? wow, nice work to who all is working on that.

slyroon said:
Actually i "reposted" as of right now, today. 126 images not 22.

But as the matter of asking people to do it them self. I think it's far more complicated, going around and asking people to upload something, they have already uploaded. than doing it my self. Besides the bvats tag ain't a secret. Anyone can "repost" images from there. Even you.

I know I can repost, but can't you just ignore if your repost and the original is identical. Look, if its smaller, bad quality, corrupted. OK reupload it. But if they are absolutely identical please don't or if you so badly want to do it tell the owner, give him a reason and then post it, the least you can do is to warn him/her.

KillingPinkamena9 said:
I know I can repost, but can't you just ignore if your repost and the original is identical. Look, if its smaller, bad quality, corrupted. OK reupload it.

That is exactly what I'm doing. I'm reuploading better version of existing posts. Some may be the same size, but with a higher quality.

slyroon said:
That is exactly what I'm doing. I'm reuploading better version of existing posts. Some may be the same size, but with a higher quality.

Uuuhm look... there were posts that I've said some things about you, and I've overreacted. Sorry for that. Its just that I've made my favorited posts in a certain way and because of your actions now its disorder. And then you reposted my avatar pic and I've lost my control. Anyway, I was wondering can I organise my favorited posts. Again sorry for my overreaction.

KillingPinkamena9 said:
Uuuhm look... there were posts that I've said some things about you, and I've overreacted. Sorry for that. Its just that I've made my favorited posts in a certain way and because of your actions now its disorder. And then you reposted my avatar pic and I've lost my control. Anyway, I was wondering can I organise my favorited posts. Again sorry for my overreaction.

Favorites are hard to organize, but try out our Sets, you have a much larger degree of customization, it's possible to have more than one, you can make them private or public, or even share moderation of those with other people you trust.

NotMeNotYou said:
Favorites are hard to organize, but try out our Sets, you have a much larger degree of customization, it's possible to have more than one, you can make them private or public, or even share moderation of those with other people you trust.

Sorry but that didn't help me. But ill keep your comment in mind.

Posting here as a sort of "safeguard" I guess

I just uploaded post #818642 making sure to use the Full version of the picture.
Funnily enough, the "not full" picture is actually larger than the full version one by some bytes, so now I'm, um, I guess I'm scared to get that picture flagged down because the full version happens to be smaller than the "not full" one, but also wondering why that's the case

Neitsuke said:
Posting here as a sort of "safeguard" I guess

I just uploaded post #818642 making sure to use the Full version of the picture.
Funnily enough, the "not full" picture is actually larger than the full version one by some bytes, so now I'm, um, I guess I'm scared to get that picture flagged down because the full version happens to be smaller than the "not full" one, but also wondering why that's the case

Just remember that as long as you generally upload good images, a few flags here or there won't affect your uploading limit, and we usually don't give you negative or neutral feedback unless you're constantly getting your images flagged. It's best to just try as hard as you can to find the best quality images. If you get a few deleted, it's not the end of the world :)

I don't care when my uploads get deleted for valid reasons, not the point here. The picture I uploaded is the High quality version, the ones you get from accessing the full URL. However this one is smaller compared to the compressed one you would normally get without modify the URL.

That would just be pretty dumb if it does indeed get deleted for being inferior while the one I uploaded is the superior one, just because it happens to have less bytes

Neitsuke said:
I don't care it gets deleted, not the point here. The picture I uploaded is the High quality version, the ones you get from accessing the full URL. However this one is smaller compared to the compressed one you would normally get without modify the URL.

That would just be pretty dumb if it does indeed get deleted for being inferior while the one I uploaded is the superior one, just because it happens to have less bytes

If I understand you correctly, you're saying the filesize of the smaller image is bigger than the bigger image?

In any case, we generally go by resolution and filetype, rather than how big the file itself is. And, of course, keeping an eye on any obvious artifacts on the image if it's been artifically stretched to a bigger size.

It is just very, um, touchy since recently, by how some malicious users would artificially inflate a picture's filesize or size then flag the actual original picture down for being inferior. A lot of people use the filesize to contest which picture is the best quality too, so it doesn't really help either. I'm just really pointing out about Inkbunny's full versions of pictures being smaller sometimes so filesize alone shouldn't be a factor to find what picture is the best to keep

Well, I can't speak for the other mods or admin, but generally I prioritize looking at the image first, then resolution, then file type, and finally file size.

if you think someone is stretching images, please feel free to DM an admin or mod with your concerns. We are here to help :)

Neitsuke said:
Posting here as a sort of "safeguard" I guess

I just uploaded post #818642 making sure to use the Full version of the picture.
Funnily enough, the "not full" picture is actually larger than the full version one by some bytes, so now I'm, um, I guess I'm scared to get that picture flagged down because the full version happens to be smaller than the "not full" one, but also wondering why that's the case

Inkbunny's Preview images are strange. The full version is (in my experience) the best one. Inkbunny for some reason create unnecessary preview images and some of those previews have a larger file-size. luckily those with a larger file-size doesn't seem to suffer from compression issues.

In my book, you are safe as long as you either post the full version, bigger file-size preview or a png image. (This mostly affects JPG images)

I think something must be done with the way admins look at file-sizes. Because in the past we only had to fear users artificially enlarge file-size. Now websites are doing it too. Inkbunny and Pixiv are the only two offenders i have run into. but i bet there is more out there.

slyroon said:
Inkbunny's Preview images are strange. The full version is (in my experience) the best one. Inkbunny for some reason create unnecessary preview images and some of those previews have a larger file-size. luckily those with a larger file-size doesn't seem to suffer from compression issues.

In my book, you are safe as long as you either post the full version, bigger file-size preview or a png image. (This mostly affects JPG images)

I think something must be done with the way admins look at file-sizes. Because in the past we only had to fear users artificially enlarge file-size. Now websites are doing it too. Inkbunny and Pixiv are the only two offenders i have run into. but i bet there is more out there.

You may want to have a look at this: https://wiki.inkbunny.net/wiki/MD5

Basically "Full" is the Inkbunny-optimized version (strip metadata, optimize compression, etc.) while "Initial" is actually the unaltered file that the artist uploaded.

I haven't been able to see really any quality difference between full and initial, but technically speaking "initial" would be the more "official/unaltered" one of the two.

Well that's, quite the big turn out of events now. To know that all those InkBunny pictures that got deleted and flagged down were actually the ones with their original filesize and quality, quite shocking

parasprite said:
You may want to have a look at this: https://wiki.inkbunny.net/wiki/MD5

Basically "Full" is the Inkbunny-optimized version (strip metadata, optimize compression, etc.) while "Initial" is actually the unaltered file that the artist uploaded.

I haven't been able to see really any quality difference between full and initial, but technically speaking "initial" would be the more "official/unaltered" one of the two.

But this isn't a case between "Initial" and "full" but "Screen"/"Large " vs. "full".

When people download straight from a Inkbunny page like this one

They would get the Screen/preview image
https://nl.ib.metapix.net/files/screen/1389/1389521_Kaittycat_kt_alphazeron.png

Unless they click on the image at the image page
https://nl.ib.metapix.net/files/full/1389/1389521_Kaittycat_kt_alphazeron.png (Link doesn't work unless you go to ink-bunny first)

This create a sort of weird almost paradoxical issue. Because:
340Kb Screen | 920px × 830px
324KB Full | 1.000px × 902px

Screen version is smaller but have a higher file size
Full version is bigger but have a smaller file size

So which is better?

And the same case can be made for a picture without the "click for full version" feature.

Lets take this one

https://nl.ib.metapix.net/files/screen/143/143157_Shiuk_001.jpg (screen)
https://nl.ib.metapix.net/files/full/143/143157_Shiuk_001.jpg (full)
(go to Inkbunny first, to get any of these links to work)

827px × 708px | 181KB (Screen)
827px × 708px | 310KB (Full)

Same size, different file sizes. But if you look closely or in my case zoom in about 400-500% you can clearly see compression artifacts all over the Screen version and if you compare it to the full version. those artifacts ain't there.

Now I'm no expert, but as i see it the "Full" version is at least better than the "screen" version.

Plus the link you gave me, is from an article that hasn't been updated since 1 January 2011. There is no telling how much Inkbunny have changed since then. The information seems outdated. Also i haven't managed to get any other version than "Full" and "screen". If you know a way, to get any of the versions, the article mentions. Do please post a link and a way to locate them. So i can Look into this whole Inkbunny thing.

In the spirit of identifying a more generally applicable guideline for comparing two versions: Using imagemagick identify utility, jpg-generated noise shows up as an increase in the entropy (output of identify -format "%[entropy]" $FILENAME).

Interpolation-generated blurring also shows up as an increase in entropy value (for example, the "screen" version of IB post 1389521 is visibly downscaled (when you flip back and forth in an image viewer, "screen" version is blurry compared to "full" version.), and the entropy values -- 0.458405 for screen, 0.414084 for full) reflect this.)
So this might be a decent method of picking between images that are of similar or same size but not identical, generally. I still need an 'upscaling' test case though -- if I guess correctly, upscaling will also show up as an entropy increase.

slyroon said:
Now I'm no expert, but as i see it the "Full" version is at least better than the "screen" version.

Plus the link you gave me, is from an article that hasn't been updated since 1 January 2011. There is no telling how much Inkbunny have changed since then. The information seems outdated. Also i haven't managed to get any other version than "Full" and "screen". If you know a way, to get any of the versions, the article mentions. Do please post a link and a way to locate them. So i can Look into this whole Inkbunny thing.

I have no idea if you can get the initial version, but I know I've seen some uploaded here (maybe by the artist?) using the md5 search. All I meant though is it might be worth checking that you aren't replacing initial with full in some cases.

Neitsuke said:
Well that's, quite the big turn out of events now. To know that all those InkBunny pictures that got deleted and flagged down were actually the ones with their original filesize and quality, quite shocking

Not exactly. Screen and full are different versions of the same file, and afaik full should be equal or higher quality in most (if not all) cases. If Slyroon was just replacing the screens with fulls, then this wouldn't actually apply.

savageorange said:
In the spirit of identifying a more generally applicable guideline for comparing two versions: Using imagemagick identify utility, jpg-generated noise shows up as an increase in the entropy (output of identify -format "%[entropy]" $FILENAME).

Interpolation-generated blurring also shows up as an increase in entropy value (for example, the "screen" version of IB post 1389521 is visibly downscaled (when you flip back and forth in an image viewer, "screen" version is blurry compared to "full" version.), and the entropy values -- 0.458405 for screen, 0.414084 for full) reflect this.)
So this might be a decent method of picking between images that are of similar or same size but not identical, generally. I still need an 'upscaling' test case though -- if I guess correctly, upscaling will also show up as an entropy increase.

Neat :)

Since all of this became so confusing, I decided to look it up for myself and here's what I got on some of my own PNG picture that didn't need to get resized down to be shown on the page

Unaltered Picture (The original one from my PC)
9,91 KB
Picture shown on the picture page
7,43 KB
Picture shown once you change the URL to full
6,69 KB

I have no idea what's going on here, the uncompressed picture is again smaller in size.

This once again scares me a bit. I can imagine that a lot of previous pictures were deleted because they simply were bigger in size, even if for a fact those were the original pictures that may have been found on another place but had Inkbunny as second source

Neitsuke said:
Since all of this became so confusing, I decided to look it up for myself and here's what I got on some of my own PNG picture that didn't need to get resized down to be shown on the page

Unaltered Picture (The original one from my PC)
9,91 KB

About 10K? very small picture. I just want to verify that is really what you meant and not nine hundred and ninety one kilobytes.

Picture shown on the picture page
7,43 KB
Picture shown once you change the URL to full
6,69 KB

Okay, what's more important than these numbers is a pixel-by-pixel comparison original vs full (assuming their image dimensions are the same.). I can describe a simple procedure to do that, involving Difference layer mode and Threshold, if you want.

The cause for the difference could be something as innocuous as running optipng/pngcrush/zopfli etc on your image. If it's a jpeg, that's less likely.

I save PNG pictures with SAI as is, so maybe the SAI PNG's algo is just pretty, pretty bad and inflates the filesize.
I saved the picture as PNG with PhotoFiltre and the size would indeed drop to 7,14 KB, unless I would use some filter options that would actually bump the size to 10 and more.
The picture I tested isn't that big because the background is bland and pretty boring, and the picture is 600 X 600, can be seen here http://www.pixiv.net/member_illust.php?mode=medium&illust_id=38267531

Also looks like even Pixiv is doing this, the filesize for the picture on Pixiv is 5,71. It's a really big mess

Neitsuke said:
I can imagine that a lot of previous pictures were deleted because they simply were bigger in size, even if for a fact those were the original pictures that may have been found on another place but had Inkbunny as second source

This is pretty much why I've been leaning on us just keeping whatever the older post is. It's gotten to the point where I've been seeing less and less gain compared to the considerable amount of time we have to spend micromanaging what are essentially negligible differences in quality.

I just spotted this forum post , and it prompted me to run a few quick tests on relative entropy of upscaling.

• failed to detect waifu2x upscaling (lower entropy on upscaled version)
• linear, cubic, DCCI, and GMIC diffusion upscales were all correctly detected (higher entropy value on upscaled version)
• (as previously noted, additional jpeg encoding and downscaling are also detected as higher entropy)

So it looks like it (ImageMagick identify -format "%[entropy]") might be a suitable metric for quickly distinguishing basic downscales/upscales from the original; something that could be used to speed up moderation decisions.

@Neitsuke
For PNGs at least, having the original invalidated is not too big a problem. Use pngcrush / optipng etc to optimize it. Picture now almost certainly has a different md5sum, with the exact same (pixel-for-pixel) content. (metadata may be stripped; using the right arguments to optipng or your tool of choice can prevent that)

JPEGs are less easy, but in their case EXIF manipulation can produce a file with a different md5sum and the exact same pixel content.

(btw, in case you haven't already guessed -- I would bet that Inkbunny etc are using something like optipng. Running optipng with a high optimization level -- eg -o7 -- might reveal that it's possible to get it even smaller than Inkbunny's version without any change to pixel data.)

EDIT: looks like Pixiv is definitely doing this, optipng -o7 on the image at the pixiv link you gave reports ".. is already optimized." Pixiv actually does one byte better than optipng, so it's probably using some combination of optipng, pngcrush, zopfli..)

I just don't see myself using an external program since from what I tested, the space saved is very negligible and I would just find it an annoyance to have to crush every PNG I would plan on uploading, or even just doing it on my own pictures.

But clearly, there has to be some reform about this whole best picture situation since it has become more clear now that filesizes are very easy to be different from various websites without having the quality being different at all

Neitsuke said:
I just don't see myself using an external program since from what I tested, the space saved is very negligible and I would just find it an annoyance to have to crush every PNG I would plan on uploading, or even just doing it on my own pictures.

But clearly, there has to be some reform about this whole best picture situation since it has become more clear now that filesizes are very easy to be different from various websites without having the quality being different at all

I think you are missing the point of what I'm saying, which is that duplicate images are detected by md5sum. Hence, if an image is wrongly deleted, the original can still be reuploaded as long as the md5sum is changed. The easiest way to change md5sum without changing pixels is optipng or similar.

So valid 'original' images aren't **really** blocked from being reuploaded, despite the deduplication mechanism.

On your other point, the 'entropy' measure I proposed to use (already implemented in ImageMagick) doesn't care about file size -- it's taken purely from the actual pixels of the decoded image.

Not too sure if I'm getting this right then, but if I did now then you suggest to change the md5 formula for the website so it only considers pixels and size instead of filesize + pixels and filesize.

That would be a much better idea than what I thought to understand that the plan was to unify every pictures with that opticrush thing to check if they are the same quality

Neitsuke said:
Not too sure if I'm getting this right then, but if I did now then you suggest to change the md5 formula for the website so it only considers pixels and size instead of filesize + pixels and filesize.

md5 doesn't care about the contents of the file, it's just a generic hash. It doesn't matter whether it's a jpg, pdf, encrypted zip, corrupted exe, or even complete gibberish; their hash all generates using the same methods.

parasprite said:
md5 doesn't care about the contents of the file, it's just a generic hash. It doesn't matter whether it's a jpg, pdf, encrypted zip, corrupted exe, or even complete gibberish; their hash all generates using the same methods.

What he means is:
Generate the MD5 from the raw data in the video buffer.
You'd have to make sure that the pixels always have the same color depth and RGB order.
And maybe some other things.

Wow, ok, no. Parasprite has it right.

Munkelzahn, generating an MD5 from raw pixel data is actually an interesting, albeit tricky to do 100% reliably, idea for reducing occurrence of 'false differences', but, not what I meant.

My statements purely hinge on the way hashes are calculated (from the entire file's bytes), and the fact that there are millions of unique possible ways to encode the same image pixels into a PNG*. Result = If you have a legitimate need to do an end-run around the MD5-based deduplication, as in the case of the original getting deleted wrongly in favor of an upscale, it's relatively simple to do so.

PNG is just an example; but any lossless format with compression, or any container-style format (jpeg, webp?,..), or any format where you can harmlessly append arbitrary bytes... so in practice, probably every format e621 supports.

@Neitsuke: I get the impression you don't understand what a hash (as in MD5, SHA1, etc) is. If parasprite's explanation wasn't detailed enough, try this:

Wikipedia:Fingerprint_(computing) (read at least the summary, it's fairly short)

This is how e621 identifies files (and determines that an incoming file is a duplicate of an existing one). The fingerprint of the file (which is in this case an MD5 hash ) is calculated from the bytes of the file, and the file is saved on the server as FINGERPRINT.EXT (where FINGERPRINT is the md5 hash, eg 3e9ac737fb4238dcd926a3263ce542c8, and EXT is the extension of the file, eg png).

But first, it checks that a file with that name doesn't exist already.
If it does, then the upload is considered a duplicate and rejected. My strategy can circumvent that, by changing the file bytes (without changing the pixels that that file decodes to). Changing the file bytes naturally changes the fingerprint of the file, so the newly uploaded file is not considered a duplicate of the existing file.

savageorange said:
If you have a legitimate need to do an end-run around the MD5-based deduplication, as in the case of the original getting deleted wrongly in favor of an upscale

In case of wrong one being deleted, the proper approach would be to inform admin so they can restore original and delete the upscale - not to re-upload a tinkered original.

I know what a md5 is and how it is generated, but I do have a hard time to understand the actual solution you try to propose here, because at one time you say to keep the md5 being different so the original picture isn't blocked, but then later you say that your solution will only take in count the pixels and the dimensions of the picture, which should just be really that for solution. The filesize keeps being different from site to site despite being the exact picture pixel per pixel and by only comparing the pixels and dimensions that would shut down the "My picture is best quality because it has Full in the file name" problem.

We may end up not being able to have the "pure original" version of the picture, but as long the quality is 100% original and the file isn't inflated (Which should be take into account too with some algo checking if that's possible) then I don't really see a problem here

If a discussion on getting a unique image fingerprint, this is what I do to generate image fingerprints:

# Scale everything to the same size
image = scale::nearestNeighbor(image 1024,1024)

# We store data here
let result = ""
# current value
let i = 0
# Didn't have a better name for this
let flip = false

# Each pixel in the image, left to right, and down when x > image width
for each pixel in image:
    # Temporary variable
    let tmp = 0
    
    # Bitwise for each pixel, basically if red is > 128 and green is < 128 and
    #  blue is > 128, and alpha is > 128, tmp will hold 1011
    if pixel.r > 128:
        tmp = tmp | 8
        
    if pixel.g > 128:
        tmp = tmp | 4
        
    if pixel.b > 128:
        tmp = tmp | 2
        
    if pixel.a > 128:
        tmp = tmp | 1
        
    
    if flip == true:
        # If flip is set, unset it and continue
        flip = false
        # merge the two bitwise variables: 1101 and 1001 become 11011001
        i = i | tmp
        # convert the integer to a ascii character
        result = result + chr(i)
        # reset I to zero
        i = 0
    else:
        # Push the bitwise over 4 bytes: 1101 becomes 11010000
        i = tmp < 4
        
# Result now has the fingerprint of the image
result = hash::md5(result)

It works for the most part, but still needs human intervention as there is still a very VERY light chance for it to get a false positive.

Neitsuke said:
I know what a md5 is and how it is generated, but I do have a hard time to understand the actual solution you try to propose here, because at one time you say to keep the md5 being different so the original picture isn't blocked, but then later you say that your solution will only take in count the pixels and the dimensions of the picture, which should just be really that for solution.

Okay, maybe I've been confusing. I suggested two things:

a) When an image is flagged as duplicate of another, use ImageMagick to calculate 'entropy' measures for each of them. These values would be usable by the mods in more efficiently deciding which one is the 'correct' one to keep. It would have no other effect (not on uploading, automatic deduplication, or anything else); it's purely informational. Also, it would not be informative in the case where images are pixel-wise identical; the entropy will be exactly the same.

b) Since fixing a bad file (non original, and not pixel-wise equivalent to original) is excessively involved, it's possible to instead 'work around' a bad file by performing a process that modifies the file so its fingerprint is different (considered unique, rather than a duplicate of existing post); you can then upload the correct file and flag the incorrect one. This is only for that purpose, not as a general purpose measure. There are 3 elements involved here: 1. bad file that got picked as 'correct', 2. original file that was actually correct and got deleted, 3. Tweaked version of #2, so that #3 has a different hash from #2 but is pixel-wise identical.

c) I have no proposition to modify the general automatic deduplication system in any way. Munkelzahn suggested that I meant such a thing, but this suggestion was wrong.

These three things are not related (except insofar as the original problems they were addressing were brought up in this thread)

BTW: In my opinion, making e621 compare the actual image pixels could be done but would be surprisingly tricky. It wouldn't be a complete solution, as metadata also needs to be compared (things like author, comment, copyright info is important. ICC profile is important if attached, as it can dramatically effect display of the image).
Not too hard to see why such an involved thing hasn't been implemented yet, when a simple MD5 comparison has generally achieved a 'good enough' result with much less effort.

the file isn't inflated (Which should be take into account too with some algo checking if that's possible)

The simplest way to do such a thing would probably be to compare the output size of pngcrush / optipng / whatever to the original. That's for PNGs (and potentially any other lossless/mostly-lossless format, like SWF). For JPGs and WEBMs (and, AFAICS, any lossy encoding format), it's not so easy to do (may not be possible at all).

Wodahseht said:
In case of wrong one being deleted, the proper approach would be to inform admin so they can restore original and delete the upscale - not to re-upload a tinkered original.

I addressed this partly above -- AFAIK it has been stated by mods that it's NOT actually that simple to resolve bad duplications/flaggings. Pretty sure true originals have been rejected in the past on this basis.

If I'm wrong about this, then the measure b) I'm talking about is irrelevant.

savageorange said:
I addressed this partly above -- AFAIK it has been stated by mods that it's NOT actually that simple to resolve bad duplications/flaggings. Pretty sure true originals have been rejected in the past on this basis.

I'm not sure what you're referring to but restoring a wrongly-deleted original isn't that difficult (as long as you know which one is the original). We definitely don't want people uploading altered versions of originals to circumvent the true original being deleted - we can just restore the true original and delete the dupe.

TonyLemur said:
I'm not sure what you're referring to but restoring a wrongly-deleted original isn't that difficult (as long as you know which one is the original). We definitely don't want people uploading altered versions of originals to circumvent the true original being deleted - we can just restore the true original and delete the dupe.

That's good to know, then.

[This post isn't about determining which duplicate is superior/original/best.]

Has an automated reverse image check and notify system for finding duplicates, upscales, lower quality versions, and (unauthorized) edits been discussed? It's not ideal for sketches and plain background pictures, but it seems more effective than anything done with MD5 hashes, on paper anyway.

If there's a high similarity--let's say a 90% similarity threshold is defined--the uploader can be asked to check the similar existing post(s) and confirm that their post has new content. That prompt would also be a great place to state any posting rules or guidelines pertaining to duplicates and edits. Likewise, janitors and above could have some index of all the high similarity posts. Old posts can be reviewed and have the appropriate personnel "sign off" that posts are merely similar to move them out of the "similarity queue", and new posts would have the reverse image check incorporated into the approval process.

Pardon me if any of this has been discussed before or is perhaps in use--I wouldn't know about that.

No offense sly, but I saw you do what the title of this thread says on a few of bulletsoups Ambrose images. Same image, Same proportions, larger file size... but sly flags the smaller file size.

Just a little heads-up about Slyroon's Full Quality pictures from InkBunny.

PNJ's quality and colors seem to be unaffected by whenever it's full or not : There are no difference whatsoever after a quick look up through my very bland aliasing picture with the color picker : No color changed, no blurry effect on the edges and such.

BUT, it does hold true for JPG pictures. The compressions are very noticeable enough to see and the Full version is obviously the best quality.

I'd assume that the quality only changes with lossy image formats, while lossless are completely unaffected, so I'd assume that GIFs' quality won't matter too, unless it has an impact with transparency or the way compressed frames work (Using transparency to get the color from the next frame)

What about post #583788 and post #822656? The newer one is same resolution, but with smaller filesize and lower color count. Both JPG. Yet the original one was deleted. What's the logic behind that?

You uploaded the Master version of the picture, the one that appears with the entire image set. Get both version of the same picture and zoom on the background color then quickly compare : You will see some artifacts on the Master version of the picture (The one you got deleted)

EDIT: Just noticed that : It's actually a PNG, not a JPEG, so here's your answer (Still doesn't explain how the fuk the JPG is bigger than the PNG file)

Okay, the filetype explains it. Though I have to admit that I don't see any difference at all. I've been flipping between the two for a while now, and they look completely identical to me.

vvv Still not seeing it, but I'll take your word for it.

Zoom in on the JPG picture (without any smoothing) and look at the background, I can see some light artifacts on the brown background close to the edges

..Use an actual image comparison tool.
Throwing the two images into GIMP as layers, setting the upper to Difference, doing "new from visible" and autoleveling the result for best visibility, shows this . (black areas are areas of no difference)

Neitsuke said:
EDIT: Just noticed that : It's actually a PNG, not a JPEG, so here's your answer (Still doesn't explain how the fuk the JPG is bigger than the PNG file)

That's not very strange. JPG is awfully good for photos, and progressively worse the further you get away from photo-like content. PNG is the reverse.

Is normal to have to edit the address bar to find the higher resolution version of the image in Tumblr? Because that was the only way I seem to be able to access the higher resolution versions of the posts that you replaced.
BTW: I'm going to add the descriptions back.

Naccul1 said:
Is normal to have to edit the address bar to find the higher resolution version of the image in Tumblr? Because that was the only way I seem to be able to access the higher resolution versions of the posts that you replaced.
BTW: I'm going to add the descriptions back.

Yes, actually.
Tumblr always generates several differend sizes of the image, so blogger can easily embed the size he/she wants to make blog post look neat and load up fast for reader. Usually clicking the image takes you to highest quality variant, but it can be disabled if blogger embedded the image in certain way.

To get the best image manually, replace number in the end before file extension with 1280 (e.g. *_500.png -> *_1280.png).

There's a thread for this: https://e621.net/forum/show/169987

Moving comments has it's ups and downs, upside being comments are preserved, downside stuff relating to the quality of the older post.
And it is also a method of secretly getting rid of creepy comments(though they will come back, they always come back).

As for descriptions, not everyone views them, and often they have little to do with the original author's information on the image.
As for translations(I am suspecting notes), these should be moved over if someone posts a better version(not sure why they weren't), but they must be moved manually as e621's note system is based off pixel grid, it's not like a UV map which has a range of 0.0-1.0.

BismuthGalaxy said:
Really? I thought that descriptions were ONLY for notes from the artist. That's how I always treat them, and sometimes it's an attached story which adds another level of depth to, or explains the picture.

Nope, for example post #967200