Bug overview description.
Attempting to upload this image. The error is received regardless of the url being the raw version or not. Even occurs if I attempt a direct file upload... I am unsure why, and would like some help in fixing this...

What part(s) of the site page(s) are affected?
Upload page

What is the expected behavior?
Image should upload

What actual behavior is given instead?
Error pops up

Can you reproduce the bug every time?
Yep. Error is prompted when using raw url, resized url, or even when directly uploaded.

IIRC this has been reported before (for a different image) and seems to be because the image has a particular type of ICC profile attached.

(Yes, I've verified that this image has an ICC profile attached too. Haven't had time to extract it and examine it yet.)

EDIT: exiftool -icc_profile:\* image.jpg gives an warning Warning: [minor] Incomplete ICC_Profile record . If that is the case, then probably the sample should be generated by copying the file and stripping the erroneous profile data out.

(So far I've had no luck doing that with exiftool -icc_profile= -o stripped.jpg image.jpg -- the fact that the ICC profile is incomplete to begin with seems to prevent the file from being written.)

savageorange said:
IIRC this has been reported before (for a different image) and seems to be because the image has a particular type of ICC profile attached.

(Yes, I've verified that this image has an ICC profile attached too. Haven't had time to extract it and examine it yet.)

Ah I see, my apologies. Is there any way to rectify this issue, or is the image just unuploadable?

Not sure; it seems to specifically be an incomplete ICC profile, see my edit.

Theoretically, the upload system could strip out the bogus icc profile and use the stripped version of the image to successfully create a sample. However, like I say above, I haven't figured out exactly how to do that yet.

Any rectification of the image should be done without re-encoding image pixels, so it probably must be done by a dedicated EXIF-manipulation tool rather than a general image editing program.

EDIT:

• Here's a working usage of exiftool to strip incomplete ICC profile. It should be noted that this invocation also strips other, possibly important, metadata; a better method would remove ONLY the ICC profile.: exiftool -all= -tagsfromfile @ -colorspace -interopindex image.jpg
• AFAICS stripping needs to happen within E621's sample creation code, not on the uploader's end, since modifying the image results in a different md5sum, which would cause problems for duplicate detection.

DiceLovesBeingBlown said:
Ah I see, my apologies. Is there any way to rectify this issue, or is the image just unuploadable?

you could probably open it in some image editing software and save it as a png and upload that.
The file is actually considerably smaller converted into png format for some reason.

darryus said:
you could probably open it in some image editing software and save it as a png and upload that.

That doesn't remove the bogus ICC profile, though. It would just be a bogus ICC profile attached to a PNG rather than a JPG. Maybe that would give a different result, but who knows?

savageorange said:
That wouldn't remove the bogus ICC profile, though. It would just be a bogus ICC profile attached to a PNG rather than a JPG. Maybe that would give a different result, but who knows?

Would it not? I'm not really sure how to use exiftool but it looks like at least Paint.NET strips the ICC profile as well as most of the rest of the metadata away when saving it. I'm guessing that might be a factor as to why it's so much smaller.

darryus said:
Would it not? I'm not really sure how to use exiftool but it looks like at least Paint.NET strips the ICC profile as well as most of the rest of the metadata away when saving it. I'm guessing that might be a factor as to why it's so much smaller.

If it does strip the ICC profile silently, that would normally count as corruption (ie. a bug), since the ICC profile actually effects the appearance of image colors, it's not 'just annotation' like author, timestamp etc are.

This ICC profile is ~65k, so if it's been removed, I agree it could account for some of the size difference.

It could also be that the image is in lab color or some other color space that the sample tool doesn't know how to support. It's a really small piece of code, and it doesn't have a lot of the magic support for the full spec of each image type as a result of that. You see that with gamma profiles as well, where it doesn't know how to apply the gamma profile to an image and they come out too bright or too dark. It's on my list of things to work on to make it support more of the complete image spec.

KiraNoot said:
It could also be that the image is in lab color or some other color space that the sample tool doesn't know how to support.

Someone posted a CMYK jpeg before IIRC, but ImageMagick identify reports this particular one as being semi-normal YCbCr.

To be specific:

Colorspace: sRGB
  Type: TrueColor
  Gamma: 0.454545   (IM reports this value for all sRGB images, AFAICS)
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
 Properties:
    exif:ColorSpace: 65535   ("Uncalibrated")
    icc:description: U.S. Web Coated (SWOP) v2
    jpeg:colorspace: 2   (I think this specifies YCbCr color as opposed to 1=Y greyscale, but wasn't able to find definitive info)

Something that I concluded from my research is that JPG does not really support any colorspaces other than YCbCr and Y, so files that claim otherwise may be unofficial hacks.

I had the same problem yesterday when I tried to upload one of the images from:
https://www.bzga.de/presse/pressemotive/liebesleben/
(it's a German ad campaign to raise awareness of sexually transmitted infections - conservatives think it encourages promiscuity)

Of course I wanted to upload the one with the deer in the background.
( Motiv "Wild in freier Wildbahn" )

ImageMagick identify:

LL-0416_Cartoon_2018_Online_A4_Hoch_RZ4.jpg JPEG 2480x3508 2480x3508+0+0 8-bit CMYK 3.475MB 0.010u 0:00.009
LL-0416_Cartoon_2018_Online_A5_Quer_RZ4.jpg[1] JPEG 2480x1748 2480x1748+0+0 8-bit CMYK 2.446MB 0.000u 0:00.000

What's the offical, admin-approved way to deal with this?
a) Save the offending images as PNG's
b) Run them through a tool that removes the color space info

edit:
Of course the color is in YMCA (joke) format, because you're supposed to print the images.

Munkelzahn said:
What's the offical, admin-approved way to deal with this?

Site goes by visual quality and artificially bloating the files is bad, bad, bad.

Most ideal situation is if the file can be modified as low amount as possible to make it useable and uploadable, meaning modifying the part that's causing issues without re-saving the JPG file (and thus doubling the amount of compression).

If instructions unsure, save as PNG (and run trough OptiPNG, PNGGauntlet, etc.) and explain the situation in post description.

Mairo said:
Site goes by visual quality and artificially bloating the files is bad, bad, bad.

Most ideal situation is if the file can be modified as low amount as possible to make it useable and uploadable, meaning modifying the part that's causing issues without re-saving the JPG file (and thus doubling the amount of compression).

If instructions unsure, save as PNG (and run trough OptiPNG, PNGGauntlet, etc.) and explain the situation in post description.

I need some help.
The image has to be converted anyway, because it doesn't have a color profile.
If I open it with an image viewer, or convert it with "convert -colorspace RGB infile.jpg outfile.jpg", then the colors look much too bright.
The preview image looks ok, though: https://www.bzga.de/pressemotive.pixbig.php?id=5f8a6b3f46a482379a376d5104c61403
So I need a color profile.
How do I find one that makes the downloadable image look like the preview?

Munkelzahn said:
I need some help.
The image has to be converted anyway, because it doesn't have a color profile.
If I open it with an image viewer, or convert it with "convert -colorspace RGB infile.jpg outfile.jpg", then the colors look much too bright.
The preview image looks ok, though: https://www.bzga.de/pressemotive.pixbig.php?id=5f8a6b3f46a482379a376d5104c61403
So I need a color profile.
How do I find one that makes the downloadable image look like the preview?

I'm inexperienced with this particular stuff (I know how to rotate and crop JPG files without quality loss, but this is something entirely else), so in this case I would refer to second paragraph of my last message and fallback to just using PNG.

Munkelzahn said:
The image has to be converted anyway, because it doesn't have a color profile.

It does have a color profile.
identify -verbose image.jpg ->

Properties:
    icc:copyright: Copyright 2007 Adobe Systems, Inc.
    icc:description: Coated FOGRA39 (ISO 12647-2:2004)

  Profiles:
    Profile-8bim: 28 bytes
    Profile-icc: 654352 bytes

If I open it with an image viewer, or convert it with "convert -colorspace RGB infile.jpg outfile.jpg", then the colors look much too bright.

I've tried this, both with -colorspace RGB and -colorspace sRGB (which are not the same thing, FWIW), and I confirm these results.

Converting to PNG will normally automatically convert to sRGB.

(It doesn't solve the brightness issue, though)

It's possible that ImageMagick is not actually using the ICC profile during the conversion. I'm currently trying opening the file in GIMP, which IIRC supports both ICCv2 and ICCv4.

EDIT: heh, GIMP doesn't support CMYK, so I'll try Krita instead.

EDIT2: Krita correctly recognizes the ICC profile. However, the header text is not white (and after experimenting with a range of options, I'm beginning to suspect that the thumbnail containing white header text is generated from an older version of the image.)

Mairo said:
If instructions unsure, save as PNG (and run trough OptiPNG, PNGGauntlet, etc.) and explain the situation in post description.

post #1566776

Done. Explained situation in description, hope this was the proper way to go about things.

savageorange said:
EDIT: heh, GIMP doesn't support CMYK, so I'll try Krita instead.

At least 2.10 seems to support that quite well.

DiceLovesBeingBlown said:
post #1566776

Done. Explained situation in description, hope this was the proper way to go about things.

I have TN-panel gaming monitor, so color accuracy for me is already pretty poor and then there's also all of the software support where I have no idea what programs fully support original JPG file and what does not, which can also result them looking wrong. However it does seem like everything will alter colors to some degree. With Gimp export, I get similar colors but darker, while that upload and pnggauntlet output appear more reddish.

I would actually really like it if someone with bit more knowledge would chip in on this discussion.

Mairo said:
At least 2.10 seems to support that quite well.

Not sure what you mean, given that I am in fact using 2.10, and it has no CMYK image mode. Opening the offending image causes it to be assigned the 'GIMP default RGB' ICC profile; It seems a reasonable presumption that if GIMP supported CMYK profiles fully, it would preserve the image's "Coated FOGRA39" ICC profile.

Presumably, that's not possible because of the lack of CMYK mode (-> must be converted to RGB). In a case where correctness is in doubt, I'd much prefer to examine the raw channel data before conversion is done.

I have TN-panel gaming monitor, so color accuracy for me is already pretty poor and then there's also all of the software support where I have no idea what programs fully support original JPG file and what does not, which can also result them looking wrong.

Original JPG file (assuming you mean the Undertale one) is probably broken, because of the definitely broken ICC profile, so it's questionable whether there is anything with 'full support'. Maybe whatever program the artist used to generate this file would load it ""correctly""? (ie. using the same dodgy assumptions that led it to write this incomplete ICC profile)

There are two possible cases: The ICC profile is a broken version of standard sRGB, in which case ignoring the profile is a somewhat adequate solution; or it's a broken version of something other than standard sRGB, in which case you would probably have to talk to the artist to find out.

As a general rule, assuming that most programs ignore ICC profiles -- and thus display the image incorrectly to some degree -- is fairly safe.

Assuming that most programs that support ICC profiles do not have complete support for ICC profiles is also safe (eg. Firefox handles RGB profiles, but I'm not sure about whether it handles Lab profiles or Gray profiles)

Some image viewers apply ICC profiles correctly, but most do not.

Image editors are the most likely to display a non-broken image that has an ICC profile correctly.

The definition of 'apply ICC profile correctly' is also dependent on already having configured a suitable ICC profile for your monitor. The image pixels are interpreted according to the image ICC profile, and translated into the closest approximation available in the monitor ICC profile.

If you have no profile assigned, it's typical for conversion to assume a monitor profile of standard sRGB. This is a compromise; most monitors have some resemblance to sRGB, but it's not necessarily great. sRGB is actually a generalization of CRT display characteristics, not LCD ones.

Professional artists who understand color management may choose to buy a colorimeter, which can be used in conjunction with software to build an accurate ICC profile of their particular monitor at this particular time. I say at this particular time because the monitor's characteristics change over time(thus, if the manufacturer provides an ICC profile, it may be suitable initially, but you would be better off periodically updating it.)

..
Also, a lot -- most, even -- of images do not include ICC profiles, so you have the same problem in reverse: the image was probably made with a colorspace that's not exactly sRGB, and actually could be something quite different, but assuming it is sRGB is the best you can do.

In the case of scans of traditional art, a correct ICC profile for the scanner is also relevant.

TL;DR: Unless both you and the artist have all their ducks lined up, you can have no significant assurance of seeing 'correct color'
(which I think we can agree would generally be, the colors that the image uses on the artist's monitor; the way they intended it.)
My impression is that in the majority of cases, very few of the people concerned even realize that there are ducks, let alone have them lined up.

The answer I'd give to most people is 'use a lcd calibration suite to sort out your monitor's calibration. Unless you're willing to invest a lot more time, that's about as good as you can get.'

EDIT:
This explanation is very basic. Elle Stone's writing on color management go into much more detail; for example 6 articles explaining the questionable level of actual standardization of sRGB, its history, the degree to which it differs from modern LCD monitor characteristics, ..

However it does seem like everything will alter colors to some degree.

All transforms destroy some information, but in a correct workflow, this should not be enough to be visible to the naked eye.

It's necessary to consider at least some of the variables outlined above.

For example, your image editor is most likely to give you a 'correct display'. But this doesn't necessarily mean that you can just export and all will be well.
If the image has a color profile (and it's not sRGB), you should probably convert the image to sRGB color profile before exporting. This is because you cannot trust apps in general to color manage your image correctly, they usually just fling the raw pixel values from the image directly onto the screen.

So sRGB is a kind of lowest common denominator - it won't get you the best display, but the image will probably look kinda decent even if the program displaying it is clueless about color management.

The export looking visibly different from the original file is generally an indicator that something is wrong. You need to compare carefully. For example, if you export from GIMP, and then open the exported image in GIMP, does it look the same as the original image (when both of them are viewed in GIMP). If yes, then whatever other software you were using is probably not properly color-managed; the usual solution for that is to squash it down to sRGB, for the reasons outlined in the above paragraph.
It's also possible that you need to alter export settings (eg. to attach the ICC profile, or to specify the gamma)

Anyway, the broken ICC profile is a more basic problem than discoloration. We're not going to find out what's correct without talking to the artist, because the image itself is almost certainly wrong[1]; I've confirmed that with multiple tools.

[1] to be precise, I mean: the pixels in an image are just numbers, they are not colors; this is a crucial distinction. The color profile (which may be ICC, or something more basic, like simply specifying gamma) defines what color a given set of numbers represents. Without at least some of this information, getting a 'correct' outcome is impossible, because what a correct outcome is has not been defined.

(often it seems you have gotten away with this, because monitors are not totally different from one another -- the artist's display and yours probably display some RGB numbers similarly. You can say the result 'doesn't look too bad', but naturally this is a different thing from 'correct'!)

...

So, the really short version is, most of the images you see are displayed wrongly, it's usually just a matter of how wrong. If you can't prove every step was done correctly, it's pretty likely at least one of them wasn't.

When I open the large images ("Download der Feindaten") at https://www.bzga.de/presse/pressemotive/liebesleben/ with Firefox or Gimp, the colors are OK, like those in the preview images:
https://www.bzga.de/pressemotive.downpix.php?id=5f8a6b3f46a482379a376d5104c61403
https://www.bzga.de/pressemotive.pixbig.php?id=5f8a6b3f46a482379a376d5104c61403

So my first idea was to open the JPG with Gimp and save it again as a JPG.
But I have no idea if that reduces the quality, even at the highest settings.

So I exported it as a PNG.
To my surprise, the PNG has 1.4 MB, while the original JPG has 2.4 MB (landscape version).
The portrait version has 3.5 MB (original JPG) vs. 2.1 MB (exported as PNG).

IIRC I noticed that the title text was a different color (blue rather than the white shown in the sample), though I agree many of the other colors lined up with the preview reasonably well.

PNG is better at compressing cartoony graphics, JPG is relatively bad at it.

And yeah, re-encoding a high-quality JPG would be worst of both worlds: large filesize, and further compression artefacts.