Topic: [Bug/Fixed] Pixiv uploads by url are forbidden (403 error)

Posted under Site Bug Reports & Feature Requests

This topic has been locked.

Bug overview description.
When uploading from pixiv by URL (i.pximg.net/img-original/img/YYYY/MM/DD/HH/SS/*_p0.png), images fail to upload.

What part(s) of the site page(s) are affected?
Uploading images by URL.

What is the expected behavior?
Image address shows proper image, image preview functions properly, submission works.

What actual behavior is given instead?
Opening the image address in the browser in which you are logged in to Pixiv shows the image properly. Opening the image address in a different browser (logged out of Pixiv) shows the error. Similarly, the image preview in e621 works in the browser in which you are logged in to Pixiv, and fails ("Not Found") if done in a different browser. Attempting to submit the image by URL in either browser fails, returning an error message from ee621 indicating the image is forbidden (403 error).

Can you reproduce the bug every time?
Yes.

What steps did you take to replicate this bug?
Used Chrome to log in Pixiv, Firefox to remain logged out. Logged out from e621 from one browser before logging in and testing in the other.

Errors or other messages returned (if any).
Error: Source couldn't be opened: HTTP error code: 403 Forbidden

Updated by Chaser

Well crap. That's going to be hard to fix. Can you please provide an example pixiv illustration url and the resulting direct image url for testing purposes.

Updated by anonymous

I found some info that may be helpful. Clicking the image in Pixiv only maximizes the resolution and removes the rest of the page. You have to right click and open it in a new tab or new window to get the actual image url or it will only give you a regular preview page. The page itself doesn't change from clicking to full-view.

The thing is, you do get a different page if you open in a new tab after maximizing the resolution than you do if you clicked before.

I just tested the preview on this site stopping just before I would have hit the upload button. The image showed up and it said the image was allowed. I'm not sure why, since it was the same format, though I'm wondering if it might have worked because I was using an image that pixiv considered SFW.

Updated by anonymous

kamimatsu said:
I found some info that may be helpful. Clicking the image in Pixiv only maximizes the resolution and removes the rest of the page. You have to right click and open it in a new tab or new window to get the actual image url or it will only give you a regular preview page. The page itself doesn't change from clicking to full-view.

The thing is, you do get a different page if you open in a new tab after maximizing the resolution than you do if you clicked before.

I just tested the preview on this site stopping just before I would have hit the upload button. The image showed up and it said the image was allowed. I'm not sure why, since it was the same format, though I'm wondering if it might have worked because I was using an image that pixiv considered SFW.

They stated the url which fails is i.pximg.net/img-original/ which is direct image url for initial version.

Updated by anonymous

KiraNoot said:
Well crap. That's going to be hard to fix. Can you please provide an example pixiv illustration url and the resulting direct image url for testing purposes.

Sorry for the late reply.

Example 1: Bears (single image)
Page hosting the image : https://www.pixiv.net/member_illust.php?mode=medium&illust_id=51647509

Direct image URL : https://i.pximg.net/img-original/img/2015/07/28/01/38/11/51647509_p0.png

Example 2: More bears (one image of an image series)
Page hosting the images : https://www.pixiv.net/member_illust.php?mode=manga&illust_id=62089689

Blowup of one of the images : https://www.pixiv.net/member_illust.php?mode=manga_big&illust_id=62089689&page=0

Direct URL of the same image in the image series : https://i.pximg.net/img-original/img/2017/03/25/21/21/36/62089689_p0.png

Imgur

Updated by anonymous

This is hotlink protection? Do any of the workarounds described here work for Pixiv?

Updated by anonymous

Normal referer spoofing, eg wget --referer='https://www.pixiv.net/member_illust.php?mode=manga_big&illust_id=62089689&page=0' https://i.pximg.net/img-original/img/2017/03/25/21/21/36/62089689_p0.png, works.

Updated by anonymous

So I can fix the server side, but the preview image is not going to work. Bleh.

Updated by anonymous

KiraNoot said:
So I can fix the server side, but the preview image is not going to work. Bleh.

Generate a warning up top when such a url is posted?

Updated by anonymous

KiraNoot said:
So I can fix the server side, but the preview image is not going to work. Bleh.

It doesn't generate one when posting from twitter :orig either, so that's not that huge issue.

Updated by anonymous

So, if and when the issue is resolved, could a formal announcement be made? It still has not, based on the problems I'm still facing when uploading from pixiv.

Updated by anonymous

A fix is sitting in approval. :D

NOTE: Code approval takes a few days as sometimes changes are required. Deploy can take a few weeks(but not always).

Updated by anonymous

Chaser said:
[...]

i am intensely afraid of any infrastructure that requires "a few weeks" to push a code change.

Updated by anonymous

fewrahuxo said:
i am intensely afraid of any infrastructure that requires "a few weeks" to push a code change.

Multiple staff have to look it over and make sure it isn't going to add any vulnerabilities. AFAIK, this is the first time we're including an Unless fix.

Updated by anonymous

fewrahuxo said:
i am intensely afraid of any infrastructure that requires "a few weeks" to push a code change.

We like to bulk up updates, as it results in less downtime and lets us make sure all of them are safely compatible and lets us make last minute changes if needed.
If there are security vulnerabilities on the live code, code is pushed as soon as possible to fix it.

Keep in mind there is only like 2 active devs right now, and Kira is working on f-list, and I am working on migrating my site slowly to postgresql. THERE MIGHT be more but I don't see anyone else chat in the dev channel much.

Rest assured, we do this to make sure the community gets the best experience possible.

Updated by anonymous

  • 1