Topic: Direct upload whitelist requests

Posted under General

vex714

Privileged

Weibo: 

https://wx1.sinaimg.cn/large/0069wtiRly1g9qehhi5qij32jv3lxh7x.jpg

Updated by anonymous

Versperus

Moderator 
https://baraag.net/system/media_attachments/files

Updated by anonymous

Pup

Privileged

the_real_inaire said:
why is https://twitter.com/ not whitelisted if i may ask?

Siral_Exan said:
It is on the whitelist page, so maybe something changed?

The accepted one is https://pbs.twimg.com, as that's their image domain.

Also, just in case, remember to swap &name=<size> to &name=orig before uploading, and you might as well test to see if format=jpg can be swapped to format=png, just in case there's a better quality.

Wiki link:
https://e621.net/wiki/show/howto:sites_and_sources_-_n_to_z#twitter

Edit:
Thanks Mairo

Updated by anonymous

Mairo

Janitor

Pupslut said:
...and you might as well test to see if format=jpg can be swapped to format=png, just in case there's a better quality.

This is generally not adviced.
The format that's embedded is also the format it's stored on twitters servers, so changing that results either 404 or to files we need to replace, because users change that to PNG without changing the sample size to orig.

Updated by anonymous

donteven

Member

It would appear that Pixiv just changed their image hosting domain to i-f.pximg.net

Updated by anonymous

vex714

Privileged

donteven said:
It would appear that Pixiv just changed their image hosting domain to i-f.pximg.net

yeah, Pixiv did a change and now it says that the direct url is not permitted.

says "tc-pximg01.techorus-cdn.com" is not permitted.

Image still shows up fine if I paste the URL in the "Upload from URL" section.

Updated by anonymous

donteven

Member

Bump for attention.
Direct uploads from Pixiv are still blacklisted due to their domain change.

Updated by anonymous

MyNameIsOver20charac

Member

yFlash said:
This thread should be pinned.

We kinda have a sticky clutter already. Maybe add it to 'collection of interesting forum threads'?

Updated by anonymous

mapachito

Privileged

pixiv is not on whitelist after changing domain.

Uploads from techorus-cdn.com are not permitted. Reason given: techorus-cdn.com not in whitelist

Millcore

Former Staff

mapachito said:
My mistake, I copied the error on the upload form, it only showed the last part. I think the updated domain for pixiv is https://tc-pximg01.techorus-cdn.com/

Example image:
https://tc-pximg01.techorus-cdn.com/img-original/img/2020/03/06/01/10/15/79921171_p0.png

I still got the same error saying: Uploads from techorus-cdn.com are not permitted. Reason given: techorus-cdn.com not in whitelist

Does it work properly now for you? I checked it with the example image you included and it says permitted.

mapachito

Privileged

millcore said:
Does it work properly now for you? I checked it with the example image you included and it says permitted.

Thank you! It's working well now!

user 127567

Member

*.desu-usergeneratedcontent.xyz/*

Image servers for desuarchive.org, which archives posts from a handful of 4chan boards. This includes /trash/, which is where most furry stuff on 4chan is concentrated.

BananaPancake

Member

Redgifs, Gfycat's new site for porn:

https://thumbs1.redgifs.com/*

For a while Redgifs content was actually just hosted on Gfycat.com, but now it's all hosted on Redgifs itself.

EDIT: Hmm, all the Redgifs uploads I've previously seen have been on thumbs1.redgifs but I just found one that's on thcf3.redgifs... so I guess this instead?

https://*.redgifs.com/*

Updated

JAKXXX3

Member

A lot of artists have started using Pillowfort. I'm not certain if this pattern is used by all images on the site, particularly the img3 part.

https://img3.pillowfort.social/posts/*

Mairo

Janitor

jakxxx3 said:
A lot of artists have started using Pillowfort. I'm not certain if this pattern is used by all images on the site, particularly the img3 part.

https://img3.pillowfort.social/posts/*

I would simply wildcard the first part just in case, so it doesn't need to be updated later on.

And yes, I don't know why sudden boom with this site, but many artists are using it right now and the site seems to keep original copies of uploads.

ebea57

Member

Why is Deviantart's domain for images, the wixmp.com, still not whitelisted? Is there any problem with it? This not only blocks direct uploads from Deviantart, but also makes impossible to search similar images by URL with the iqdb.

Watsit

Privileged

ebea57 said:
Why is Deviantart's domain for images, the wixmp.com, still not whitelisted? Is there any problem with it? This not only blocks direct uploads from Deviantart, but also makes impossible to search similar images by URL with the iqdb.

How do you get a usable direct download URL from DA? If I try to right-click the download link and download it directly, I get a 404 (though clicking the link in-browser works). If I right-click the image and copy the image location, I get the resized/preview image from that. Any attempt to get the URL of a full image downloaded in the browser and directly download that results in a 403 (forbidden).

ebea57

Member

watsit said:
How do you get a usable direct download URL from DA? If I try to right-click the download link and download it directly, I get a 404 (though clicking the link in-browser works). If I right-click the image and copy the image location, I get the resized/preview image from that. Any attempt to get the URL of a full image downloaded in the browser and directly download that results in a 403 (forbidden).

Not all images on DA have a download option. Often artists disable it and the only way to download the image is to get the one from preview, which is hosted on wixmp.com domain. Therefore, it should be possible to use a URL in this domain for upload. That's one reason why this domain should be on the whitelist. The other reason, as I mentioned before, is to enable image search with the iqdb for URLs in this domain.

As for download links, they work fine for me, no matter if I click the link or open it directly in a new tab. I only get 404 when I copy the link and try to open it in another browser or incognito mode (which is not surprising since the download option requires login). This makes those links unusable for use anywhere else, because they are valid only for you and for limited time.

I don't know how did you get 403 error. I guess you were trying to modify the URL, which won't work, because DA adds a signature to direct image URLs.

Watsit

Privileged

ebea57 said:
Not all images on DA have a download option. Often artists disable it and the only way to download the image is to get the one from preview, which is hosted on wixmp.com domain. Therefore, it should be possible to use a URL in this domain for upload. That's one reason why this domain should be on the whitelist.

I'm not sure that would be a good idea, as it would encourage people to direct-upload the preview images even when a full size download may be available. Twitter direct upload URLs get rejected if they're the preview or resized version (if it's not the orig image link), so unless there's a way to tell when the link is for the best version available, it seems like it would end up causing lower quality versions to be uploaded that need to be replaced.

ebea57

Member

watsit said:
I'm not sure that would be a good idea, as it would encourage people to direct-upload the preview images even when a full size download may be available. Twitter direct upload URLs get rejected if they're the preview or resized version (if it's not the orig image link), so unless there's a way to tell when the link is for the best version available, it seems like it would end up causing lower quality versions to be uploaded that need to be replaced.

If this is the only concern, then there should be a separate whitelist for uploads and the reverse image search. It is annoying, that you have to download and upload the image every time you want to check it instead of just entering URL. Popular sites like DA should be allowed for this purpose.

jockjamdoorslam

Member

mairo said:
I would simply wildcard the first part just in case, so it doesn't need to be updated later on.

And yes, I don't know why sudden boom with this site, but many artists are using it right now and the site seems to keep original copies of uploads.

+1 for adding Pillowfort. I've seen a few artists start posting there. I guess its appeal is being like Tumblr before they nuked the porn.

Penguinempire-Dennis

Member

jakxxx3 said:
A lot of artists have started using Pillowfort. I'm not certain if this pattern is used by all images on the site, particularly the img3 part.

https://img3.pillowfort.social/posts/*

strikerman said:

https://baraag.net/*
https://img.pawoo.net/*

too, Pawoo is affiliated with Pixiv and liable to see a lot of art submissions.

Also, should Palcomix really be on the whitelist when the whole site is DNP?

user 761964

Member

I know I'm probably REALLY late to this party, but I'm trying to upload on this site, and the "Source" I looked up from the artist gallery on Furaffinity was not whitelisted when I tried to upload it. Can someone explain to me how a whitelist can be obtained, and what I must do so I can upload the portrait?

Millcore

Former Staff

imagaygermanshep said:
I know I'm probably REALLY late to this party, but I'm trying to upload on this site, and the "Source" I looked up from the artist gallery on Furaffinity was not whitelisted when I tried to upload it. Can someone explain to me how a whitelist can be obtained, and what I must do so I can upload the portrait?

You didn't provide a domain or link example.

bitWolfy

Former Staff

jockjamdoorslam said:
It looks like FurAffinity have changed how their images are hosted, they're using https://d.furaffinity.net/... instead of d.facdn.net. Changing the url manually to facdn still works for now though.

That was a stopgap to avoid the recent cloudflare issue. I presume that it will return to normal at some point in the near future.

Strikerman

Janitor

bitwolfy said:
That was a stopgap to avoid the recent cloudflare issue. I presume that it will return to normal at some point in the near future.

Who knows when "at some point" is, though. Might as well whitelist the other domain for convenience's sake.

Watsit

Privileged

strikerman said:
Who knows when "at some point" is, though. Might as well whitelist the other domain for convenience's sake.

FWIW, in the mean time you can replace the d.furaffinity.net with d.facdn.net in the URL you're trying to upload from and it will still work... as long as cloudflare isn't acting up at the time.

faucet

Member

bitwolfy said:
That was a stopgap to avoid the recent cloudflare issue. I presume that it will return to normal at some point in the near future.

May as well whitelist it anyway, "soon" by Fur Affinity standards could even be a few years. I had no idea all day why the reverse image search tool wasn't working until I saw this thread, I'm sure I'm not the only one in this situation.

EDIT 2022-08-05: Over a year and a half later, it still hasn't returned to normal. That was a good guess.

Updated

Alien Fluff

Janitor

booruhitomi said:
VK has since changed their URL format:

https://sun*-*.userapi.com/*

I'm still waiting for VK's whitelist entry to be updated; I just recently discovered an artist who has a whole bunch of artwork on there that I want to post.

Update (2021-04-15): I just found out that changing the prefix to the already whitelisted pp still works and will grab the image no problem.

Updated

Alien Fluff

Janitor

I really think we should finally get around to adding https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/* to get direct images from DeviantArt. Let me explain why.

I've recently messaged an artist about why their uploads are coming from FA where it's downscaled and converted to JPEG as is typical (and I told them about the downscaling and the workaround, which I don't know if they're going to do so in the future) instead of from their DA where the full images are available to download.

Turns out that they're Russian and have to use a VPN to access eSix. They use the URL upload to post their art here because directly uploading the full PNGs over here takes a lot of time through their VPN, so it's not worth it for them. And since DA's current URL pattern is not permitted, their only choice is uploading from an FA URL. (I don't know of any other art gallery sites they upload to, but it seems that it's only FA and DA for them.)

I will say that some artists do allow for viewing the full-size images directly on a deviation (DeviantArt submission) page by clicking on the image twice, so it's still possible to at least get the direct images from there sometimes.

Xerxes I

Member
https://trello-attachments.s3.amazonaws.com/*

requested because some artists use trello for the whole commission process, including posting the final piece

Mairo

Janitor

booruhitomi said:
I really think we should finally get around to adding https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/* to get direct images from DeviantArt. Let me explain why.

I've recently messaged an artist about why their uploads are coming from FA where it's downscaled and converted to JPEG as is typical (and I told them about the downscaling and the workaround, which I don't know if they're going to do so in the future) instead of from their DA where the full images are available to download.

Turns out that they're Russian and have to use a VPN to access eSix. They use the URL upload to post their art here because directly uploading the full PNGs over here takes a lot of time through their VPN, so it's not worth it for them. And since DA's current URL pattern is not permitted, their only choice is uploading from an FA URL. (I don't know of any other art gallery sites they upload to, but it seems that it's only FA and DA for them.)

I will say that some artists do allow for viewing the full-size images directly on a deviation (DeviantArt submission) page by clicking on the image twice, so it's still possible to at least get the direct images from there sometimes.

The problem here is that those URLs have tokens attached to them, so if you take the URL and share it with someone, they cannot open that link because the token in it is attached to your connection specifically. What this means is that e621 servers cannot fetch the content either, hence why it's not whitelisted.

There would be couple solutions: instruct the author to upload their content at full quality to FA first by resubmitting it there.
Another would be for artist to use other source, e.g. Discord, Imgur, Puush or Dropbox to upload their file.

bitWolfy

Former Staff

mairo said:
The problem here is that those URLs have tokens attached to them, so if you take the URL and share it with someone, they cannot open that link because the token in it is attached to your connection specifically. What this means is that e621 servers cannot fetch the content either, hence why it's not whitelisted.

Well, https://images-wixmp*.wixmp.com/* has been on the whitelist for about a week now.
I haven't tested whether uploading from it actually works, though.

Alien Fluff

Janitor

bitwolfy said:
Well, https://images-wixmp*.wixmp.com/* has been on the whitelist for about a week now.
I haven't tested whether uploading from it actually works, though.

Really? How come nobody said anything until now?

Anyway, I just tried this out with a DA post and it does work, complete with token.

ShadyGuy

Privileged

It's been a while since Pillowfort launched, and unlike Twitter it keeps the original png files. It's already been suggested by a few people already

Alien Fluff

Janitor

shadyguy said:
It's been a while since Pillowfort launched, and unlike Twitter it keeps the original png files. It's already been suggested by a few people already

Pillowfort is actually not a great source because it changes the color space to sRGB no matter what. Here's a two-frame animation showing the differences between post #2349805 and the version from its Pillowfort source.

ShadyGuy

Privileged

booruhitomi said:
Pillowfort is actually not a great source because it changes the color space to sRGB no matter what. Here's a two-frame animation showing the differences between post #2349805 and the version from its Pillowfort source.

Good to know, thanks. So it should only be used if the image is higher resolution compared to any other source, despite being a .png

Alien Fluff

Janitor

DeviantArt direct download URLs (only possible to see using Viewhance's forced media inlining feature):

https://api-da.wixmp.com/*

Gyro

Member

While aryion is good to have, can we get a blacklist for urls of this format?

https://aryion.com/g4/derivative/*

anything in the derivative is the small thumb, rendered in JPEG and needs to be expanded to get the full size image. I've messed this up a couple times, forgetting to expand, and there's no reason to do these files on the site.

ebea57

Member

https://rule34.xxx/ needs an update. The image domain img.rule34.xxx is no longer used. Now it's wimg.rule34.xxx.

I suggest using https://*.rule34.xxx/* as they change the domain prefix every once in a while.

Updated

ShadyGuy

Privileged

I think images on F-List are uncompressed full-res, https://static.f-list.net/images/charimage/*.

gattonero2001

Janitor

Weibo: weibo.com -> https://wx1.sinaimg.cn/original/*

Pawoo: pawoo.net -> https://img.pawoo.net/media_attachments/files/*/original/*

Baraag: baraag.net -> https://baraag.net/system/media_attachments/files/*/original/*

faucet

Member

Not sure if this thread is still checked but... support for ArtFight would be useful - it seems to host the original PNG files. The gallery is hosted all year round rather than just when the event is running.

https://images.artfight.net/*

faucet

Member

alien_fluff said:
VK has since changed their URL format:

https://sun*-*.userapi.com/*

alien_fluff said:
I'm still waiting for VK's whitelist entry to be updated; I just recently discovered an artist who has a whole bunch of artwork on there that I want to post.

Update (2021-04-15): I just found out that changing the prefix to the already whitelisted pp still works and will grab the image no problem.

Bringing this one up again, while the workaround does work it's a little annoying to have to do it every time I want to reverse search an image. Whitelisting https://*.userapi.com/* would probably be the perfect solution as it'll handle both the new and old image URLs.

faucet

Member

Not sure what the difference is but some Discord content seems to be media.discordapp.net rather than cdn.discordapp.com, which is the only one on the whitelist. The URLs do seem to be interchangable.

Earlopain

Admin

alien_fluff said:
I see a good amount of artwork on Reddit:

https://i.redd.it/*

alien_fluff said:
VK has since changed their URL format:

https://sun*-*.userapi.com/*

Done. And it only took 2 years.

ebea57 said:
https://rule34.xxx/ needs an update. The image domain img.rule34.xxx is no longer used. Now it's wimg.rule34.xxx.

I suggest using https://*.rule34.xxx/* as they change the domain prefix every once in a while.

shadyguy said:
I think images on F-List are uncompressed full-res, https://static.f-list.net/images/charimage/*.

gattonero2001 said:
Weibo: weibo.com -> https://wx1.sinaimg.cn/original/*

Pawoo: pawoo.net -> https://img.pawoo.net/media_attachments/files/*/original/*

Baraag: baraag.net -> https://baraag.net/system/media_attachments/files/*/original/*

I think Nimmy did these a while ago.

faucet said:
Not sure if this thread is still checked but... support for ArtFight would be useful - it seems to host the original PNG files. The gallery is hosted all year round rather than just when the event is running.

https://images.artfight.net/*

Sure.

faucet said:
Not sure what the difference is but some Discord content seems to be media.discordapp.net rather than cdn.discordapp.com, which is the only one on the whitelist. The URLs do seem to be interchangable.

Reading up on it it looks like media.discordapp.net is their image variant processor. cdn vs media example, notice the extra url parameters. So it would be best to just always switch them out to the cdn url.

  • 1
  • 2
  • 3