Topic: Using a get request on posts.json throws an error no matter what I do

Posted under e621 Tools and Applications

I am trying to implement a tool to help me filter through posts more easily, and I am trying to pull posts from the site, however the fetch command always returns an error. I have looked up these errors and I cannot find a method to get it to return anything.

In the past, I have used the endpoint specified in a promised fetch, looking like this on my first attempt:

fetch("https://www.e621.net/posts.json?tags=male&limit=5", {mode: 'no-cors'})
    .then(response => {
         if(response.ok) return response.json()
        throw new Error("Request failed")
    }, networkError => {console.log(networkError.message)})
    .then(jsonResponse => {console.log(jsonResponse)})

However, this threw an error of "Request failed". I could not find anything wrong with my breakpoint or policy, so I added a console log for response, and I get an object that doesn't have the ok status. I did some googling, and found out that the no-cors policy is causing the returned response to be opaque, meaning it doesn't have a body to process. However, when I remove this header there is an error saying that I do not have access to the endpoint. I have even tried adding the credentials for my user as a header, but it also didn't help.

Other programs I have written use other endpoints and I have not come across this issue with them before. What should I do?

No-CORS is explicitly for getting the opaque response. If you want to avoid the issue, you have to run the code in an environment where CORS doesn't apply; if I remember correctly, browser extensions can bypass it, or you can always run the code outside of a browser.

Edit: browser extensions can definitely bypass it. After all, the RE621 user script makes quite a few requests.

scth said:
No-CORS is explicitly for getting the opaque response. If you want to avoid the issue, you have to run the code in an environment where CORS doesn't apply; if I remember correctly, browser extensions can bypass it, or you can always run the code outside of a browser.

Edit: browser extensions can definitely bypass it. After all, the RE621 user script makes quite a few requests.

How do I use these? I would prefer to use a purely programmatic approach if possible, and I have been able to do this in the past and I cannot find a difference

One solution is to use literally any other language; pretty much only JavaScript inside of a web browser respects CORS. It may have worked on other pages due to them including an access-control-allow-origin header.

Oddly, I do get an Access-Control-Allow-Origin header while trying the request manually outside of JavaScript, so not sure what's going on there.

alteria said:
It's not working, the same errors keep showing up

Odd, you could technically just use corsproxy.io. The reason re621 can make requests is because it's same origin, the requests come from e621 itself, so there's no issue of CORS. Same reason you can execute:

(async() => {
    console.log(await (await fetch("https://e621.net/posts.json")).json())
})()

from the console while on e621, but not from another website. If you can't do these requests from the back end, you would need to proxy it. To be honest, having CORS on an API is one of the oddest things about this site, but then again the site also puts a human verification over the API sometimes as well, so I think it's an artifact of this site working from a pretty old codebase...

  • 1