Topic: [Feature] Make it so Cloudflare's protection doesn't get in the way of API requests.

Posted under Site Bug Reports & Feature Requests

Requested feature overview description.
If possible, mess with Cloudflare settings to see if API requests can avoid getting captchas meant for humans, especially if the requests are coming from already logged in users.
Ideally, replacing those captchas with a "403 - here's a unique session cookie, try again in X minutes" would be perfect as it would give whatever tool is using the API a way to recover gracefully, but I doubt Cloudflare can handle such sophisticated traffic control given how maybe 5 requests over an hour (accompanied by valid login cookie and custom user agent, of course) was enough to trigger the stupid captcha.

Why would it be useful?
There is literally zero reason to send the "I'm not a robot" captcha page in response to an API request: APIs are for machines and not for users so there's no one to solve the captcha, to add insult to injury json parsing doesn't require javascript support but the captcha page does.
With how things are right now, even simply testing tag import a handful of times to debug your parser can be a pain in the ass.

What part(s) of the site page(s) are affected?
All API requests probably, only tested /posts.json as that's the API I needed for tag import.

/me wonders if maybe Cloudflare's spamming captchas intentionally on so many sites

Hmm, yeah, sounds like we need some API key to get around this nonsense. Requiring a donation (even $5/year?) would be fair?

pup said:
It's not something they can fix while still having Cloudflare's DDoS protection.

Rate limiting is an option and cloudflare offers a lot of fancy configurations for it, including the ability to use HTTP 429 "retry-after" to cause minimal inconvenience with false positives: see https://support.cloudflare.com/hc/en-us/articles/115001635128 for details.
I would be surprised if the DDOS protection didn't offer a similar amount of options, for example at https://support.cloudflare.com/hc/en-us/articles/200504045 you can find ways to help Cloudlfare play nice with APIs by (among other things) changing user agent checks from the default "block anything that doesn't look like a browser".
There may be reasons why the admins can't use said options, but it would be nice if they took a look at the issue and checked for possible config changes to fix or at least mitigate it: "lol spam request until it goes through", "lol hop IPs until you don't get captcha" and "lol add a web browser to your json parser to solve captchas" aren't great options.

  • 1