I know it's been said, but I get the occasional 500.

Not sure if it's just me but I'm still getting the occasional 502 error when using my ipad (IOS 6.2.1 if that matters) on Safari but if I use an app other than safari I've yet to encounter an error.

Love the new upgrade, I can see a major speed increase.

Nice to know it's just my wifi messing up now and not the site. Nice job guys.

yes it is running great very fast load times are great

Just started getting an occasional 404 error as of about 5 minutes ago. Seems to pop up on any page. Sometimes it will stick around even through a refresh, but a Shift+F5 usually clears it.

Edit: Actually Shift+F5 doesn't seem to work consistently either. It's starting to become more frequent now as well.

Edit 2: No more 404s :)

every so often i get these 404 Not Found errors wine i change pages the site is no longer slow but it i have started getting those 404 errors

Go to any page.
Random 404. Refresh. 404'd. Refresh again.
Works then.

I'm getting a bunch of 404 errors lately on pages that definitely exist. The errors go away after a reload, but still.

Recently, i´m getting an error:404 not found

I seem to be getting very few errors. The site is running much faster(even on an old computer). The errors are to be expected, as this is new upgrade and they JUST did it.

I haven't seen any more errors lately, and yes, the site is significantly faster.

Suineder said:
I haven't seen any more errors lately, and yes, the site is significantly faster.

The 404's were caused by Dari bringing another server online. They shouldn't show up again any time soon.

Captain_No_Name said:
I seem to be getting very few errors. The site is running much faster(even on an old computer). The errors are to be expected, as this is new upgrade and they JUST did it.

Yeah, they were just bringing the last of them online at the time, so everything wasn't quite "in sync" so-to-speak. I don't want to say that the errors are "normal" exactly because, ideally, the changeover would all be transparent on to you guys, but it's nothing to actually worry about since they are only temporary.

But as far as I know they are done messing with it, so there shouldn't be any more errors like that; if anyone is still seeing them pop up frequently let us know as soon as possible.

NotMeNotYou said:
The 404's were caused by Dari bringing another server online. They shouldn't show up again any time soon.

Awesome!

It seems like some pictures wont load after the upgrade, it displays a broken picture icon.

Griffonex said:
It seems like some pictures wont load after the upgrade, it displays a broken picture icon.

What browser (Firefox, Internet Explorer, etc.) and OS (Windows 7, OS X Mavericks, etc.) are you running?

parasprite said:
What browser (Firefox, Internet Explorer, etc.) and OS (Windows 7, OS X Mavericks, etc.) are you running?

Dolphin browser on android 4.4.2, jetpack disabled.
It mostly happens to older pictures though (some on thumb, some on the full pictures) heres a example picture that doesnt load for me: https://e621.net/post/show/400427. If i try to download it, it returns error 400:bad request

Ps: did a cache clean, still doesnt load

Griffonex said:
Dolphin browser on android 4.4.2, jetpack disabled.
It mostly happens to older pictures though (some on thumb, some on the full pictures) heres a example picture that doesnt load for me: https://e621.net/post/show/400427. If i try to download it, it returns error 400:bad request

Ps: did a cache clean, still doesnt load

• Disable adblock. If it works now, it may be related to a bad entry in adblock's cache (I'm not entirely sure how to clear the adblock cache on Android Dolphin, but it may be worth looking into).
• Browse in a private tab. If it works now, it may be related to a corrupt cookie. You should be able to fix it by simply wiping your cookies from within the app itself (note that this may log you out of other websites).

My posts still happen to have no space after .!, and ?s. I have to edit and save to fix it.

Sorrowless said:
My posts still happen to have no space after .!, and ?s. I have to edit and save to fix it.

You can actually just reload the page, you don't have to edit the post. It's actually just a temporary display issue and the post will show up normally for everyone else without a reload. It's mildly annoying but it doesn't actually affect anything.

I'm not sure where they are at as far as fixing it, but I'll see if I can work with the devs to help resolve this.

parasprite said:

• Disable adblock. If it works now, it may be related to a bad entry in adblock's cache (I'm not entirely sure how to clear the adblock cache on Android Dolphin, but it may be worth looking into).
• Browse in a private tab. If it works now, it may be related to a corrupt cookie. You should be able to fix it by simply wiping your cookies from within the app itself (note that this may log you out of other websites).

Its fixed now, thanks!

I'm finding the upgrade has really improved the surfing experience.

It does seem a lot more stable so far.

performance has always been good on my end, fast connection and download speeds, very rarely get the cloudflare error. other than that, purrfect (sorry, i couldn't help it).

thanks to all the Admins that maintain this amazing website. you guys are the greatest.

parasprite said:
You can actually just reload the page, you don't have to edit the post. It's actually just a temporary display issue and the post will show up normally for everyone else without a reload. It's mildly annoying but it doesn't actually affect anything.

I'm not sure where they are at as far as fixing it, but I'll see if I can work with the devs to help resolve this.

Good to know. Thanks.

getting 500's once in awhile, but running a lot more smoothly.

So that's why everything feels faster now.

Thanks for the new CPU's :3

It's working great for me. Super fast. Hopin' it stays this way.

Congratulations e621, your cpu is shit. I'm getting lag like a mofo

Goombafan333 said:
Congratulations e621, your cpu is shit. I'm getting lag like a mofo

I hate to have to agree with something so bluntly negative and unhelpful, but I'm having similar issues here. Frequently, pages are taking a really long time before it even starts to load. It seems to load quickly once it does, but before that happens it just... Hangs.

I feel the need to beg the administrators and developers of this website to truly investigate and trace these performance problems. I have a feeling it is *not* hardware, and that something in the software is causing a lot of lag. There is quite possibly a fundamental architecture problem.

Please investigate this seriously. Even if there are areas of the site you KNOW work perfectly fine, don't rule them out as possible problems. Don't be afraid to be wrong.

Tynach said:

Please investigate this seriously. Even if there are areas of the site you KNOW work perfectly fine, don't rule them out as possible problems. Don't be afraid to be wrong.

We know exactly where the problem is, it's that clusterfuck of an unoptimized database that was never intended to reach 650k images in combination with this amount of tagging.
The real problem however is that we can't just optimize the database without also needing to replace almost everything else on the back end. And this is sadly an undertaking that will take quite some time longer than we could wait for.

So yes, this is far from over, but it should be at least bearable to get us until we can solve this at the root.

If it's interesting to anybody else, this is a problem that all *booru style image boards have (for example the reason why paheal doesn't allow any tags besides the franchise), with the exception of derpibooru because they wrote the entire thing from scratch, with a very large database in mind.

It is quiet faster but it was fast for me even before.

Goombafan333 said:
Congratulations e621, your cpu is shit. I'm getting lag like a mofo

Ever considered that your internet connection might be shit?

I don't see a single error or problem feels about the same.... But not a single lag with flash!

Prolly I won't be the first to say this, buuuuuuuuuuuuuuuuuuut I'm having some 522's and timeouts since few minutes ago

I haven't noticed any site errors since the upgrades!

So now that we don't have to worry about server load, how are talks on the tag search limit?

It has a lot of loading failures.

NotMeNotYou said:
... with the exception of derpibooru because they wrote the entire thing from scratch, with a very large database in mind.

Really? I had no idea that they had. I've gotten semi-good at SQL and so forth these days, and I've been tempted to write my own *booru from scratch before... Didn't realize that Derpibooru had already beaten me to it!

I am curious though, what is the current e621 database schema like? What are the actual problems with it that are causing these issues? I have a sinking feeling that what I've thought up as a good tagging system is essentially what you have, and I'll run into similar pitfalls when it comes time to implement it. I'd also just in general like to know what all exists out there already.

Tynach said:
Really? I had no idea that they had. I've gotten semi-good at SQL and so forth these days, and I've been tempted to write my own *booru from scratch before... Didn't realize that Derpibooru had already beaten me to it!

I am curious though, what is the current e621 database schema like? What are the actual problems with it that are causing these issues? I have a sinking feeling that what I've thought up as a good tagging system is essentially what you have, and I'll run into similar pitfalls when it comes time to implement it. I'd also just in general like to know what all exists out there already.

Tags on any post are stored inside a string, not inside an array, and other crap.
If you want more information feel free to download the danbooru source code and look at it yourself.

NotMeNotYou said:
Tags on any post are stored inside a string, not inside an array, and other crap.

I.. I see.

I understand why it'd be a pretty big undertaking to convert an entire existing website's database to something that isn't... That. Have there been any discussions on a migration path? While I'm tempted to say, "Nuke it from orbit," I don't want to lose all the porn.

I have some experience (through personal projects though, not 'real' work experience) designing database schemas, but I've never had to migrate data from one schema to another. Is there any way I could help? And since there probably isn't, is there any way for people who have more of a clue to get involved so they could help?

Tynach said:
I.. I see.

I understand why it'd be a pretty big undertaking to convert an entire existing website's database to something that isn't... That. Have there been any discussions on a migration path? While I'm tempted to say, "Nuke it from orbit," I don't want to lose all the porn.

I have some experience (through personal projects though, not 'real' work experience) designing database schemas, but I've never had to migrate data from one schema to another. Is there any way I could help? And since there probably isn't, is there any way for people who have more of a clue to get involved so they could help?

Spight and Varka are already working on it, converting the database is one thing, that database then needs play nice with the rest of the backend, which is why it's taking so long, along the way we would like to update our current ruby version as well (another performance gain) and that is also causing slight problems.
Nuking it from orbit has been seriously considered, but importing everything would take just as long, so we just go with this route.
Alas, we have no ETA to share yet, the changed source code from Arcturus and aurali doesn't make it easier since those need to be basically reverse engineered as well.

NotMeNotYou said:
Spight and Varka are already working on it, converting the database is one thing, that database then needs play nice with the rest of the backend, which is why it's taking so long, along the way we would like to update our current ruby version as well (another performance gain) and that is also causing slight problems.
Nuking it from orbit has been seriously considered, but importing everything would take just as long, so we just go with this route.
Alas, we have no ETA to share yet, the changed source code from Arcturus and aurali doesn't make it easier since those need to be basically reverse engineered as well.

I understand. Thank you for the transparency, it sounds like you guys really know what you're doing :)

One thing I've noticed from the Danbooru sources, is that it does use Ruby On Rails' own database management - which includes the schema creation. I sadly have zero knowledge of Ruby or RoR, so I see myself as even less and less qualified than I had already thought.

Most of my 'experience' with databases has been with raw SQL, particularly with MySQL (and it seems RoR uses PostgreSQL). Heck, most of it has been specifically on this giant blob of hand-written SQL, which I've not worked on in a while and I'm sure is probably full of normalization problems.

As far as an ETA is concerned, all that matters is that it is being worked on - which it seems it certainly is. That is wonderful news, and I look forward to seeing the result!

NotMeNotYou said:
Tags on any post are stored inside a string, not inside an array, and other crap.

Not quite. Tags are stored how they should be (in a tag_id -> post_id junction table) and there's also a string column in the posts table containing a plaintext list of the tags for caching purposes, but a lot of the backend stuff could be rewritten to improve performance.

TonyLemur said:
Not quite. Tags are stored how they should be (in a tag_id -> post_id junction table) and there's also a string column in the posts table containing a plaintext list of the tags for caching purposes, but a lot of the backend stuff could be rewritten to improve performance.

Ah, ok. That does make it sound significantly better, and... Rather similar to what I have (minus the plaintext list for caching). What sorts of things cause performance problems with this sort of setup?

You guys should do something silly/cool like ehentai did with the hentai@home thing. Let your users be your bandwidth ontop of your sites own bandwidth :P I mean i have 100/100 FTTH that does nothing 90% of the day id gladly donate some of it if i could. Sure others would as well. Probably waaaaaaaaayyyy too hard to setup tho.

LordKitsuna said:
You guys should do something silly/cool like ehentai did with the hentai@home thing. Let your users be your bandwidth ontop of your sites own bandwidth :P I mean i have 100/100 FTTH that does nothing 90% of the day id gladly donate some of it if i could. Sure others would as well. Probably waaaaaaaaayyyy too hard to setup tho.

We've already got CloudFlare, which is pretty much more bandwidth than we'll ever need. Bandwidth has become an utter non-issue now because of them.

Hey, I had this problem a lot before the servers got upgraded and then it went away but now it's back.

I've been getting this error almost every other time I open an e621 link and it's making it really difficult to post stuff.

Sneaky said:
Hey, I had this problem a lot before the servers got upgraded and then it went away but now it's back.

I've been getting this error almost every other time I open an e621 link and it's making it really difficult to post stuff.

Yeah, I'm getting a lot of 502s too. It started happening to me yesterday but nowhere near as bad is it is right now.

We just disabled view count updating on posts since it was the thing causing all of the recent problems. Once we streamline it they'll be available again.

I have so many things blacklisted, that sometimes, I have a completely blank page, I think that this site needs a setting that makes it shows 75 (or whatever # someone has per page) of non-blacklisted pictures, etc. Instead of having a lack of pictures. If this is already a setting, show me where it is.

MintyMofo said:
I have so many things blacklisted, that sometimes, I have a completely blank page, I think that this site needs a setting that makes it shows 75 (or whatever # someone has per page) of non-blacklisted pictures, etc. Instead of having a lack of pictures. If this is already a setting, show me where it is.

Not a solution, but it should decrease your problem slightly (unless you've already done this): You can increase the number of posts per page before blacklist is enforced on your settings page, just below the blacklist settings.

I can't access e621.net. I'm posting through e926.net, which I can (obviously) access to. When I try e621 I get 522 connection timed out error. Halp plz.

Azarboc said:
I can't access e621.net. I'm posting through e926.net, which I can (obviously) access to. When I try e621 I get 522 connection timed out error. Halp plz.

I've noticed that users have been reporting similar issues are from certain geographical locations and I suspect it may be related to the connection between one of Cloudflare's nodes and the site's servers. However, I need to know a couple things to help narrow it down a bit more.

• Are you using a proxy?
• Visit this page (Edit: Fixed link), copy the line colo=***, and paste it here (the *** will be 3 letters (ABC), and it will tell us which Cloudflare node you are connecting to)

parasprite said:
I've noticed that users have been reporting similar issues are from certain geographical locations and I suspect it may be related to the connection between one of Cloudflare's nodes and the site's servers. However, I need to know a couple things to help narrow it down a bit more.

• Are you using a proxy?
• Visit this page, copy the line colo=***, and paste it here (the *** will be 3 letters (ABC), and it will tell us which Cloudflare node you are connecting to)

No I don't use a proxy

Also, don't know if I did something wrong or what but I got that data by replacing e926.net with e621.net because the e926 one just said that I either typed the URL wrong or someone gave me a bad link.

Azarboc said:
No I don't use a proxy

colo=MIA

Also, don't know if I did something wrong or what but I got that data by replacing e926.net with e621.net because the e926 one just said that I either typed the URL wrong or someone gave me a bad link.

That one is my fault. I have a habit of using relative URLs (without e621/e926) since it prevents people from bouncing back and forth between sites, but I forgot that e926 connects differently so that specific link won't work on there (it's actually a Cloudflare page, it has nothing to do with us).

However, since you already found the correct link and info, I'll forward it to our sysadmin to see if they can figure out what is going on.

parasprite said:
That one is my fault. I have a habit of using relative URLs (without e621/e926) since it prevents people from bouncing back and forth between sites, but I forgot that e926 connects differently so that specific link won't work on there (it's actually a Cloudflare page, it has nothing to do with us).

However, since you already found the correct link and info, I'll forward it to our sysadmin to see if they can figure out what is going on.

Hey. Wanted to let you know that I discovered I can access 621.net if I use mobile data in my cellphone instead of using my home's network and checking the Cloudflare node it seems to be the same one. Dunno if this info is useful but I thought I'd let you know.

is it just me or is the site loading rather slowly? i think it started some time yesterday or the day before for me.

For the past day I've been having connection issues (error 500s etc.). Now I'm having this weird problem where if I open e621 and go to the posts list, it works fine. If I have open a post in a new tab that works fine too. However if I then proceed to open a second post in a new tab, it loads for a long time before giving me a timed out error (err_timed_out or 504). A few seconds after that it loads by itself into the page I want and everything else in the page is normal except the image itself is broken and I have to reload it again. The weird thing is that it only ever happens to the second image I open. If I open any subsequent posts it works fine, if a bit slowly, but if I close all e621 tabs and open e621 again in an entirely new tab the problem happens again, again on the second page I open.

I hope this is the right thread. Thanks for the help.

We're aware of the connection issues and are working for a fix.

Kyubii said:
Been getting a lot of 500 errors in the last hour or so.

Kataphraktarii said:
For the past day I've been having connection issues (error 500s etc.). Now I'm having this weird problem where if I open e621 and go to the posts list, it works fine. If I have open a post in a new tab that works fine too. However if I then proceed to open a second post in a new tab, it loads for a long time before giving me a timed out error (err_timed_out or 504). A few seconds after that it loads by itself into the page I want and everything else in the page is normal except the image itself is broken and I have to reload it again. The weird thing is that it only ever happens to the second image I open. If I open any subsequent posts it works fine, if a bit slowly, but if I close all e621 tabs and open e621 again in an entirely new tab the problem happens again, again on the second page I open.

I hope this is the right thread. Thanks for the help.

treos said:
is it just me or is the site loading rather slowly? i think it started some time yesterday or the day before for me.

The 50x errors and slowness we've been seeing today are mostly related to the work/troubleshooting being done on the site for an unrelated issue where certain users have been unable to connect to the site for the last month or so (if you aren't getting 522 errors on every page load, it probably doesn't affect you).

I don't have much more information to give at this point, but I'll let you know if I find anything new out.

no 50x errors here just slowness so i guess i'm just seeing that side effect is all.

I'm not getting any 5XXs either but sometimes interactive posts will cause Firefox to either crash or freeze if clicked.

The site is still very slow for me and I'm getting a lot of timeout errors (Err_timed_out, Err_connection_timed_out) whenever I try to do anything :( Is it just me or is the server still having trouble? If it's the server any idea when it will be fine again?

Kataphraktarii said:
The site is still very slow for me and I'm getting a lot of timeout errors (Err_timed_out, Err_connection_timed_out) whenever I try to do anything :( Is it just me or is the server still having trouble? If it's the server any idea when it will be fine again?

Not just you, many timeouts and 500 errors and about twenty minutes got me to this page, hopefully this actually posts too.

Our host is still switching switches as if they believe it's needed to only run them for like 24h tops.

We do hope they stop doing that soon.

The last few days I've been noticing several 5xx errors along with a generally increased loading time. Saw a nginx gateway error trying to open a this page actually.

Alright, so it's not just on my end, good to know. It's terribly slow for me though, honestly, sometimes it takes up to ten minutes to load one page. I'm not kidding either when I say that every other page gives me a timeout error, even accessing the main page or my account homepage gives me timeouts too.

Hope the servers will get better soon, it's really difficult to use the site when it's like this :(

I'm getting the same problem Kataphraktarii. It's been going on for a least a week... maybe longer...

RESTRICTIONBREAKER said:
Heyyy... turns out that the prefix "user:" needs to be added before the profile name. Since it autofills in your name without the prefix when you click "search your own comments", it doesn't work and has become confusing. It'd be nice to have the site autofill that prefix so that the comment search works!

"Search your own comments"? Where is that link? I just click the number after the text "Comments:" in my profile, seems to work okay.

Do you perhaps mean 'Search for yourself in comments'? AFAIK that is intended to behave that way -- find -mentions of your username- in any comments, rather than finding comments -you made-.

NotMeNotYou said:
Our host is still switching switches as if they believe it's needed to only run them for like 24h tops.

We do hope they stop doing that soon.

Wow, so to me sounds like your guys' host is run by a bunch of incompetent douche canoes.

Does anyone plan on knocking some sense into them?

This morning I am now starting to get frequent Error 522 that sometimes redirects me to the cloudflare always online, as well as pages that take a very long time to load that say 'waiting for available socket'. Any idea what any of that means? Sigh, it's been days since the servers started getting all wonky. It's really worrying for me.

From what I've been told we're currently working on limited hardware at the moment while we work with our host to fix some related timeout issues (some users have been seeing 522 errors on every site load for the last couple months).

I don't have a timeframe to give for when this will be all fixed, but we're aware of the issues and are working for a fix.

Kataphraktarii said:
This morning I am now starting to get frequent Error 522 that sometimes redirects me to the cloudflare always online, as well as pages that take a very long time to load that say 'waiting for available socket'. Any idea what any of that means? Sigh, it's been days since the servers started getting all wonky. It's really worrying for me.

From what I understand, the "waiting for available socket" message is actually a Chrome bug, and isn't related to the current site issues. The bug is usually triggered when there are a lot of resources on the page and Chrome fails to free up the connection pool.

A temporary fix is to go to chrome://net-internals/#sockets and hit "Flush sockets". Just note that this is only a temporary fix and it will likely happen again.