Topic: An update on the recent site issues, shiny new server upgrades in the works

Posted under General

Hey everybody, I've got a bit of an update on the problems we've been experiencing with performance and I hope to give a bit of an insight the various 5xx errors as well.

Like many websites, e6 doesn’t run off of a single computer, but actually runs on several different computers. Some of the machines can’t handle as much of the load, so we give them less work to compensate. If this balance isn’t exactly right during peak times, you end up maxing out one of the machines early and it gets unresponsive and starts throws up weird errors. If you ever see the site not work on one page load and work perfectly fine on the next, you’re essentially accessing a different machine.

It’s like trying to manage a traffic jam with a mix of fast, wide highways and slow, narrow alleys (that are also full of potholes). The current plan is to retire the alleys and replace them with several, new identical highways, which will both be able to handle more traffic, and greatly simplify the load balancing (if they are all equally powerful, you can take a lot of the guesswork out of how much work each machine gets).

As of right now I don’t have an exact timeframe to give you for all this, but I’d expect a more formal announcement about the upgrades within the next couple weeks or so. 

Updated by Queen Tyr'ahnee

Jesus, just how many perks does being an Admin give you? It wasn't that long ago when you were recently made Admin, Parasprite.

Great news to hear BTW.

Updated by anonymous

GameManiac said:
Jesus, just how many perks does being an Admin give you? It wasn't that long ago when you were recently made Admin, Parasprite.

We all sit in a skype group together with Varka and Dari, who is our specialized IT guy, so we all have that information, but para decided to put it into nicer words for you guys. :V

Also, if absolutely everything works out the better servers will be up and online sometime next week, but depending on some factors we may need to push that back a bit, although definitely not as long as FA who lets a shiny new server sit for over a year...

Updated by anonymous

GameManiac said:
Jesus, just how many perks does being an Admin give you? It wasn't that long ago when you were recently made Admin, Parasprite.

7 + a key to get access to the pool on the top floor.

Updated by anonymous

NotMeNotYou said:
We all sit in a skype group together with Varka and Dari, who is our specialized IT guy, so we all have that information, but para decided to put it into nicer words for you guys. :V

Also, if absolutely everything works out the better servers will be up and online sometime next week, but depending on some factors we may need to push that back a bit, although definitely not as long as FA who lets a shiny new server sit for over a year...

*Sigh* The life of an Admin sounds pleasing.

parasprite said:
7 + a key to get access to the pool on the top floor.

Not surprising. You can't go any higher than Admin.

Updated by anonymous

GameManiac said:

Not surprising. You can't go any higher than Admin.

Except site owner. :3

Updated by anonymous

parasprite said:
7 + a key to get access to the pool on the top floor.

Can you invite people to the pool and throw parties up there?

Good to hear that problems are being worked on!

Updated by anonymous

Just_Another_Dragon said:
Can you invite people to the pool and throw parties up there?

That's what sets are for. :V

Updated by anonymous

GameManiac said:
Not surprising. You can't go any higher than Admin.

Not until Spight decides on adding that Developer user rank he was talking about :3

Updated by anonymous

furballs_dc said:
Except site owner. :3

Which is why Varka is hovering above the pool in his private heli.

Also, nice to hear something's being done about this, thank you guys!

Updated by anonymous

TonyLemur said:
Not until Spight decides on adding that Developer user rank he was talking about :3

Artist rank, artist rank, artist rank.

Updated by anonymous

NotMeNotYou said:
Artist rank, artist rank, artist rank.

Where would they be on the totem pole?

Updated by anonymous

Super_Hornet said:
Where would they be on the totem pole?

Same as user but auto approval for their stuff.

Updated by anonymous

Will this make the site load faster? E6 is the slowest loading site for me compared to other places I visit regularly.

Updated by anonymous

Dogenzaka said:
Will this make the site load faster? E6 is the slowest loading site for me compared to other places I visit regularly.

On your end a better answer might be "It'll stay fast more consistently during peak hours, and not throw up 502/524 errors for no apparent reason", but it'll probably feel faster as well.

Disclaimer: Everything in here is silly (but informative)

A slightly longer answer is that the older machines are both worn out and slower. They tend to get stressed under heavy load and will stop responding until it gets enough paperwork done, but the paperwork keeps getting piled onto it so it looks like it froze up.

Your computer, however, doesn't know that the reason the machine isn't responding is because it's busy doing something else, all it knows is that it isn't responding (Why should your computer care? It should respond, right?). So your computer will continue to pick up the phone and tell Cloudflare (the middleman in all this) to call the servers with the message Hey, you there e6? I can't find you..... Even after they don't get a response, your computer will keep telling Cloudflare to call e6 with I need a response!...HELLLOOOOO....RESPOND NAO PLS. This goes on while the machine (the one that froze up) is having a panic attack and is trying to stuff crumpled, half-finished tagging/deletions paperwork into drawers hoping nobody will notice, and is to far gone to realize that the phone is ringing.

Cloudflare will start out by patiently telling your computer Give it a moment...they were there a moment ago but this doesn't last long as Cloudflare is kind of irritable and will eventually get fed up with your annoying computer and yell I can't find it, it's probably not even there anymore, I GIVE UP. GG MATE, ERROR 502. At this point if your computer keeps trying to tell Cloudflare to call e6, it'll eventually respond with The computer probably died or something lol. But whatever dude, I'll try again in a couple minutes. Stop spamming that F5 key, mmkay? ERROR 524.

So we hire some new machines that are better at multitasking (they'll be able to talk on the phone and do paperwork at the same time), can do more a lot more work under heavy load, and also look way better in a suit jacket than the old machines ever did. All of this will help to stop this whole process from happening in the first place.

TLDR version:

Yeah, it should be faster.

Updated by anonymous

parasprite said:
On your end a better answer might be "It'll stay fast more consistently during peak hours, and not throw up 502/524 errors for no apparent reason", but it'll probably feel faster as well.

Disclaimer: Everything in here is silly (but informative)

A slightly longer answer is that the older machines are both worn out and slower. They tend to get stressed under heavy load and will stop responding until it gets enough paperwork done, but the paperwork keeps getting piled onto it so it looks like it froze up.

Your computer, however, doesn't know that the reason the machine isn't responding is because it's busy doing something else, all it knows is that it isn't responding (Why should your computer care? It should respond, right?). So your computer will continue to pick up the phone and tell Cloudflare (the middleman in all this) to call the servers with the message Hey, you there e6? I can't find you..... Even after they don't get a response, your computer will keep telling Cloudflare to call e6 with I need a response!...HELLLOOOOO....RESPOND NAO PLS. This goes on while the machine (the one that froze up) is having a panic attack and is trying to stuff crumpled, half-finished tagging/deletions paperwork into drawers hoping nobody will notice, and is to far gone to realize that the phone is ringing.

Cloudflare will start out by patiently telling your computer Give it a moment...they were there a moment ago but this doesn't last long as Cloudflare is kind of irritable and will eventually get fed up with your annoying computer and yell I can't find it, it's probably not even there anymore, I GIVE UP. GG MATE, ERROR 502. At this point if your computer keeps trying to tell Cloudflare to call e6, it'll eventually respond with The computer probably died or something lol. But whatever dude, I'll try again in a couple minutes. Stop spamming that F5 key, mmkay? ERROR 524.

So we hire some new machines that are better at multitasking (they'll be able to talk on the phone and do paperwork at the same time), can do more a lot more work under heavy load, and also look way better in a suit jacket than the old machines ever did. All of this will help to stop this whole process from happening in the first place.

TLDR version:

Yeah, it should be faster.

We don't have bandwidth issues, almost all perceived slowness (index page taking a couple seconds to load) is related to our server not being fast enough to sift through our database, which slows done page generation.

Right now it takes almost 20 seconds for me to open the index page (since it's peak time in the USA) and during "normal" times it's somewhere between 3-4 seconds. The new servers should make bring these times a lot lower during peak hours.

Updated by anonymous

  • 1