Topic: [Bug/Fixed] VERY long blacklists have ceased working, no posts being blocked

Posted under Site Bug Reports & Feature Requests

This topic has been locked.

Bug Overview:
Although I'm sat at 3850 characters in my blacklist, and there were no problems previously, all posts are now visible, including ones blacklisted. The blacklisted tab on the left of the Index, that shows what images are blacklisted, is gone, as if my blacklist is empty.

What part(s) of the site page(s) are affected?:
https://e621.net/index

What is the expected behavior:
Posts I have blacklisted shouldn't be visible.

What actual behavior is given instead:
Visibility of 100% of posts regardless of Blacklisting.

Time of Incident:
8/5/16, around Noon.

Can you reproduce the bug every time?:
I can't not reproduce it. It's still happening.

What steps did you take to replicate this bug:
Don't know how to replicate, but cause could be related to Clean and Sort option under blacklist box.

Errors or other messages returned (if any):
N/A

Updated by KiraNoot

I don't know what your problem is but basic blacklisting is working for me.

Maybe you could try to isolate the problem by removing line by line from your blacklist. I'm sure the devs would appreciate that as well in case there really is some bug.

Updated by anonymous

Please try clearing your browser cache. You may have an outdated copy of the blacklist code, which can cause it to fail when you update your blacklist after the update on 2016-08-04

Updated by anonymous

KiraNoot said:
Please try clearing your browser cache. You may have an outdated copy of the blacklist code, which can cause it to fail when you update your blacklist after the update on 2016-08-04

Tried afterwords with the same list. No positive result, and sadly rest in peace TiTS progress. Oh well. I think it is what Chessax said, and is something on my massive 3850 character list. Hopefully I'm patient enough to try and sort through it.
I doubt it's small enough to post here for others to try or something, or if that is allowed.

Updated by anonymous

NineyWhoovesButt said:
Tried afterwords with the same list. No positive result, and sadly rest in peace TiTS progress. Oh well. I think it is what Chessax said, and is something on my massive 3850 character list. Hopefully I'm patient enough to try and sort through it.
I doubt it's small enough to post here for others to try or something, or if that is allowed.

Another tip would be to try a different browser to make sure it's still not a cache issue or corrupt cookie, etc.

Anyway, just dump it on http://pastebin.com or something and post a link if you're afraid of spamming the thread. Although for debugging purposes it might be interesting to have it inside the thread, but not sure what would be considered "too much text for posting", at least put it in a \[section]blacklist\[/section] block in that case.

Updated by anonymous

Chessax said:
Another tip would be to try a different browser to make sure it's still not a cache issue or corrupt cookie, etc.

Anyway, just dump it on http://pastebin.com or something and post a link if you're afraid of spamming the thread. Although for debugging purposes it might be interesting to have it inside the thread, but not sure what would be considered "too much text for posting", at least put it in a \[section]blacklist\[/section] block in that case.

http://pastebin.com/t9b3EZkV Here is the list during the issue. I currently use a substitute, just having the barebones things I dislike seeing the most in there and it works. So It most likely is some sort of error with either the browser, or a certain tag.

Updated by anonymous

NineyWhoovesButt said:
http://pastebin.com/t9b3EZkV Here is the list during the issue. I currently use a substitute, just having the barebones things I dislike seeing the most in there and it works. So It most likely is some sort of error with either the browser, or a certain tag.

I tried your blacklist, and it doesn't work for me either. E.g. simply cutting at the following works (cut at terrorism):

...
terror

While the following doesn't:

...
terrori

However blacklisting terrorism works just fine. Tried various other things as well but couldn't figure anything out, so probably just have to wait for a dev to investigate.

The strange thing is that there is, from what I can tell, a valid Set-Cookie header in the response, but the cookie is not stored by the browser, disabling JavaScript made no difference so that's not the culprit.

Updated by anonymous

After some diagnostics, it appears that the total size of the cookie is coming into play here and preventing the cookie from being set. The size of the cookie is being inflated above what is being shown because of required URL encoding that is taking place, pushing it beyond the size that can be set in a browser.

I made an adjustment that should lower the size of the cookie slightly and allow it to work again, but this really needs to be moved over to local storage long run. I'll put in some work on doing that instead of messing around with cookies all day trying to keep them small and working.

I'll see about deploying the workaround soonish.

Updated by anonymous

Pushed a deploy forwards with this one. The blacklist should work again, as is, although it is dangerously close to the limit.

Updated by anonymous

KiraNoot said:
I made an adjustment that should lower the size of the cookie slightly and allow it to work again, but this really needs to be moved over to local storage long run. I'll put in some work on doing that instead of messing around with cookies all day trying to keep them small and working.

Facepalm moment over here for me for not thinking... But that's probably all good, but still curious about forum #203072

Updated by anonymous

KiraNoot said:
Pushed a deploy forwards with this one. The blacklist should work again, as is, although it is dangerously close to the limit.

Thanks. It was always close to the limit given how picky I was. Still, I think this was a good thing to get resolved. I think it was being shortened and inflated so much due to the Clean And sort, but that's just a hypothesis.

Updated by anonymous

Preview of things to come:
Out of the interest of making blacklists more reliable and in general less annoying to work with internally, I'm moving them out of cookies for everything but anonymous blacklists.

Anonymous blacklists are still in cookies because private browsing modes on mobile browsers like to throw exceptions on setting localStorage items, and I didn't feel the issue was big enough to warrant writing a whole library to migrate things around between cookies, localStorage and test for "broken" mobile browsers in private browsing modes.

I also avoided caching it locally because it made for too many edge cases related to fetching the blacklist, and overall made the whole feature very complicated and error prone. I'm sure we can front the ~1.5kB gzipped now that it isn't being tossed back and forth twice through a cookie.

Updated by anonymous

  • 1