Topic: RE621 [v.1.5.68] Feature-packed toolkit and mass downloader for e621

Posted under e621 Tools and Applications

bitWolfy

Former Staff
Patch 1.4.10

Fixes
- Applied hover zoom to thumbnails in the comments index
- Added some extraneous information to hover zoom display
- Dynamically remove thumbnail hover text to stop it from interfering with hover zoom
- Restored the wiki page link in the tag list to its original position

Here are the suggestions I've built up that I mentioned I'd post. Thanks for your work again, bitWolfy. Your script has been amazing!

  • 'X' button next to the set nagivator(s) on /posts/ pages/ (Where it says what sets the post you are viewing is in.)
  • A way to clear a hotkey or set a hotkey to not have a key assigned.
  • A 'clear-all notifications' button.
  • Button/Hotkey when viewing a set to jump to a random image from the current set.
  • Custom Flag support for "Popular Posts" page. (https://e621.net/explore/posts/popular)

Then these last two are big ones but are ones that'd be a huge deal.

  • Toggle-able "Image-Only Mode". Basically, while active, any e6 post page redirects to the full-image/source of that post. (Same effect as clicking the "Fullscreen" button that this script has). The check for if this is active should ideally be at the start of the script so that upon page loading, if active, it doesnt have to run the rest of the script.
  • Lastly, some batch processing options for sets/pools/favorites would be insanely useful. Main things that come to mind are as follows:
    • Check duplicate posts within set A and B, remove duplicates from A or B.
    • Check set A for posts with one of their children/parents/alternate-versions in the same set as them. (Example, Set A has post A and post B, among other posts. Post A is a parent of post B. This would find that post B is a alt-version/parent of post A.) Then it could display all it found somehow? This is alot, im aware though.
    • Various other conditionals and effects to combine. "For posts that X, do Y to those posts", etc.

bitWolfy

Former Staff

frans7 said:
Here are the suggestions I've built up that I mentioned I'd post. Thanks for your work again, bitWolfy. Your script has been amazing!

Thanks for the kind words. Your feedback is very much welcome, as always.
I'll work through your suggestions one at a time.

frans7 said: 'X' button next to the set nagivator(s) on /posts/ pages/ (Where it says what sets the post you are viewing is in.)

I'm not sure what you intend for this to do. Remove the post from set? Remove the set navbar? Something else?

frans7 said: A way to clear a hotkey or set a hotkey to not have a key assigned.

That is already a feature. Press Escape when assigning a key, and the binding will be released.

frans7 said: A 'clear-all notifications' button.

This is also already a feature, although probably not a very well-marked one.
Go no Notifications -> Info tab. At the very bottom, there is a button that says "clear cache" that should remove all notifications.

frans7 said: Button/Hotkey when viewing a set to jump to a random image from the current set.

Simple enough, I'll see what I can do.

frans7 said: Custom Flag support for "Popular Posts" page. (https://e621.net/explore/posts/popular)

With the overhaul of the thumbnail system for the search pages, support for CustomFlags on other pages was temporarily dropped.
I do plan on brinding it back - not just for the popular page, but on profile and comment pages as well. The fact that e621 outputs thumbnails in no less than three different formats makes this a little bit more difficult, though.

frans7 said:
Toggle-able "Image-Only Mode". Basically, while active, any e6 post page redirects to the full-image/source of that post. (Same effect as clicking the "Fullscreen" button that this script has). The check for if this is active should ideally be at the start of the script so that upon page loading, if active, it doesnt have to run the rest of the script.

The page simply cannot be redirected immediately.
The script begins execution before any document structure has loaded. This means that it can patch the DOM as it is being created, which should make the layout jump around a lot less.
Unfortunately, it also means that the only information about the post that the script has is the post number. To redirect to the full-sized image page, it either needs the URL of said image, or its md5 hash and extension. Access to both requires either an API call, or for the image container (which contains said data) to load. Neither of these require too much time, but they are not instantaneous either.
In other words, I will implement this, but the page will most likely have to go through some loading before it gets redirected.

I also have no idea why would you want such a mode.
But that's neither here nor there.

frans7 said:
Lastly, some batch processing options for sets/pools/favorites would be insanely useful. Main things that come to mind are as follows:
Check duplicate posts within set A and B, remove duplicates from A or B.
Check set A for posts with one of their children/parents/alternate-versions in the same set as them. (Example, Set A has post A and post B, among other posts. Post A is a parent of post B. This would find that post B is a alt-version/parent of post A.) Then it could display all it found somehow? This is alot, im aware though.
Various other conditionals and effects to combine. "For posts that X, do Y to those posts", etc.

No promises on this one.
I will investigate the possibilities here, but making as customizable as you want it to be would require a considerable amount of work.

Given that the recommended posts feature determines tags based off of your favorites, it should probably be possible to include "-fav:Username" in the query without typing it in manually.

bitWolfy

Former Staff

lafcadio said:
Given that the recommended posts feature determines tags based off of your favorites, it should probably be possible to include "-fav:Username" in the query without typing it in manually.

That should probably be the default, yes.
Will fix.

bitwolfy said:
I'm not sure what you intend for this to do. Remove the post from set? Remove the set navbar? Something else?

My bad. I seem to have forgot to elaborate on that one. I mean for the 'X' button to remove the post from the set that the 'X' button corrasponds to. (If the post is in multiple sets, then there are multiple set navigators. Therefore, multiple 'X' buttons)

That is already a feature. Press Escape when assigning a key, and the binding will be released.

I see. I didnt think to try Escape. Thanks for clarifying.

This is also already a feature, although probably not a very well-marked one.
Go no Notifications -> Info tab. At the very bottom, there is a button that says "clear cache" that should remove all notifications.

Oh okay. Yeah I definitely wouldnt have realized that on my own. That works but a button for it separately would be nice either way.

With the overhaul of the thumbnail system for the search pages, support for CustomFlags on other pages was temporarily dropped.
I do plan on brinding it back - not just for the popular page, but on profile and comment pages as well. The fact that e621 outputs thumbnails in no less than three different formats makes this a little bit more difficult, though.

Sounds good. Looking forward to it hopefully making it in!

The page simply cannot be redirected immediately.
The script begins execution before any document structure has loaded. This means that it can patch the DOM as it is being created, which should make the layout jump around a lot less.
Unfortunately, it also means that the only information about the post that the script has is the post number. To redirect to the full-sized image page, it either needs the URL of said image, or its md5 hash and extension. Access to both requires either an API call, or for the image container (which contains said data) to load. Neither of these require too much time, but they are not instantaneous either.
In other words, I will implement this, but the page will most likely have to go through some loading before it gets redirected.

I also have no idea why would you want such a mode.
But that's neither here nor there.

"Instantly" was perhaps a bit of a strong word to use on my part. I understand what you mean on how it wouldnt be instantaneous though. As for it the script logic, I dont do much in this area so I wasnt aware. I see. My general point was along the lines of; Since redirecting to the full-screen source content doesnt need the rest of the script running first, that it may be wise to handle this feature's check and execution ASAP. As for why I'd want it, basically if I open a bunch of posts at the moment, I end up just clicking 'Fullscreen' on each tab. So this basically automates that. But you say you intend to add this? If so, I'm very gratefull for that. Honestly, I wasnt sure if this one would stick since it's a bit more fringe.

No promises on this one.
I will investigate the possibilities here, but making as customizable as you want it to be would require a considerable amount of work.

No worries on this one. It'd be immensely useful like I said, but I understand that this one is a huge undertaking comparatively. This one though, unlike the suggestion before it, Im sure a fair amount of people would benefit from. Either way, cant complain!

Thanks for the reply and the work as always.

Another suggestion here, I think it'd be nice to have enlarged thumbnails in the subscription tabs as well.

Bulk downloading from search results doesn't appear to be working anymore - entering Select mode doesn't suppress default click behaviour so you just go to the image page. (The required classes/etc don't appear to be added.)

bitWolfy

Former Staff

krabitz said:
Another suggestion here, I think it'd be nice to have enlarged thumbnails in the subscription tabs as well.

I'll see what I can do.

ingot said:
Bulk downloading from search results doesn't appear to be working anymore - entering Select mode doesn't suppress default click behaviour so you just go to the image page. (The required classes/etc don't appear to be added.)

Yet another consequence of the performance tweaks in 1.4.6.
Will fix in a patch later today.

Updated

bitwolfy said:
Yet another consequence of the performance tweaks in 1.4.6.
Will fix in a patch later today.

Sweet :)

bitWolfy

Former Staff
Patch 1.4.11

Fixes
- Moved subscription cache out of settings and into local storage
- Fixed the crash when searching for order:random
- Fixed MassDownloader not initializing properly

Subscriptions doesn't seem to be working for me. It keeps saying "Initializing..." but won't load anything.

bitWolfy

Former Staff

itskilroy said:
Subscriptions doesn't seem to be working for me. It keeps saying "Initializing..." but won't load anything.

Hm. Seems to be working fine on my side.
It would only get stuck as early as "Initializing" if it's failing to read your subscription list.
Are there any errors in the console?

bitWolfy

Former Staff
Patch 1.4.12

Changes
- PostSuggester: Excluded already favorited posts from the suggestions
- PostSuggester: Fetch tags from the last 500 favorited posts, instead of last uploaded ones
- HoverZoom: Added an option to display the upload time as an absolute value, instead of a relative one
- CustomFlagger: Moved the flags above the tag field on the post edit page
- BetterSearch: Added two new search modes. "Fullscreen" opens the clicked post's image in fullscreen mode. "Blacklist" blacklists it.

Fixes
- SubscriptionsManager: Fixed a crash caused by empty subscriptions cache

bitWolfy

Former Staff
Patch 1.4.13

Changes
- Added a "random post" to set cover pages
- Added a hotkey to open post history
- Added custom flags to thumbnails on non-search pages
- Made thumbnail ribbons slightly larger
- Fixed a typo

Is there any way for the script to run on un-viewed tabs? For example, when I open a new tab, the script doesnt seem to take effect on that tab until I view the tab. I've made sure it isnt just slow by waiting a minute then viewing it. Even then, I can see the visual changes that the script makes take effect.

Im not sure if it's something you can do, honestly. No idea. It could be on the Userscript Manager's end. I posted an issue about it a while back (Issue #205) and back then you mentioned it was addressed in version 1.3.14. It still seems to be happening though.

Im curious if anyone else has this happen also, to make sure it isnt just me.

bitWolfy

Former Staff

frans7 said:
Is there any way for the script to run on un-viewed tabs? For example, when I open a new tab, the script doesnt seem to take effect on that tab until I view the tab. I've made sure it isnt just slow by waiting a minute then viewing it. Even then, I can see the visual changes that the script makes take effect.

Im not sure if it's something you can do, honestly. No idea. It could be on the Userscript Manager's end. I posted an issue about it a while back (Issue #205) and back then you mentioned it was addressed in version 1.3.14. It still seems to be happening though.

Im curious if anyone else has this happen also, to make sure it isnt just me.

It was a deliberate change, made to combat your issue ( #228 ), no less.
While it did not resolve that problem, it is generally not a bad idea.

The problem here is API access.
There are several re621 modules that can call on the e621 API on page load. Subscription Manager is one of them, for example. This site imposes limits on how often the API can be accessed, which is at most twice per second. These requests can be queued within the script to adhere to this requirement, but that only applies within one tab. If someone opens several dozen tabs at once – like what you have been doing – this bombards the servers with requests and can cause an error due to rate limiting.

It should be possible to only have modules that actually require API access wait for the window to come into focus. I'll see what I can do.

bitWolfy

Former Staff
Patch 1.4.14

Fixes
- Only load modules that require API calls in focused windows to avoid rate limiting
- Set the focus properly on the Formatting Helper textarea when switching between editing and preview modes
- Updated dependencies and cleaned up the code

bitwolfy said:
It was a deliberate change, made to combat your issue ( #228 ), no less.
While it did not resolve that problem, it is generally not a bad idea.

The problem here is API access.
There are several re621 modules that can call on the e621 API on page load. Subscription Manager is one of them, for example. This site imposes limits on how often the API can be accessed, which is at most twice per second. These requests can be queued within the script to adhere to this requirement, but that only applies within one tab. If someone opens several dozen tabs at once – like what you have been doing – this bombards the servers with requests and can cause an error due to rate limiting.

It should be possible to only have modules that actually require API access wait for the window to come into focus. I'll see what I can do.

Thanks for the explanation. So you believe it's simply/just API-related?

bitWolfy

Former Staff

frans7 said: Thanks for the explanation. So you believe it's simply/just API-related?

Your particular issue?
I think that it's somehow related to the rate-limiting that the site imposes, yes. It's not just API access, it's present on all pages.

Regardless, I have added a bit of logic in patch 1.4.14 to only have modules that require API access wait for the page to come into focus. The rest of the script should load normally.

bitwolfy said:
Your particular issue?
I think that it's somehow related to the rate-limiting that the site imposes, yes. It's not just API access, it's present on all pages.

Regardless, I have added a bit of logic in patch 1.4.14 to only have modules that require API access wait for the page to come into focus. The rest of the script should load normally.

Alright, much appreciated!

Hey, so I just downloaded the most recent patch (1.4.14) and now the script seems to be disabled. I've tried updating it again and turning tampermonkey off and on, and neither seems to have worked. Does anybody have any fixes?

bitWolfy

Former Staff

binagon said:
Hey, so I just downloaded the most recent patch (1.4.14) and now the script seems to be disabled. I've tried updating it again and turning tampermonkey off and on, and neither seems to have worked. Does anybody have any fixes?

Seems to be working on my side.
Are there any errors in the console?

bitwolfy said:
Seems to be working on my side.
Are there any errors in the console?

I'm getting 2:

Failed to load resource:
net::ERR_BLOCKED_BY_CLIENT

and

Uncaught (in promise) TypeError:
Cannot read property 'replace' of undefined
at Function.run (userscript.html?name=re621%20-%20e621%20Reimagined.user.js&id=37ff37ed-f501-411a-a2e5-3078b57eb33c:113)
at async eval (userscript.html?name=re621%20-%20e621%20Reimagined.user.js&id=37ff37ed-f501-411a-a2e5-3078b57eb33c:113)

bitWolfy

Former Staff

binagon said:
I'm getting 2:

Failed to load resource:
net::ERR_BLOCKED_BY_CLIENT

and

Uncaught (in promise) TypeError:
Cannot read property 'replace' of undefined
at Function.run (userscript.html?name=re621%20-%20e621%20Reimagined.user.js&id=37ff37ed-f501-411a-a2e5-3078b57eb33c:113)
at async eval (userscript.html?name=re621%20-%20e621%20Reimagined.user.js&id=37ff37ed-f501-411a-a2e5-3078b57eb33c:113)

Thank you.

The first one is not an issue with my script. It's not an issue at all - it's most likely your ad blocker / anti-tracker / whatever doing what it must.
The second one is re621 hard crashing. Is that the entire message, or is there a longer tracelog that appears when you click on the error?

bitWolfy

Former Staff
Patch 1.4.15

Changes
- Added three new blacklist formats: width, height , filesize. Range values are supported.
- Spell-checked the entire script. This may or may not have broken something.
- Added some extra error handling for the config patcher.

bitwolfy said:
Thank you.

The first one is not an issue with my script. It's not an issue at all - it's most likely your ad blocker / anti-tracker / whatever doing what it must.
The second one is re621 hard crashing. Is that the entire message, or is there a longer tracelog that appears when you click on the error?

OK well I updated it again just now and it seems to work again, so all appears to be well. But in case you were wondering, that was the entire message, clicking on it didn't change anything.

bitWolfy

Former Staff

binagon said:
OK well I updated it again just now and it seems to work again, so all appears to be well. But in case you were wondering, that was the entire message, clicking on it didn't change anything.

I addressed a potential cause of the crash in 1.4.15.
A bit of a shot in the dark, but I am glad that it has worked.

bitwolfy said:
I addressed a potential cause of the crash in 1.4.15.
A bit of a shot in the dark, but I am glad that it has worked.

Righto, nice work my dude. And thanks for the help!

bitWolfy

Former Staff

binagon said:
Righto, nice work my dude. And thanks for the help!

No problem. Let me know if you run into any other issues.

bitWolfy

Former Staff
Patch 1.4.16

Changes
- Replaced the hotkey library. Again.
- Prevented hotkey conflicts. For example, a hotkey bound to Ctrl+E should not trigger a hotkey bound to E.
- Re-added "Submit Form" hotkey. Check your settings, the old binding might have been removed.
- Reworked subscription update rendering. No visual changes, but should improve performance.
- Added 'favcount' blacklist filtering option. Range values are supported.

bitWolfy

Former Staff
Patch 1.4.17

Quick hotfix release that addresses a couple of hotkey-related issues introduced in 1.4.16

Fixes
- Resolved an issue with shift-modified number keys being treated as invalid
- Fixed symbolic hotkeys not working correctly in some cases

Hey bitWolfy, me again~

So I tried it out and while I like the ability to put favcount requirements in the blacklist, it helped me realize what I'm actually trying to do, which turns out I can do on the regular site (although it might take some work, which I'll explain in a moment). It's mostly good, my only gripes are things that could probably be gotten over by using it more, but I'll share them anyway:

- I recommend the default being absolutely zero differences in appearance from the normal site, that way it doesn't scare off folks who hate unfamiliarity or don't have the patience to tweak it back to how they're used to seeing it (or just learn to live with it). So that means exact fonts, font sizes, no difference in thumbnail quality, none of your extra cool stuff on by default, etc. Until of course the user chooses to change things themselves, which is much easier to get used to.

- On the normal site I have some custom CSS (which someone else helped me make, because like I said I don't understand programming, haha) that does some things like hide the blacklisted tags list (above or under the search bar, I forget), hide the 'Random' option, and other things that tempt a lot of time-wasting. When I installed your tool set the CSS no longer worked, so I guess find a way to make custom CSS still work or have options like 'blackhole this and that so you never see it,' etc.

- Speaking of which, one cool feature you could do (as an option) is 'blackhole everything but the images,' essentially turning the site into a tumblr-esc scrolling site, with nothing but the images and your tool set's settings up in the corner. I mess around with tags too much to use that idea myself, but if I ever 'finally' get my blacklist perfect, I might use something like that.

- I don't know what it was, but one of my extensions was constantly reminding me that it blocked a page pop-up or overlay. When I allowed it (just one time) it opened up another tab which was just the e612 front page with the mascot and such. I don't know if that was my extension doing that or the tool set, but just thought I'd let ya know.

Other than that stuff, great work so far. You're making all kinds of updates rather quickly so I'm sure it won't be that long before I decide to give it another try :]

Until then I'm gonna try working on my silly little endeavor, which I had a question for you about. One tool that I could get a lot of use out of is some way to know about how many days it'll take for a post to go from one page to the next in line--based on the blacklist and the specifics you've put in the search bar. The ideal would be for me to only see what I would click, and my restrictions making so the page I frequent moves slow enough that it takes around a full week to go from the top of Page 1 (brand new) to the top of Page 2, by which time I've already seen it so there's no catching up to do. It's been annoying, but I enjoy making things more efficient and cutting back on wasting time.

Would you have any ideas for tools that can make that easier? Right now all I have is literally watching a post travel from Page 1 to Page 2 (checking it daily), and if it's too fast I have to make the blacklist and search requirements more strict, etc. Is there any way to predict how fast the site moves as far as upload frequency, for example?

Thanks if you get a chance to comment. You've been great with that but take your time if you have to :]

Updated

bitWolfy

Former Staff

human-shaped said:
Hey bitWolfy, me again~

So I tried it out and while I like the ability to put favcount requirements in the blacklist, it helped me realize what I'm actually trying to do, which turns out I can do on the regular site (although it might take some work, which I'll explain in a moment). It's mostly good, my only gripes are things that could probably be gotten over by using it more, but I'll share them anyway:

Howdy.
Thanks for trying out the userscript.

human-shaped said:
- I recommend the default being absolutely zero differences in appearance from the normal site, that way it doesn't scare off folks who hate unfamiliarity or don't have the patience to tweak it back to how they're used to seeing it (or just learn to live with it). So that means exact fonts, font sizes, no difference in thumbnail quality, none of your extra cool stuff on by default, etc. Until of course the user chooses to change things themselves, which is much easier to get used to.

I'll change the default thumbnail settings to (roughly) match what the site looks like normally in the next patch. It will not be exactly the same, since I have effectively torn out and replaced the default system, but it should be pretty similar.
Some of the "cool stuff" I don't want to disable by default, since I am absolutely convinced that people will never find some of those otherwise.

human-shaped said:
- On the normal site I have some custom CSS (which someone else helped me make, because like I said I don't understand programming, haha) that does some things like hide the blacklisted tags list (above or under the search bar, I forget), hide the 'Random' option, and other things that tempt a lot of time-wasting. When I installed your tool set the CSS no longer worked, so I guess find a way to make custom CSS still work or have options like 'blackhole this and that so you never see it,' etc.

The blacklist I also tore out and replaced in order to work with InfiniteScroll and to overhaul some of the filtering options.
I'll see if I can add the default site's classes to the new structure that I made. Oh, and I'll add an option to remove the blacklist section entirely in the next patch, that should be pretty easy to do.

Not sure what's going on with the "random" option. Might be a bug - I don't think that I'm doing anything to it at all. I'll investigate.

Edit: I've taken a quick look, not sure what's going on. We are talking about the "random" link that's in the "Related" section, right?
RE621 does not touch that link, but it does add a "Recommended" button right underneath it. I wonder if it's interfering somehow with your custom CSS.
Could you link me to where you got those custom styles? I would like to take a look at those.

human-shaped said:
- Speaking of which, one cool feature you could do (as an option) is 'blackhole everything but the images,' essentially turning the site into a tumblr-esc scrolling site, with nothing but the images and your tool set's settings up in the corner. I mess around with tags too much to use that idea myself, but if I ever 'finally' get my blacklist perfect, I might use something like that.

I was going to add an option to effectively "collapse" the sidebar at some point, I just never got around to it.
I'm kind of busy with my other project at the moment, so I can't say when it will be implemented. But it's on the todo list.

human-shaped said:
- I don't know what it was, but one of my extensions was constantly reminding me that it blocked a page pop-up or overlay. When I allowed it (just one time) it opened up another tab which was just the e612 front page with the mascot and such. I don't know if that was my extension doing that or the tool set, but just thought I'd let ya know.

Not sure what's going on here. RE621 should not have any popups, but it does have a couple of floating overlay windows for settings and notifications.
What browser are you using, and which extension was giving you those warnings?

human-shaped said:
Until then I'm gonna try working on my silly little endeavor, which I had a question for you about. One tool that I could get a lot of use out of is some way to know about how many days it'll take for a post to go from one page to the next in line--based on the blacklist and the specifics you've put in the search bar. The ideal would be for me to only see what I would click, and my restrictions making so the page I frequent moves slow enough that it takes around a full week to go from the top of Page 1 (brand new) to the top of Page 2, by which time I've already seen it so there's no catching up to do. It's been annoying, but I enjoy making things more efficient and cutting back on wasting time.

That's actually a relatively easy thing to do. I might add something like this, since it seems like a useful statistic to have.
If I just subtract the upload time of the first post on the page from that of the last post on the page, the result would the number of days (hours / weeks / whatever) it took to fill that page.
Wouldn't work with stuff like order:score, obviously. Actually, any ordering outside of the default one would not function with it.
But still, should be a pretty simple thing to add.

human-shaped said:
Thanks if you get a chance to comment. You've been great with that but take your time if you have to :]

Sure.
I check the forums pretty frequently. Let me know if you have any other issues or suggestions.

Updated

bitWolfy

Former Staff
Patch 1.4.18

Changes
- Added search result statistics: approximate number of posts and update frequency
- Added an option to hide the blacklist filters section in the sidebar

Fixes
- Tweaked default thumbnail settings to match the normal site look
- Prevented repeated actions when holding down a hotkey. Again.
- Fixed subscription updates sometimes not getting marked as new

Thanks for the 'Random Set from Post' feature! Can you get it's hotkey to work on /post pages? Only issue might be if a post belongs to multiple sets I guess

bitWolfy

Former Staff

frans7 said:
Thanks for the 'Random Set from Post' feature! Can you get it's hotkey to work on /post pages? Only issue might be if a post belongs to multiple sets I guess

No problem.
I could probably tie it into the same system as the navigation hotkeys - pick a random post from whichever navbar is selected.

bitwolfy said:
No problem.
I could probably tie it into the same system as the navigation hotkeys - pick a random post from whichever navbar is selected.

That'd be great! Hopefully if there are multiple nav bars it'll work fine

The odd issues I had before aren't happening now, although I do remember clicking 'stop notifying me for 30 days,' so it might come back I'm not sure. I use Chrome and the extension is just called Pop Up Blocker.

Thanks for adding so many new things so quickly. I tested the new statistics thing and it does change based on what's in the search and what you have your images-per-page set to. At first I thought what I'm trying to do would need it to also refer to the blacklist (so it would only focus on the number of visible images and not blacklistted ones), but whether a page hits 250 with images all visible or not, it's still eventually going to send the visible stuff to Page 2, so yeah. I can use this to find the ideal speed! :]

One thing that does confuse me a bit is the estimated number changes as you look through Page 1, 2, 3, etc. Why does it do that? :o

On a different topic, one thing I noticed a moment ago was while the blacklist was keeping everything hidden like on the normal site, while using your userscript if you click on 'next page' and land on a blacklisted post, you see the white box with the black text saying 'blacklisted' for a moment, but then it loads and ends up showing the image. Same happens if you click on the white boxes in the comic's pool section. So whatever's causing it to show the images instead of keep it as the white box, hopefully that can be fixed.

Updated

bitWolfy

Former Staff

human-shaped said:
The odd issues I had before aren't happening now, although I do remember clicking 'stop notifying me for 30 days,' so it might come back I'm not sure. I use Chrome and the extension is just called Pop Up Blocker.

I'll take a look, thanks.

human-shaped said:
Thanks for adding so many new things so quickly. I tested the new statistics thing and it does change based on what's in the search and what you have your images-per-page set to. At first I thought what I'm trying to do would need it to also refer to the blacklist (so it would only focus on the number of visible images and not blacklistted ones), but whether a page hits 250 with images all visible or not, it's still eventually going to send the visible stuff to Page 2, so yeah. I can use this to find the ideal speed! :]

One thing that does confuse me a bit is the estimated number changes as you look through Page 1, 2, 3, etc. Why does it do that? :o

No problem. Those changes are pretty easy to implement.
The update frequency is calculated by finding the difference in the upload times of the first post on the page and the last one. Of course, those would be somewhat different on different pages.
I could make it more precise by fetching several pages of posts and getting an average of those, but that would require more API calls, which is something I would like to avoid, especially for a non-essential feature like this one.

human-shaped said:
On a different topic, one thing I noticed a moment ago was while the blacklist was keeping everything hidden like on the normal site, while using your userscript if you click on 'next page' and land on a blacklisted post, you see the white box with the black text saying 'blacklisted' for a moment, but then it loads and ends up showing the image. Same happens if you click on the white boxes in the comic's pool section. So whatever's causing it to show the images instead of keep it as the white box, hopefully that can be fixed.

Huh. Never noticed that.
The image rescaler must be bypassing the blacklist somehow. I'll fix it in the next patch.

You're welcome, and that's understandable. I can just take the first x amount of pages and come up with an average that way. Shouldn't be too painful doing a little math, haha~

Cool additions! Is there a way to make videos loop by default when in fullscreen mode? And maybe also have an option to mute videos by default.
And this may or may not be realistic, but it'd be great to receive a notification if one of your favorited posts was deleted, I hate whenever you can't find a post, and you don't know if it's because you got the tags wrong or if it was removed.

Updated

bitWolfy

Former Staff

human-shaped said: You're welcome, and that's understandable. I can just take the first x amount of pages and come up with an average that way. Shouldn't be too painful doing a little math, haha~

I'll set it to show the frequency in days when you hover over it. Hope that helps with the math.

krabitz said: Cool additions! Is there a way to make videos loop by default when in fullscreen mode? And maybe also have an option to mute videos by default.

Thanks.
There is a way, but it won't make it into this script. At least, not for a while.
The current script expects at least some basic document structure from the page, and the "fullscreen more" lacks any of it. For the time being, I created a small external userscript that does what you are looking for, you can get it here. It's literally five lines of code, as you can see - I just don't have any way to plug it into the main script right now that's not janky in one way or another.

krabitz said: And this may or may not be realistic, but it'd be great to receive a notification if one of your favorited posts was deleted, I hate whenever you can't find a post, and you don't know if it's because you got the tags wrong or if it was removed.

Not difficult from a technical perspective, just confusing. The way to do it would be to look up fav:krabitz + status:deleted every once in a while, and check if those have been updated recently updated. Unfortunately, there is no way to differentiate a post being deleted from someone updating the tags on said deleted posts, which can happen sometimes. So, you could get repeated deletion notifications, I suppose.

Unfortunately, the bigger problem right now is that I don't have anywhere to put those notifications. At least, not until the updated subscription feed gets revamped, which is what I am working on right now.
So, it might not make it into the script for some time.

bitWolfy

Former Staff
Patch 1.4.19

Fixes
- Prevented image scaling from overriding the active blacklist
- Resolved an issue causing subscriptions to crash when there is no internet connection

bitwolfy said:
I'll set it to show the frequency in days when you hover over it. Hope that helps with the math.

That would be helpful! The other night I found just the right search requirements where both Page 1 and Page 2 say '1 week,' that way I bookmark Page 2 and everything on the first page is given a week to meet my requirements before making it to Page 2 where I frequent the site. An awesome feature so far! Great for folks trying to slow things down and predict speeds in general. The 40 search amount helped a lot as well, haha~

One thing I kept noticing is Page 1 might say 1 week, but if you click through some of the later pages, you'll find it saying things like 4 or 5 days. I guess the uploads on Page 1 and 2 were slower upload days (according to my search requirements, at least) and those previous days had more uploads going on at once, hence those pages saying it only took around 4 or 5 days to fill? I have no idea, haha. If it's that unpredictable, you setting it to show the frequency in days could help come to an 'actual' average than my half-assed average by checking the last ten days or so :]

bitWolfy

Former Staff

human-shaped said:
That would be helpful! The other night I found just the right search requirements where both Page 1 and Page 2 say '1 week,' that way I bookmark Page 2 and everything on the first page is given a week to meet my requirements before making it to Page 2 where I frequent the site. An awesome feature so far! Great for folks trying to slow things down and predict speeds in general. The 40 search amount helped a lot as well, haha~

One thing I kept noticing is Page 1 might say 1 week, but if you click through some of the later pages, you'll find it saying things like 4 or 5 days. I guess the uploads on Page 1 and 2 were slower upload days (according to my search requirements, at least) and those previous days had more uploads going on at once, hence those pages saying it only took around 4 or 5 days to fill? I have no idea, haha. If it's that unpredictable, you setting it to show the frequency in days could help come to an 'actual' average than my half-assed average by checking the last ten days or so :]

There are some problems with that number.
For example, if you search for airfly-pony, the frequency it gives you is 2.2 years. And that's true - the first and the last posts on the page have been uploaded 804 days apart.
However, in reality, I have uploaded 40 out of those 87 posts in the past three days. This means that the actual upload frequency is way off. And while this is an extreme case, and the difference wouldn't normally be as stark, I would still like to do something about it.

I think, this calls for some sort of formula that would describe the change in spacing between uploads. Unfortunately, math was never my strong suit... so I'll have to figure this out somehow.

bitwolfy said:
There are some problems with that number.
For example, if you search for airfly-pony, the frequency it gives you is 2.2 years. And that's true - the first and the last posts on the page have been uploaded 804 days apart.
However, in reality, I have uploaded 40 out of those 87 posts in the past three days. This means that the actual upload frequency is way off. And while this is an extreme case, and the difference wouldn't normally be as stark, I would still like to do something about it.

I think, this calls for some sort of formula that would describe the change in spacing between uploads. Unfortunately, math was never my strong suit... so I'll have to figure this out somehow.

Haha, yeah I noticed that for some tags. Like 'whoa, there must be a reason this particular thing only has one or two pages in the entire site.' And someone actually checking on it for years, assuming your userscript's estimation is 100% accurate. xD

Well like you said before, it's a non-essential feature so by all means take your time. You've convinced me to keep this userscript installed and given me reason to check your thread now and then for updates. Yeah I'm not a math guy either. Who knows though, maybe one of your buddies is a math whiz who can help you with a formula to make the feature much more accurate :]

bitWolfy

Former Staff

Patch 1.4.20

Fixes
- Show the page update frequency in days when hovering over the corresponding text
- Added two variables to title customizer: general and all

bitWolfy

Former Staff

helioeccentric~ said:
For some reason, the script doesn't always start when I access the site using https. If I remove the "s" from a url then it runs fine. Is that an issue with the script or with Tampermonkey? Weirdly, it doesn't happen every time I use https, but changing it to the non-secure one always fixes it.

Strange. I'm using Tampermonkey as well, and I am not experiencing that issue.
Are there any errors in the console when the script fails to load?

bitWolfy

Former Staff
Patch 1.4.21

Changes
- Added a post frequency graph, as well as more detailed post update information on hover
- Created a new settings tab for blacklist settings
- Added an option to exclude your favorited posts from all blacklist filters
- Added an option to exclude your uploaded posts from all blacklist filters
- Implemented a "Tag Changes" section in the post edit form
- Added some validation to source URLs on the upload form

bitWolfy

Former Staff

darthdesolus said:
Hello, bitWolfy,
Really appreciate what you do, I've made some userscripts for e621, but once I found RE621 - I gave up and decided to try your creation.
Blown away. Thank you so much for your work!

I am glad that you like it.

darthdesolus said:
Hello, bitWolfy,
Don't want to be mean or picky, just wanted to point out an (only!) issue I found with InstantFilters.
Let's say I have a posts listing after a search for a tag ( https://e621.net/posts?tags=male+ ).
If I input something in the Filters field and press its corresponding mag.glass button - whole page kinda reloads
and jumps to all posts listing (not searched, nor filtered), anything there were in Search field disappears as well.
During this reload, browser searchbar shows that the page changes to
https://e621.net/posts? and then instantly to https://e621.net/posts

That's a very nice way of saying "InstantFilters straight up do not work most of the time".
It's not "mean" or "picky", it's just a bug report.

I've got to admit that InstantFilters don't get much love from me, mainly because I don't really like that feature. I did not write it but I'm kind of stuck maintaining it.
It appears that my rewrite of the thumbnail rendering system a month ago broke it, and nobody (including myself) noticed.

I'll fix it in the next patch later today.

darthdesolus said:
UPD: enabled Debug, having this in console
DevTools failed to load SourceMap: Could not load content for chrome-extension://randomlettersblah.map
HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME
But I have that message on any page, btw.

That it an issue with jquery.hotkeys, a library that I use to manage key bindings. It's pointing to a source map that it cannot find, producing this error.
It's generally harmless. In either case, this issue will most likely be fixed once my ticket goes through, and I can switch to a less janky CDN.

bitWolfy

Former Staff
Patch 1.4.22

This is the project's 100th release. We have come a long way since the original alpha-0.1.0 - 1356 commits, to be precise.
The work is far from done, though. There are still more changes to be made, and more features to implement.

post #1535437

I wish I could offer something more exciting for this release, but unfortunately, this is mostly just bug fixes.

Changes
- Added a whitelist - posts with tags specified there will never be filtered out
- Fixed IntantFilter compatibility issues with the new thumbnail rendering system
- Fixed meta-key presses not going through on the subscription update previews
- Minor styling fix on the search statistics section

bitWolfy

Former Staff

libertarianhorsefukr said:
Very nice. This (the tag fetcher) is actually super handy to have for archival or moving content

I'm glad to hear that. Let me know if you have any other feature requests.

bitWolfy

Former Staff
Patch 1.4.23

Changes
- Removed post update frequency statistics from the favorites page
- Added a button that collapses the search page sidebar

bitWolfy

Former Staff
Patch 1.4.24

Hotfix release, addressing an issue introduced in 1.4.23.
Fixes
- Resolved an issue causing the sidebar to expand when hovering over source links
- Forced the external links under "related" on the post page to open in a new tab

I have an idea for the Subscription feature: What if there was a option to add a rating to certain tags or the over all Tag List. So now you can filter out unwanted ratings on tags.
For example a S Q E could be next to each tag so all you have to do is highlight the ones you want and now it will only notify you when that tag is with that rating.
Also there can be an option to have a all around rating for the Tags List at the top or bottom.

bitWolfy

Former Staff

itskilroy said:
I have an idea for the Subscription feature: What if there was a option to add a rating to certain tags or the over all Tag List. So now you can filter out unwanted ratings on tags.
For example a S Q E could be next to each tag so all you have to do is highlight the ones you want and now it will only notify you when that tag is with that rating.
Also there can be an option to have a all around rating for the Tags List at the top or bottom.

Sure, I can add that.
I am currently reworking the subscription system from scratch, so it might take some time.

bitwolfy said:
Sure, I can add that.
I am currently reworking the subscription system from scratch, so it might take some time.

Sweet, take all the time you need!

bitWolfy

Former Staff

darryus said:
seems like the pool navigation hotkeys have stopped working.

Site updated, breaking the navbars and navigation hotkeys.
I will have a release ready shortly.

bitWolfy

Former Staff
Patch 1.4.25

Changes
- Fixed pool and set navigation hotkeys that broke after a site update
- Added support for the new "both" option in the ThemeCustomizer's navbar settings
- Added video duration display to thumbnails
- Added blacklist support for duration meta tag. Range values are also supported.
- Added an option to suppress script update notifications

Congrats on the recent 100th release. Fantastic userscript, I love it :]

I wanted to ask at some point, I’ve noticed I have to manually click ‘check for updates’ on the script menu in Tampermonkey for it to update to the latest version, but I have it set to ‘update daily’ in the settings. Is there a step I missed?

Also here are some feature ideas I thought might be cool:

- Date of upload on hover for thumbnails. Or fixed somewhere in/around the thumbnail instead of hover. Just as a fun way to instantly see the time gaps between posts for various things.

- When you run a search with the max 40 limit and click on a post, above the image it shows all those tags and it often obscures the upper part of the image as well as the titles/pools and previous/next page links of comics. It would be nice to just hide or move that.

- While on the posts page, it would be helpful to have more ability to scroll the blacklist when running tests. If you have a blacklisted tag with lots of exceptions, it takes up a lot of space and it won’t let you scroll the blacklist without scrolling most of the entire page; sometimes making it impossible to see what’s being hidden/unhidden when you click off something on your blacklist side bar. I hope that made sense, haha

Updated

bitWolfy

Former Staff

human-shaped said:
Congrats on the recent 100th release. Fantastic userscript, I love it :]

I wanted to ask at some point, I’ve noticed I have to manually click ‘check for updates’ on the script menu in Tampermonkey for it to update to the latest version, but I have it set to ‘update daily’ in the settings. Is there a step I missed?

Thanks.
I'm not sure what the problem is with Tampermonkey. Are you running the Github version, or the Greasyfork one? The latter can delay updates for up to 24 hours.

human-shaped said:
Date of upload on hover for thumbnails. Or fixed somewhere in/around the thumbnail instead of hover. Just as a fun way to instantly see the time gaps between posts for various things.

Pretty sure that if you enable "Preserve Hover Text" under "Thumbnails" on the "General" settings page, it would show a block of information that includes the upload date on hover.
I don't think that there is any space to put it anywhere on the thumbnail itself.

human-shaped said:
- When you run a search with the max 40 limit and click on a post, above the image it shows all those tags and it often obscures the upper part of the image as well as the titles/pools and previous/next page links of comics. It would be nice to just hide or move that.

Huh. That actually seems like a vanilla site bug.
I think, I'll just have it cut off after the first line.

human-shaped said:
- While on the posts page, it would be helpful to have more ability to scroll the blacklist when running tests. If you have a blacklisted tag with lots of exceptions, it takes up a lot of space and it won’t let you scroll the blacklist without scrolling most of the entire page; sometimes making it impossible to see what’s being hidden/unhidden when you click off something on your blacklist side bar. I hope that made sense, haha

I can't seem to find a combination of tags that will have the blacklist extend outside the viewport, but I can see it potentially being a problem, especially with a fixed sidebar.
I will limit the length of the blacklist filters section to 50% of the viewport height - after that, it'll get a scrollbar.

bitWolfy

Former Staff
Patch 1.4.26

Fixes
- Limited the blacklist filter section in the sidebar to 50% of the viewport's height to avoid visual glitches with a fixed sidebar
- Fixed the search navbar on the post page overflowing when searching for a large number of tags

Oh nice, yeah you're right it does show the date when you have preserve text active. I must have turned it off before. xD

I picked the one that says 'Stable,' which I think is the github one. At the moment my version is 1.4.25 and the last update was 43 hours ago. Untill right now because I might as well update it again, haha. I wonder why it isn't being automatic daily :o

bitWolfy

Former Staff

human-shaped said:
Oh nice, yeah you're right it does show the date when you have preserve text active. I must have turned it off before. xD

I picked the one that says 'Stable,' which I think is the github one. At the moment my version is 1.4.25 and the last update was 43 hours ago. Untill right now because I might as well update it again, haha. I wonder why it isn't being automatic daily :o

I really dislike that hover text, which is why it's off by default. This script's default settings are still largely set up in a way that I personally like them by default. Might consider changing that in favor of something that more closely mimics the vanilla site at some point in the future.

Not sure what's going on with the updating. The "stable" version is from Github, yes.
I myself don't use the automatic updating feature at all, so I'll have to test it a bit.

Ah okay that makes sense now.

Yeah it's not a big deal it doesn't update automatically, but it would be great since you update so frequently :]

Also, sorry but I noticed two more things haha:

1.) I was browsing some comics and posts appearing as thumbnails on the main posts and artist page, but become blacklisted when on the images' post and pool pages.

I think it has to do with your new 'make score and favcount work on blacklist.' It's not allowing exceptions through, I think.

So I've got 'female comic' in the blacklist with a bunch of tag exceptions, but one of those exceptions is based on score and favcount. So imagine 'female comic -favcount:>x' Well even though a comic is over that X amount, it gets blacklisted when on the image page as well as the pool page.

2.) From that, I noticed if you click on a 'blacklisted' thumbnail, it shows the image. I can see the use for running tests (which I've been doing of late), but it would be great if it just stayed blocked, perhaps as an option. It's similar to the 'hide blacklist' feature you added. Just cutting off all possible chances of seeing blacklisted stuff :]

Updated

bitWolfy

Former Staff

human-shaped said:
1.) I was browsing some comics and noticed these two posts appeared as thumbnails on the main posts and artist page, but become blacklisted when on the images' post and pool pages.

https://e621.net/posts/2423431
https://e621.net/posts/2422797

Strangely, both have tags that my blacklist allows through. They both also meet my blacklisted score/favcount requirements. So something is saying 'this is allowed as a thumbnail but not as a full image or a thumbnail in the comic pool.' I've only noticed those two so far, but I figured I'd let ya know just in case there's a chance it has to do with the userscript in any way. Maybe it's the site. Or just me not seeing something obvious, haha

It's an ongoing issue. I am quite aware of it.
There are two separate blacklisting systems - the vanilla one, and the one I wrote. The vanilla one only supports the following meta-tags: id. rating, user, userid, and flags (listed here in the source code). I have expanded that list to support other meta-tags.

However, at the moment, my blacklisting system only works on the search page. Everywhere else, the normal blacklist picks up the slack - and it completely ignores meta-tags that it does not recognize, as if they weren't even there. So, it's likely that the search page shows the thumbnails because it matches the score / favcount requirements, and the post page hides the full image because it does not take those things into account.

I do plan on expanding my blacklisting system to other pages to avoid these kinds of issues. I'm still working on that, though.

human-shaped said:
2.) From that, I noticed if you click on a 'blacklisted' thumbnail, it shows the image. I can see the use for running tests (which I've been doing of late), but it would be great if it just stayed blocked, perhaps as an option. It's similar to the 'hide blacklist' feature you added. Just cutting off all possible chances of seeing blacklisted stuff :]

Not an error, but a deliberate feature. I'll add a setting to turn it off, though.

Sorry, I edited my last post too late it seems. But you already got what I figured in the edit, it being about metatags in the blacklist. I'm sure you'll find solutions in time, no rush~

And cool~! Thank you

Updated