Read the rules before proceeding!

Topic: TagMe! [v.1.0.3] Tag Project Assistant

Posted under e621 Tools and Applications

TagMe! Tag Project Assistant v.1.0.1

There are a lot of tagging projects that are unfinished - and many are unstarted. Some are getting worse with each passing day, to the point where taking on such a project becomes a daunting task.
And yet, a good chunk of those projects are quite simple at their core. Some require missing tags to be added, others to have tags removed or replaced. The vast majority can be resolved by asking a multiple-choice question.
This is exactly what this tool aims to do.

TagMe.dev

The basic concept is quite simple. You log in via your e621 API key, choose a project, and start clicking away. Let's take this project as an example:

pencil_(disambiguation): Should be replaced with either pencil_(object) for a writing tool, pencil_(artwork) for the art medium, or both.

While seemingly simple, this project still requires one to click on the post, look through the tags, manually remove the ambiguous tag, and replace it with the appropriate one. While this process does not take too much time, it still requires several actions, and allows for human error. Even for privileged users, an easy replacement like this would require fiddling with tag scripts. More complex projects are even more involved.

Meanwhile, this is how the same project would look on tagme.dev: https://i.imgur.com/MhTfyaG.png
It is quite obvious that the original tagger meant the physical pencil, not the art medium. So, to fix it, all you need to do is click the appropriate button, and press "submit". The utility will perform all the necessary tag changes for you, and you can move on to the next picture immediately.

To speed up tagging, the following hotkeys are available:
- Number keys select their relative options
- Tab skips to the next post
- Enter submits the post changes

Security and Vandalism

The TagMe! utility uses your username and API key to log in and submit changes to e621. The API key is not stored on the server, and is thus neither visible nor accessible to anyone but you.
This authentication method also means that all post changes are made in your name. E621's code of conduct fully applies, in regards to both tagging and comments left on project pages.

To prevent spam and ensure higher quality tagging, only users with 100 manual post edits can resolve projects.
Additionally, once you reach 1000 edits, you may create your own tagging projects with this utility.

Member-level users are limited to 150 post changes per hour. This is a limitation enforced by e621 itself, and cannot be bypassed.

Bug Reports / Feature Requests

Feel free to request feature and report bugs in either this thread, or on github. Either way, the progress to the next version is tracked through the projects page.
Alternatively, feel free to contact me on Discord @bitWolfy#7932

Changelog

Version 1.0.0

Initial Release

Patch 1.0.1
Changes
- Added a placeholder comment index. Far from ideal, but better than nothing.
- Added wildcard support to the removed tags field

Fixes
- Relaxed the size restrictions on project tag string length
- Possible solution to skipped images being cycled over and over again
- Added a fallback for post with missing sample images
- Fixed misaligned text on some action buttons
- Set a fixed height to added/removed tags sections
- Fixed the comment editing threshold not working correctly
- Fixed the page not reloading when submitting a comment

Patch 1.0.2
Changes
- Added an option to make a project unlisted. It will still be available through a direct link, but will not appear in the project index.
- Added an alternative wide-screen viewing mode, available by clicking on the link on the side, under the theme switcher.

Patch 1.0.3
Changes
- Project now records your progress. You can continue from the same image that you left on.
- Improved post sequencing in an effort to avoid repeating images that you have already skipped.

Updated

This is really cool. In the future would it be possible to make a non-public tagging project?
This would allow users to restrict a larger tagging project to only only the types of images they are interested in without cluttering the master list of projects. It would also allow for one-off tagging projects which don't really need to be brought to everyone's attention like a user working on tagging their own uploads or the uploads of an artist they are interested in.

tittybitty said:
This is really cool. In the future would it be possible to make a non-public tagging project?
This would allow users to restrict a larger tagging project to only only the types of images they are interested in without cluttering the master list of projects. It would also allow for one-off tagging projects which don't really need to be brought to everyone's attention like a user working on tagging their own uploads or the uploads of an artist they are interested in.

Sure, that would be pretty easy to implement.

I never thought the member-level hourly tagging limit would be relevant to me, but with this 150/hour isn't much at all. The good news is it's a good cue to log off for a while. I can't emphasize enough how good of an idea this is and it's extremely effective for an initial release.
My biggest roadblock so far has been image size. It would be helpful to have the image window be larger by default, but I'm not sure how that would work with the buttons.

tittybitty said:
I never thought the member-level hourly tagging limit would be relevant to me, but with this 150/hour isn't much at all. The good news is it's a good cue to log off for a while. I can't emphasize enough how good of an idea this is and it's extremely effective for an initial release.

I did not expect it either.
In fact, I completely forgot that this limit existed at all, until one of the guys who beta-tested the site for me ran into it.
That was a bit of a nasty surprise, not gonna lie.

tittybitty said:
My biggest roadblock so far has been image size. It would be helpful to have the image window be larger by default, but I'm not sure how that would work with the buttons.

You should be able to click on the image itself to zoom it in for a detailed view.
At least, that's the theory. In practice, that module is rather glitchy at the moment.

I have increased the size of the image somewhat.
In version 1.1, I will add an alternate display mode, looking something like this: https://i.imgur.com/CoUv066.png
But that's nowhere near ready yet.

Genjar

Former Staff

Excellent work. I might end up using this a lot, even though I already have my own tagging tool. I did notice a couple of issues, though:

- The length of the Tag String seems limited? I was able to fit around 11 tags, after which I got the 'Please match requested format' error. Yet the text says that maximum is 35 tags.
- The Skip-function seems to frequently loop the same posts. For instance, the interspecies project that I added constantly loads posts #1569503 and #1566979 (unbirthing warning), even though there should be 89 pages of posts for that search. (I had to temporarily add -vore to the search get around that.)

Updated

genjar said:
Excellent work. I might end up using this a lot, even though I already have my own tagging tool.

Thanks for checking it out, it means a lot to me.

genjar said:
- The length of the Tag String seems limited? I was able to fit around 11 tags, after which I got the 'Please match requested format' error. Yet the text says that maximum is 35 tags.

Might be limited by the number of characters instead of number of tags. I'll double-check.

genjar said:
- The Skip-function seems to frequently loop the same posts. For instance, the interspecies project that I added constantly loads posts #1569503 and #1566979 (unbirthing warning), even though there should be 89 pages of posts for that search.

I've noticed that as well, and I honestly couldn't tell you why it's happening.
I am not randomizing the posts myself. I simply query the API with the project's tags plus order:random, then get the first result. I suppose, it's possible that e621's random order isn't quite so random. I might have to ask Kira about that one.
For the time being, I will try alleviate the problem by getting a full page of results from the API, and then picking a random post from those. But I do not think that this would solve this issue completely.

bitwolfy said:
I am not randomizing the posts myself. I simply query the API with the project's tags plus order:random, then get the first result. I suppose, it's possible that e621's random order isn't quite so random. I might have to ask Kira about that one.

maybe check if caching might be interfering
that'd be my first guess anyway

would it be possible for this to automatically take implication trees into account when it tries to remove a tag? currently when trying to remove a tag that is being implied by one or more other tags on the post it'll just fail to do anything.
the only other solution would be when a user creates a tag project on here to just copy-paste the entire implication tree of a tag into the Removed Tags box (which is going to be super unwieldy in most situations).

genjar said:
Another small issue:
https://tagme.dev/projects/nonmammalballs/resolve/76398 consistently fails to load for me: https://i.imgur.com/XwYaxBL.jpg

No idea why, since I don't see anything particularly unusual about post #76398. The aspect ratio is non-standard, but that applies to many other posts that do load.

missing sample

Updated

Patch 1.0.1

Changes
- Added a placeholder comment index. Far from ideal, but better than nothing.
- Added wildcard support to the removed tags field

Fixes
- Relaxed the size restrictions on project tag string length
- Possible solution to skipped images being cycled over and over again
- Added a fallback for post with missing sample images
- Fixed misaligned text on some action buttons
- Set a fixed height to added/removed tags sections
- Fixed the comment editing threshold not working correctly
- Fixed the page not reloading when submitting a comment

All of this looks great.

bitwolfy said:
- Possible solution to skipped images being cycled over and over again

Sad to report that this is still a problem in my experience.

tittybitty said: Sad to report that this is still a problem in my experience.

I just deployed a hotfix that should fix that issue for good. Now, you should not get any repeated images at all until you cycle through all of them.
Or until you go back to a previous image, since that will reset the sequence. But it's still better than before.

order:random is very weird in what it displays. In my experience, it really likes to output a very specific set of images before moving on to something else, and that set is what kept getting cycled.

Thanks to Kira for some expert advice on the matter.

could the project name/meta name be added to the Edit Reason? I think being able to track individual tag projects done with this utility would probably be a good idea.

I'm kind of neurotic about adding my own tagging projects to this list because they kinda felt like my babies during my time on here, and I just want a way to be able to know if stuff is going smoothly.
It's not that I don't trust the peeps that are doing contributions at this point (or even in the future, since most of the people that are going to be using this utility are going to know what's up) or anything, I just want to be able to track the progress of any tag projects I let the wider tagging community get a hold of.

Updated

darryus said:
could the project name/meta name be added to the Edit Reason? I think being able to track individual tag projects done with this utility would probably be a good idea.

The edit reason currently uses the project ID because I don't want to bloat the it too much.
You can track changes made by individual projects by searching for something like this: https://e621.net/post_versions?search[reason]=tagme.dev/p/1

I might have to show the project ID somewhere on the project's page, though.

bitwolfy said:
The edit reason currently uses the project ID because I don't want to bloat the it too much.
You can track changes made by individual projects by searching for something like this: https://e621.net/post_versions?search[reason]=tagme.dev/p/1

I might have to show the project ID somewhere on the project's page, though.

I somehow didn't realize that the ID number was in there or that projects even had ID numbers since they don't show up anywhere. that's probably fine, then.
although, /post_versions?search[reason]=tagme.dev/p/1 is also showing projects 10-19 because how searching tag reason works.

Genjar

Former Staff

There's currently a serious bug. Untagging human also removed valid humanoid_genitalia and humanoid_penis tags. And I'm positive that I didn't have those listed as to-be-removed in the project.

genjar said:
There's currently a serious bug. Untagging human also removed valid humanoid_genitalia and humanoid_penis tags. And I'm positive that I didn't have those listed as to-be-removed in the project.

Was it this change ?

I replaced a very robust tag replacement system with a slightly wonkier one in 1.0.1 in order to implement wildcards in the removed tags field.
Just rolled back those changes. That was a very dangerous idea regardless.
My apologies.

darryus said:
I somehow didn't realize that the ID number was in there or that projects even had ID numbers since they don't show up anywhere. that's probably fine, then.
although, /post_versions?search[reason]=tagme.dev/p/1 is also showing projects 10-19 because how searching tag reason works.

Adding the project's meta name does not help much with the latter.
Apparently, e621 does something very kinky with the search string before it looks for it, because even after adding the meta name to the reason (tagme.dev/p/1/charnum) still returns the results for tagme.dev/p/10/bodytypes.

Searching for just the meta name works better.
I insist on keeping the ID in the link because the meta name can be changed - the option to do so is just disabled for normal users. Meanwhile, project ID will always remain the same.

Updated

bitwolfy said:
I just deployed a hotfix that should fix that issue for good. Now, you should not get any repeated images at all until you cycle through all of them.
Or until you go back to a previous image, since that will reset the sequence. But it's still better than before.

My experience now is that images are less likely to cycle back and things seem more random, but I'm still seeing images multiple times long before cycling through the whole list. Overall much improved, great work.

Is there a way to close or remove your own project if it's finished or becomes unnecessary or people decide it's a bad idea?

tittybitty said:
My experience now is that images are less likely to cycle back and things seem more random, but I'm still seeing images multiple times long before cycling through the whole list. Overall much improved, great work.

Hmm. The sequence might be breaking somewhere. I'll keep working on it tomorrow.

tittybitty said:
Is there a way to close or remove your own project if it's finished or becomes unnecessary or people decide it's a bad idea?

You should be able to delete the project from its main page.
Perhaps, the word "delete" isn't correct here, as the project simply gets hidden from view. You should still be able to see it from your user page, and can restore it at any time.

Edit: there's a bug with permissions that prevents you from deleting your own projects at the moment. Will fix shortly.
Edit2: fixed it

Updated

Genjar

Former Staff

I think I'm seeing less repeats in general, but on the other hand, one project got stuck repeating the same post over and over. post #2401981 for the pokemorph project. Definitely not because it's running out of posts, since that project has 155 pages of posts left.

Would it be possible to, for instance, add some feature to exclude posts from a project? Would be useful for false positives and for posts that are just too difficult to tag without asking for second opinions.

Updated

genjar said:
Would it be possible to, for instance, add some feature to exclude posts from a project? Would be useful for false positives and for posts that are just too difficult to tag without asking for second opinions.

I understand why this would be useful, but if multiple people are working on a project I would be worried about someone with incomplete knowledge using the exclude button where skipping or tagging would be more appropriate and permanently excluding things which shouldn't be excluded.

genjar said:
I think I'm seeing less repeats in general, but on the other hand, one project got stuck repeating the same post over and over. post #2401981 for the pokemorph project. Definitely not because it's running out of posts, since that project has 155 pages of posts left.

Would it be possible to, for instance, add some feature to exclude posts from a project? Would be useful for false positives and for posts that are just too difficult to tag without asking for second opinions.

Do you mean that project only shows that one specific post, or is it the fact that when you start resolving the project, it always brings up that image first?
If it's the former... that's probably a bug on my side. One that I am currently unable to replicate.

As for the latter - I think that something breaks the randomization on e621's side.
If you plug in the tags from that project (plus order:random) into e621's search, you get the same output every single time.
pokemon -group -duo -pokémorph ~wide_hips ~breasts ~humanoid_hands ~thick_thighs ~hair ~big_butt -feral order:random

I am guessing that it's the ~a ~b tags that are the cause, since removing them seemingly fixes it. I don't know for sure, though, and I can't fix it anyways.

As for excluding posts - technically possible, will have to think about how to best implement it.

tittybitty said:
I understand why this would be useful, but if multiple people are working on a project I would be worried about someone with incomplete knowledge using the exclude button where skipping or tagging would be more appropriate and permanently excluding things which shouldn't be excluded.

I was thinking about tracking how many times a post has been skipped, and if it's beyond some threshold, it would be added to a separate "needs attention" project and excluded from the main one.
But that's just an idea for the time being.

Updated

bitwolfy said:
I was thinking about tracking how many times a post has been skipped, and if it's beyond some threshold, it would be added to a separate "needs attention" project and excluded from the main one.
But that's just an idea for the time being.

How difficult would it be to add a per-user "skip-list", so that any post in this list would be excluded for the user? Such a list should be editable by users, or at least have some kind of clear/reset functionality, in case they would like to see the excluded posts again.

jakxxx3 said:
How difficult would it be to add a per-user "skip-list", so that any post in this list would be excluded for the user? Such a list should be editable by users, or at least have some kind of clear/reset functionality, in case they would like to see the excluded posts again.

Simple from the technical perspective, but potentially resource-intensive. There is a reason why the blacklist filtering didn't make it into the feature list.

As you probably know, all output filtering is done client-side on e621. And while on this site, displaying no results if the entire page has been filtered out is perfectly fine, on TagMe, there is an expectation that if a project is not yet complete, there would be a post displayed. So, if all of the posts on one page are to be filtered, I would have to send another API request to fetch the next one, and if all of those are gone too, I would send another, and so on.

This would be much less of a problem if the performance impact of sending API requests wasn't that drastically different. Requesting only one random image, as is done by default right now, only takes 0.28 seconds on average. Meanwhile, requesting a full page (320 posts) takes 2.57 seconds - almost ten times as long. You can see the difference for yourself if you switch from Shuffle mode to Repeat by clicking the button on the side, under the theme switcher. The former randomizes the posts using e621's order:random, the latter fetches the last page of results and grabs a random post from there.

The basic premise of TagMe hinges on quickly going through obvious posts, leaving the complex cases to be fixed manually. Staring at a loading spinner for several seconds on every page does not really help with that.

Upon further consideration a per project skip list l would probably be fine as long as there was an option to opt out and show the skipped images. I have no idea if that's resource-wise any easier than a per-user skip list.

bitwolfy said:
The basic premise of TagMe hinges on quickly going through obvious posts, leaving the complex cases to be fixed manually. Staring at a loading spinner for several seconds on every page does not really help with that.

Is having a random order actually that important? I wouldn't mind seeing posts in upload-date order.

genjar said: Yeah, seems like random is not random. The first post in that search is the one that kept looping on skip..

Thanks to Kira, the issue on e621's side should now be fixed.
Random posts are now random again, huzzah!

jakxxx3 said: Is having a random order actually that important? I wouldn't mind seeing posts in upload-date order.

It's mainly to avoid collisions, in case several people are working on the same project.

I went through and reviewed the order:random code and found that it was producing predictable and incorrect results when used with OR terms. I had placed the scoring function in the wrong place and posts were being scored and then boosted based on a random number instead of having their score entirely replaced with a random number. The result of this is that the more optional terms that matched on a post, the higher on the list it would appear. This did not occur on normal searches because scores are static when only using required or negated required searches, so the random boost would properly randomize.

This should fix that some searches and thus, projects, would not randomize correctly.

Patch 1.0.2

Changes
- Added an option to make a project unlisted. It will still be available through a direct link, but will not appear in the project index.
- Added an alternative wide-screen viewing mode, available by clicking on the link on the side, under the theme switcher.

bitwolfy said:
h5. Patch 1.0.2
Changes
- Added an option to make a project unlisted. It will still be available through a direct link, but will not appear in the project index.
- Added an alternative wide-screen viewing mode, available by clicking on the link on the side, under the theme switcher.

Works great on both counts!

tittybitty said: Works great on both counts!

That's good to hear.
Thank you for your tagging contributions, by the way. It's greatly appreciated.

Patch 1.0.3

Changes
- Project now records your progress. You can continue from the same image that you left on.
- Improved post sequencing in an effort to avoid repeating images that you have already skipped.

I've run into a couple of cases now where a project gets stuck searching for the next image. Image number equals 0 and it just spins. If I change the searched tags I can get it to load again, but when I change them back the problem returns.

Genjar

Former Staff

tittybitty said:
I've run into a couple of cases now where a project gets stuck searching for the next image. Image number equals 0 and it just spins. If I change the searched tags I can get it to load again, but when I change them back the problem returns.

I've noticed the same with my solo_focus project. There's 15 posts left to clean at the moment, but it's stuck spinning. The first post in that search is an animated gif (post #2420190), not sure if that's what it gets stuck on.

bitWolfy

Privileged

tittybitty said:
I've run into a couple of cases now where a project gets stuck searching for the next image. Image number equals 0 and it just spins. If I change the searched tags I can get it to load again, but when I change them back the problem returns.

genjar said:
I've noticed the same with my solo_focus project. There's 15 posts left to clean at the moment, but it's stuck spinning. The first post in that search is an animated gif (post #2420190), not sure if that's what it gets stuck on.

Fixed it.
The project ran out of posts and got stuck trying to reset back to the beginning.

going through this has made me realize how many edge cases are out there aaaaaaaa

...On that note, would it be okay to discuss ambiguous posts, tagging issues, etc. here, or would you prefer to keep this to discussion of the app itself?

bitWolfy

Privileged

strikerman said:
going through this has made me realize how many edge cases are out there aaaaaaaa

...On that note, would it be okay to discuss ambiguous posts, tagging issues, etc. here, or would you prefer to keep this to discussion of the app itself?

There will always be edge cases. If it's a one-time thing, you can always add or remove any tag you want by editing the "New Tags" field below the buttons manually.
If it's something that keeps coming up over and over again, it might be prudent to create a new tagging project instead, with a narrower scope.

I don't mind tagging discussions in this thread, although you might not get good responses here.
I personally prefer discussing things like that in the #tag-discussions channel on Discord.

bitwolfy said:
I don't mind tagging discussions in this thread, although you might not get good responses here.
I personally prefer discussing things like that in the #tag-discussions channel on Discord.

I feel like I'm missing out on stuff because I'm not on the e6 Discord server, but I'm not going to mix usernames, and Discord dosn't let you multilog, so :/.

darryus said:
I feel like I'm missing out on stuff because I'm not on the e6 Discord server, but I'm not going to mix usernames, and Discord dosn't let you multilog, so :/.

Same tbh

bitWolfy

Privileged

darryus said: I feel like I'm missing out on stuff because I'm not on the e6 Discord server, but I'm not going to mix usernames, and Discord dosn't let you multilog, so :/.

strikerman said: Same tbh

There are people in there who don't use their e621 usernames. But you do you.

bitwolfy said:
There are people in there who don't use their e621 usernames. But you do you.

I just don't like the idea of using the username that I use for my Youtube and acting as a mod on a Discord server and stuff for furry stuff. it kinda goes both ways, I just don't really like random people from one community knowing what stuff I like/do in another community, only my closest friends really know more than one or two of my usernames. it's a similar reason as to why I don't fav anything.

Will there be a way in the future to tag posts individually from a group of added tags?

bitWolfy

Privileged

bara-king said:
Will there be a way in the future to tag posts individually from a group of added tags?

I am honestly not sure what you mean by that.

bitwolfy said:
I am honestly not sure what you mean by that.

When I select an option that has more than one tag (E.x if the topic covers types of hair color and my Option 1 has multiple tags such as blue_hair and green_hair), will there be a way to add either blue or green hair individually without them being grouped together when selecting "Option 1?" Hope that's better

bitWolfy

Privileged

bara-king said:
When I select an option that has more than one tag (E.x if the topic covers types of hair color and my Option 1 has multiple tags such as blue_hair and green_hair), will there be a way to add either blue or green hair individually without them being grouped together when selecting "Option 1?" Hope that's better

I do not see how that would be useful.
In this case, you should split the different hair colors into separate options. TagMe's purpose is to add tags relatively quickly, without fiddling with various sub-options.

bara-king said:
When I select an option that has more than one tag (E.x if the topic covers types of hair color and my Option 1 has multiple tags such as blue_hair and green_hair), will there be a way to add either blue or green hair individually without them being grouped together when selecting "Option 1?" Hope that's better

I see what you're thinking in the body hair project and I definitely empathize with the impulse to push the tool into more and more complex projects. My suggestion would be to try to find some way to split up the project into smaller sub-projects that are more feasible.

would it be possible to have an option that adds a post to a set rather than adding any tags to a post? that way if there are any edge case posts you can have them not show up anymore, a sort of "permanent skip".

darryus said:
would it be possible to have an option that adds a post to a set rather than adding any tags to a post? that way if there are any edge case posts you can have them not show up anymore, a sort of "permanent skip".

That's a really clever idea.

bitWolfy

Privileged

darryus said:
would it be possible to have an option that adds a post to a set rather than adding any tags to a post? that way if there are any edge case posts you can have them not show up anymore, a sort of "permanent skip".

tittybitty said:
That's a really clever idea.

It's kind of difficult to implement.
Only the creator and maintainers can add posts to a set - there is no way to make it publicly editable. So, if someone who is not on that list clicks that button, it simply won't do anything.

Of course, I could dynamically create exclusion sets for every user, but I do not think that it's a good idea.

bitwolfy said:
It's kind of difficult to implement.
Only the creator and maintainers can add posts to a set - there is no way to make it publicly editable. So, if someone who is not on that list clicks that button, it simply won't do anything.

Of course, I could dynamically create exclusion sets for every user, but I do not think that it's a good idea.

hmmm... yeah, that's what I was kind of worried about; I've never really used sets before so I wasn't 100% how they worked, or if it would be possible to create a set that people could join without needing an invite/have a bot that automatically added users to a set or whatever.

  • 1