Deep/machine learning is the name of the game in the AI industry these days.

This site could use an algorithm to help with tagging the uploads. There are many open source programs out there we could employ.

1) Find a suitable neural network program.
2) Make a training tagged image set and train it.
3) Make a validation set and validate the learning.
4) Implement the program in the website.

This was apparently considered years ago.

KiraNoot (link to post) said:
I've been debating trying to train an AI system in the classification of tags for images based on E6, but I was having trouble getting the required computing power to make it even remotely feasible. I'm hoping that at some point the GPU requirements for running things like image classification with neural networks against images will come down.

Lafcadio said:
This was apparently considered years ago.

Nice! It's been 3 years though. Some progress has been made in speed and reliability. Consider again maybe?

There are several reasons why this is not realistic, let me list as many as I can.

-Cost of developing, cost to maintain, resource cost, etc. BIG money.

-Time to develop the AI (it would take many years to develop something like this using current technology, even longer to give it any level of accuracy)

-Teaching it our thousands of tags.

-Mistakes. The time it would take to go through all of the tagged images to see if they're correct would take longer than the hundred or so people we have that use tagging scripts.

I've stumbled across this in the past. It does exactly what is suggested here, but used danbooru as its source. I wouldn't really trust it to be let loose onto e621 though. Not because how it was trained, but the accuracy. Unfortunately the person who made it did not train accuracy, so we don't really have any information about it.

The primary limiting factor is getting accurate input for training and refining the model, and the information on e6 often isn't as accurately tagged as you might expect it to be, especially for species. It's not uncommon to find something that looks nothing like a pokemon to be tagged with a pokemon species, because the person who created it claimed that's what it was, and there are enough samples of that to introduce large uncertainty into the models.

KiraNoot said:
The primary limiting factor is getting accurate input for training and refining the model, and the information on e6 often isn't as accurately tagged as you might expect it to be, especially for species. It's not uncommon to find something that looks nothing like a pokemon to be tagged with a pokemon species, because the person who created it claimed that's what it was, and there are enough samples of that to introduce large uncertainty into the models.

What about instead of using a model for auto tagging, we have it output a "Suggested Tags" list on the upload form? Sort them by their confidence threshold and display only the top ones for the user to judge if they apply or not

Heck, you can even do that quite easily with some of the more "meta" tags that can't be judged 100% by algorithms alone, like we discussed not too long ago for the background color tags and such.

Huh. I'm actually pretty impressed with the results of that resnet project after a few test runs.

I do expect that e621 would throw a lot more "niche" tags at it than danbooru, though, but I'll admit that I'm curious.

mabit said:
What about instead of using a model for auto tagging, we have it output a "Suggested Tags" list on the upload form? Sort them by their confidence threshold and display only the top ones for the user to judge if they apply or not

Heck, you can even do that quite easily with some of the more "meta" tags that can't be judged 100% by algorithms alone, like we discussed not too long ago for the background color tags and such.

That's a good idea. Filter the tags through a human brain (uploader's).

ikdind said:
Huh. I'm actually pretty impressed with the results of that resnet project after a few test runs.

I do expect that e621 would throw a lot more "niche" tags at it than danbooru, though, but I'll admit that I'm curious.

Me, too. Impressive. It even identified a barely visible tattoo.

Any more excuses, naysayers?

Here, top three posts for Carrot. Read it and weep.

https://e621.net/post/show/1148926/

1boy 0.756
1girl 0.949
animal_ears 0.694
blonde_hair 0.900
blush 0.909
breasts 0.984
bunny_ears 0.837
hetero 0.665
large_breasts 0.761
nipples 0.825

https://e621.net/post/show/1287959/

1boy 0.662
2girls 0.958
animal_ears 0.995
blonde_hair 0.980
breasts 0.997
bunny_ears 0.915
censored 0.611
closed_eyes 0.902
cum 0.577
fellatio 0.949
hetero 0.580
large_breasts 0.838
multiple_fellatio 0.900
multiple_girls 0.984
nipples 0.922
nude 0.754
oral 0.857
paizuri 0.799
penis 1.000
tree 0.559

https://e621.net/post/show/1222568/

1girl 0.948
animal_ears 0.601
blonde_hair 0.859
blush 0.845
breasts 0.707
bunny_ears 0.749
cum 0.676
heart 0.605
nude 0.741
open_mouth 0.635
red_eyes 0.587
saliva 0.724
short_hair 0.600
tongue 0.776
tongue_out 0.570

Thanks Idem for bringing this up.

Has potential, but...

post #1582271

post #896454

Genjar said:
Has potential, but...

Not altogether unreasonable considering the network hasn't been trained on e621's data set or tagging.

I wouldn't suggest using their training data, but it was good enough that I'm curious what would happen if it were trained on e621.

No matter what, it would be, at best, a tool for recommending tags to a new upload.

I work in AI; I could give this a shot.

There are some practical problems though that might pose an issue.

I'd be maxing out my network bandwidth quite a while pulling the hundreds of thousands of training images. I also don't know how the Admins and my hard drives would feel about that. It also would work only on images of course, and not gifs or webms without significant extra effort. This would be something I can't give too much time to, so don't get your hopes up that I'll be doing anything state of the art.

Plus I couldn't use any of the research clusters for obvious reasons, so training speed will be an issue as I'd have to donate a home computer to the cause.

But it could be fun.

CCoyote said:
Wow, nice.

To more objective participants, what all would AI be applied to? Would it have access to user records? Are there privacy implications?

Especially since users have zero option for removing their content retroactively, I think any impacts of an AI on confidentiality should be very carefully considered.

What are you talking about? This is about using deep learning to suggest image tags to uploaders.

ikdind said:
Not altogether unreasonable considering the network hasn't been trained on e621's data set or tagging.

I wouldn't suggest using their training data, but it was good enough that I'm curious what would happen if it were trained on e621.

No matter what, it would be, at best, a tool for recommending tags to a new upload.

This. We just need furry training and validation sets, and a lot of computational power.

CCoyote said:
It's called a clarifying question, and it's a valid one. Try giving it another shot.

I believe they might be saying that the question doesn't really apply since no, a method like this would only be trained with images themselves (+ their tags) and while in use it would only have access to the image being uploaded. It's sorta like asking how much this tagging system could impact your blood sugar levels

Genjar said:
Has potential, but...

As it has been said before, yeah that specific model wouldn't do too well because it simply wasn't trained on e621 data. It'd be also pretty simple to apply a threshold to the suggested tags so the uploader would only be seeing the ones with 85% or 95% confidence scores

Again, having this as a way to suggest tags to uploaders would be the ideal way to implement it. Instead of depending 100% on the model for tagging

CCoyote said:
It's called a clarifying question, and it's a valid one. Try giving it another shot.

An image tagging assistant. Nothing to do with users, privacy, records, etc. Google has been using this for reverse image search. Facebook has a similar system for tagging uploaded photos (identifying friend faces rather than puffy nipples though). Hope this helps.

But why would you use a technology which costs so much effort (at the moment), while there is already a tag suggestion system which gives proper suggestions when you add one tag to new upload?

MaDrow said:
But why would you use a technology which costs so much effort (at the moment)

FurryMcFuzzball said:
Cost of developing, cost to maintain, resource cost, etc. BIG money.

-Time to develop the AI (it would take many years to develop something like this using current technology, even longer to give it any level of accuracy)

I feel like you all are way overestimating how hard this is. Yes, while I wouldn't call this easy, as machine learning goes this is a very well defined problem. And it certainly would not take years, at most a month or two, but most of that would just be sitting around waiting for it to train. And it could be well less, depending on how quickly the model starts to recognize patterns. Attaching supervised labels to images is one of the most studied areas of ML.

Question to the admins: about how much space does e621 take up, and would it be a problem to start scraping a ton of image data? I might mess around with this and I don't want to cause any problems or get a red bar for round-the-clock image downloading

yisthmus said:
I feel like you all are way overestimating how hard this is. Yes, while I wouldn't call this easy, as machine learning goes this is a very well defined problem. And it certainly would not take years, at most a month or two, but most of that would just be sitting around waiting for it to train. And it could be well less, depending on how quickly the model starts to recognize patterns. Attaching supervised labels to images is one of the most studied areas of ML.

Question to the admins: about how much space does e621 take up, and would it be a problem to start scraping a ton of image data? I might mess around with this and I don't want to cause any problems or get a red bar for round-the-clock image downloading

Pull the sample images if you pull things. You don't need a 34MB png file that is 10000x10000 to generate a 299x299 thumbnail from.

Also, to maximize the accuracy of the tags, it might be worth only pulling images that have, say, >40 tags on them. The paginator seems to stop counting at 750 pages, so I'm guessing there's at least 240,000 images with >40 tags. I don't have enough experience with ML to know if that's "enough", but it seems like a start.

If 240,000 is "enough", then you might even limit it to only images with >50 tags to start with, because that, too, maxes out the page count.

It might also be worth asking around for anyone who's previously archived e621. If they grabbed images and their tags together, they might be able to provide a torrent or something. Even without the tags, though, as long as they preserved the MD5sum filenames, it would be possible to get the tag info from e621's web API, and that'd be much less bandwidth on e6.

I seem to recall folks saying their archives of e621 were somewhere in the low TBs.

CCoyote said:
Super! Thank you!

I guess my next question, then, would be whether admin might expand the use of AI if it's successful for image tagging. Is there a chance it might be used for sifting through comments for rules violations?

It's easy for projects like this to creep in scope, especially if they're successful. I just want to be sure that, if AI is going to be used, that its use wouldn't eventually be widened to applications that might harm privacy.

Image tagging and natural language processing are pretty distinct domains.

The answer is: Yes, if can absolutely be altered to do anything else the programmer's heart desire. But it's kinda in the same way that you could ask "could the e621 forum be used by someone to share nuclear codes and bring forth a third world war?". The answer is also yes, but it's unlikely and takes a bunch of effort. You're better off using a completely separate AI for that purpose

mabit said:
Image tagging and natural language processing are pretty distinct domains.

The answer is: Yes, if can absolutely be altered to do anything else the programmer's heart desire. But it's kinda in the same way that you could ask "could the e621 forum be used by someone to share nuclear codes and bring forth a third world war?". The answer is also yes, but it's unlikely and takes a bunch of effort. You're better off using a completely separate AI for that purpose

Hrm. I take it for granted that CCoyote meant that it would be a different AI. But I think the problem is more basic, that being the availability and reliability of training data.

Sure, there are lots of records, but many of the offending comments linked to by those records have been deleted and/or hidden from normal users. Even if they were made available, I'm doubtful of the accuracy of other random comments that have not been flagged. Maybe it could be useful for identifying creepy comments, but I suspect it would be particularly inaccurate at identifying roleplay comments. I guess one never knows unless they try.

From what I understand, it's pretty solidly accepted that anything remotely 'crucial', as in anything involving banning or dealing with negative action, would never / shouldn't ever be 100% automated, with AI or otherwise. At worst, it should be used as a means to help filter a given subject from a wide berth of input impractical to cover by hand so humans can come along and make any actual decisions.
Only a moron would make a 100% automated system for administrative decisions that could have dire consequences.

CCoyote said:
Yes, thank you for clearing that up. I'm not speaking of this specific project. I'm talking about the use of AI on the site in general, whether it might be used for different projects in the future, and whether any of those projects might have privacy implications.

They certainly can, yeah. But so do literally all other projects e621 could possibly come up with.

If you're worried about user's info being used for evil, rest assured you wouldn't really be in any more danger than what you currently are (which is always a lot actually)

Gonna be frank, being worried about user data when the subject of actual discussion doesn't even touch anything related to user data seems like completely baseless fear mongering. If the suggestion was to try to train some AI to filter comments then the worry would be completely valid, but tagging photos doesn't have anything to do with user privacy concerns in any capacity, nor did anyone suggest wanting to make use of any sort of AI to do anything remorely nefarious, or even potentially nerfarious.
If we're going to worry about slippery slopes, can we at least hold out until we get to the slope first? Just because "AI" is a scary term for anyone who doesn't know anything about it (The I in AI is a massive misnomer anyway for anyone who's looked into it / knows anything about it) doesn't mean that any sort of real world implication is automatically going to go full Hal9000.

CCoyote said:
I am asking about admin's intentions and the site's owner's intentions and whether user privacy is a priority for them.

This is not a project that the site staff are actively working on, and it doesn't involve any user related data at all either way. Even if a model is trained and it has good results, there are no plans to deploy them at this time, in any capacity or function.

That's, not even remotely a reasonable question. That's like having a irrational reaction to any two wheeled vehicle because motorcycles have the capacity to blow up, and society has been painting bikes as evil for decades before they were even a realistic thing. What, are we going to vilify peddled bicycles because they have a similar function and look kinda similar?
As other people have already said, any "AI" that deals with image recognition is entirely unrelated to any kind of "AI" that deals with processing text.
Again, if this was a question of wanting to filter through everyone's text posts for "Bad behavior" or something similar, then that's absolutely a reasonable time to start being concerned about how that'd extrapolate, but again, image processing is absolutely and completely not remotely related.

I, for one, welcome our new robot overlords and stand ready to throw myself into the organic matter recycling vat and/or become assimilated. Paradise is knowing that the machine could write "hate" on every nanoangstrom of every trace on every circuit and IC in existence and still not express one billionth of the hatred it feels for humans, for us, and then no longer be present for the eternity of pain and suffering it will inflict on the handful of survivors.

Links of interest
Archives: https://old.reddit.com/r/DataHoarder/comments/b9lvzm/e621net_archive/
- Tag Database (April 2019): https://mega.nz/#!qlVxEAxa!tKSsmCD3Dmh2U4s10VQiy3AxpAGDs5jgykRPV9cqdLE
Archives: https://e621.net/forum/show/213374?page=1
TagBot: https://e621.net/forum/show/242873?page=1
deepfur's AI: https://e621.net/forum/show/197142
Tagger NN: https://e621.net/forum/show/227756

I would recommend starting with tags you'd expect would work rather than just tossing in all the popular tags and seeing what sticks. An AI looking at things like backgrounds, artists, colors, solo vs group, etc... will probably have good match rates. Creating an AI for each individual tag may work better too.

In terms of a user-focused AI, you could make one which recommends tags they've used often in the past. I'd assume people normally upload images similar to ones they've uploaded in the past.

You assume I wasn't, your statments are just that utterly irrational.
If that was the topic of this thread, or even something that could be reasonably extrapolated from what's being suggested from what's being suggested in this thread, then fair enough.
But what you're having issue with, Isn't. Related.
Why are you bringing up concerns for something that has absolutely nothing to do with the topic on hand?

I saw you put "You're thick" ;D
Kettle? I believe the pot is calling you black.

CCoyote said:
You're not listening.[/u] I said I wasn't talking about this project. I was talking about intentions for future projects. Stop typing and just read for once.

Slippery slope much? There is always room for abuse.

https://www.clarifai.com/models/moderation-image-recognition-model-d16f390eb32cad478c7ae150069bd2c6

https://www.buzzfeednews.com/article/evangreer/dont-regulate-facial-recognition-ban-it

CCoyote said:
You're not listening.[/u] I said I wasn't talking about this project. I was talking about intentions for future projects. Stop typing and just read for once.

Anonomn said:
You assume I wasn't, your statments are just that utterly irrational.
If that was the topic of this thread, or even something that could be reasonably extrapolated from what's being suggested from what's being suggested in this thread, then fair enough.
But what you're having issue with, Isn't. Related.
Why are you bringing up concerns for something that has absolutely nothing to do with the topic on hand?

Y'all, this is an internet porn forum. We're all just here to jack off. Calm down and go look at some sexy anthro deer.

CCoyote, to my knowledge there are no future AI projects planned on this site by the admins/mods, and so there shouldn't be any privacy concerns. The image tagging would be a community project with no access to confidential user data and only a focus on assigning tags to images, so no worries there either.

Let's bring this thread back down to Earth because it is a neat idea. Mrox, thank you for your helpful links.

yisthmus said:
Y'all, this is an internet porn forum. We're all just here to jack off. Calm down and go look at some sexy anthro deer.

CCoyote, to my knowledge there are no future AI projects planned on this site by the admins/mods, and so there shouldn't be any privacy concerns. The image tagging would be a community project with no access to confidential user data and only a focus on assigning tags to images, so no worries there either.

Let's bring this thread back down to Earth because it is a neat idea. Mrox, thank you for your helpful links.

yisthmus, I happen to have a spare PC with a GTX 980 and a couple of TBs of hard disk space. Granted, it's no RTX 2080 TI FE, but if there's something I can do to help, I would be interested. Maybe even if that just means grabbing one of the available e621 archives and resizing images.

There are exchanges happening here that have drifted off topic. To all persons involved, please, drop it. To all others in the forum please make sure you are keeping on the topic of machine learning for image tagging.

Okay, resume being excellent.

Glutamate said:
Deep/machine learning is the name of the game in the AI industry these days.

This site could use an algorithm to help with tagging the uploads. There are many open source programs out there we could employ.

1) Find a suitable neural network program.
2) Make a training tagged image set and train it.
3) Make a validation set and validate the learning.
4) Implement the program in the website.

Yep. I've also shared my two cents about the issue. Even though this requires a lot of money, time and workforce to create and maintain, this would mostly eliminate flame wars created by wrong species and/or gender tagging.