E: Changed title from transliteration -> transcript...

Requested feature overview description.
Add in section to have transcript of text and possibly other things featured in the image.

Why would it be useful?
With foreign content, this would allow users to quickly google translate content that haven't been translated already as well as easier way to check if exsisting translation is actually proper and not a joke/made up one.

With video content this acts as semi-subtitle of sort (or better yet, as actual subtitles which user can then copy-paste to file and use).

But most importantly, this would greatly help with searching content. I'm certain that everyone will bump into situation where they know what text was in the thing that they are searching, but have no idea what else there was. Right now all they can use to search then is text.

So for example, searching transcript:what's_a_yiff? would turn the following result:
post #1367432
As post pages are indexed by google, this would also immensily help finding content here trough google as well.

What part(s) of the site page(s) are affected?
There would be extra textbox when editing posts and this would show up as collapsed section under description.

Google Translate already transliterates stuff for you, so I don't see much of a point of having "transliterated text" on an image. And it's already possible to perform text search on the notes an image has.

There's one feature that's missing tho. Searching for text on images without notes. So perhaps what you're suggesting is adding a simple "text content" field where people could write in the text that's on the image?

I forgot to comment: I really like this idea. it's a fantastic idea.

I think this would be a really cool feature. There's a handful of times I've been looking for an image and I knew exactly (or roughly) what text was in the image, but that does absolutely no good.

Delian said:
Google Translate already transliterates stuff for you, so I don't see much of a point of having "transliterated text" on an image. And it's already possible to perform text search on the notes an image has.

There's one feature that's missing tho. Searching for text on images without notes. So perhaps what you're suggesting is adding a simple "text content" field where people could write in the text that's on the image?

Google does? I cannot find the example image above (post #1367432) in any way with google when trying to search for that specific pharse. I do get kym and wikifur however thanks to google thinking "yiff" and "furry" to be possible synonyms.

Also english text in images are never noted, so you cannot search for images text if the image has it in english. Also you cannot note flash or video posts at all, so translations are provided as external subtitle file downloads and comments now.

So yes, that's exactly my point. Writing what is on the image, because english can't be noted and with non-english original language isn't noted either. Only thing that's tagged of text is text and language of it, but there's no way of searching if for example character is saying something specific right now. This is also pretty common practise for those with visual impairs.

Mairo said:
Google does? I cannot find the example image above (post #1367432) in any way with google when trying to search for that specific pharse. I do get kym and wikifur however thanks to google thinking "yiff" and "furry" to be possible synonyms.

I think they just meant the Google Translate mobile app can already be used as an OCR program (i.e. convert a picture to text) https://i.imgur.com/2MgvjGM.png. But that's irrelevant because, as you and others have said, the point of this feature is that the text could be used for searching for images, which Google obviously can't do.

+1

This is a great idea, having a transcript of a post's original text right below the content would do wonders for accessibility and translation

I'd also like to suggest that users should be able to choose a language when they add a transliteration so we can have tabs showing original text, translated into english, etc. Sorta like how youtube currently does their subtitles where users may upload translations for different languages. Those changes might make translating posts much more standardized than the current "adding translations via notes" system

mabit said:
+1

This is a great idea, having a transcript of a post's original text right below the content would do wonders for accessibility and translation

I'd also like to suggest that users should be able to choose a language when they add a transliteration so we can have tabs showing original text, translated into english, etc. Sorta like how youtube currently does their subtitles where users may upload translations for different languages. Those changes might make translating posts much more standardized than the current "adding translations via notes" system

This is also a neat idea... if we got really enthusiastic, this could possibly integrate with notes displayed onscreen, too...

so , more YAS from me

Mairo said:
Google does? I cannot find the example image above (post #1367432) in any way with google when trying to search for that specific pharse.

I think you're misunderstanding what the word "transliterate" means. Please go look up the meaning of words such as "translate", "transliterate" and "transcribe".

CrocoGator said:
I think they just meant the Google Translate mobile app can already be used as an OCR program

No one's talking about OCR. Yet. But now that you mention it, being able to perform OCR on a selected area of an image would be quite useful. Btw, you don't need the phone app, you can simply upload an image (or a crop of the text) to Google Drive, and then open it in Google Docs to get results under the image.

Delian said:
I think you're misunderstanding what the word "transliterate" means. Please go look up the meaning of words such as "translate", "transliterate" and "transcribe".

I'm not native english speaker, so I did have trouble finding the right word for what I was meaning.

But yeah, I mean taking text in the image and putting it in format that can be searched and copy-pasted easily.

mabit said:
+1

This is a great idea, having a transcript of a post's original text right below the content would do wonders for accessibility and translation

I'd also like to suggest that users should be able to choose a language when they add a transliteration so we can have tabs showing original text, translated into english, etc. Sorta like how youtube currently does their subtitles where users may upload translations for different languages. Those changes might make translating posts much more standardized than the current "adding translations via notes" system

One of the ideas was that even if we didn't go too fancy with it, users could easily copy the text to google translate themselves. Especially when only thing that services like youtube are doing is put the text trough google translate anyway.

I think this a great idea overall, even if I get the creepy feeling that more notes will be filled with Google translations as a result.

+1

SnowWolf said:
This is also a neat idea... if we got really enthusiastic, this could possibly integrate with notes displayed onscreen, too...

I thought the same, but that might require a much bigger rework of the current systems so I'd put it just as a "nice to have"

Mairo said:
One of the ideas was that even if we didn't go too fancy with it, users could easily copy the text to google translate themselves. Especially when only thing that services like youtube are doing is put the text trough google translate anyway.

While youtube does have machine-translated subtitles, I'm pretty sure it also allows users to contribute manually translated ones. A similar system could be implemented here, and since machine-translated would be one of the defaults that would encourage users to always try and write the original text
For any text that the users themselves can't transliterate, we could always have a "transliteration_needed" tag

in forum #256947 Clawdragons suggested, essentially, making transcript words blacklistable.... which is a really great idea and I honestly think should be included as part of this feature:

Maybe it's a separate transcript-only blacklist, or some context like transcript:cuuuuum... either way!

SnowWolf said:
in forum #256947 Clawdragons suggested, essentially, making transcript words blacklistable.... which is a really great idea and I honestly think should be included as part of this feature:

Maybe it's a separate transcript-only blacklist, or some context like transcript:cuuuuum... either way!

I don't believe that's too feasible

You'd be able to blacklist "cuuuum", but if we're using the same blacklisting system as the rest of the site you'd still see every post that had "cuuum", "cuuuuum", etc (unless the blacklist allows for regex matching but that's too complex for the average user, and even then it's way too specific so you'd have to add a whole separate entry just to block stuff like "coooooomiiiiiiing!")

It's just not worth the effort. When dealing with user-inputted text we have a much harder time in keeping it standardized so it's much simpler to just create tags for the most commonly blocked text content (slurs, cuuuuuuuuuuum, memes, etc)

mabit said:
I don't believe that's too feasible

You'd be able to blacklist "cuuuum", but if we're using the same blacklisting system as the rest of the site you'd still see every post that had "cuuum", "cuuuuum", etc (unless the blacklist allows for regex matching but that's too complex for the average user, and even then it's way too specific so you'd have to add a whole separate entry just to block stuff like "coooooomiiiiiiing!")

It's just not worth the effort. When dealing with user-inputted text we have a much harder time in keeping it standardized so it's much simpler to just create tags for the most commonly blocked text content (slurs, cuuuuuuuuuuum, memes, etc)

...."cuuum" was a joke :p

For several years "cuuuuuuum" was basically on every single page of a comic made by a very prolific artist group (Palcomix had about 6000 posts here before they were deleted)

Presumably, no one would blacklist ccuuuum. They'd blacklist slurs, or degrading words. Maybe someone enjoys content where characters are called sluts, but dislike the word whore. Or they can't take their scat seriously when someone says "poo" or "poop".

And while, yes, it WOULD be easier to create tags for those text ideas, we're discussing, here, a new feature on the website that currently doesn't exist.

mabit said:
I don't believe that's too feasible

You'd be able to blacklist "cuuuum", but if we're using the same blacklisting system as the rest of the site you'd still see every post that had "cuuum", "cuuuuum", etc (unless the blacklist allows for regex matching but that's too complex for the average user, and even then it's way too specific so you'd have to add a whole separate entry just to block stuff like "coooooomiiiiiiing!")

It's just not worth the effort. When dealing with user-inputted text we have a much harder time in keeping it standardized so it's much simpler to just create tags for the most commonly blocked text content (slurs, cuuuuuuuuuuum, memes, etc)

Clawdragons' suggestion in that thread seemed like it would need wildcards in any case (in order to catch plurals and other variant forms of a word). For the example of cuuuuuum, cuuu*uum seems close enough to accurate.

Regexp would allow more compact formulation and less false positives, but isn't the amount of cases where that would actually make a difference pretty small?

SnowWolf said:
...."cuuum" was a joke :p

For several years "cuuuuuuum" was basically on every single page of a comic made by a very prolific artist group (Palcomix had about 6000 posts here before they were deleted)

Presumably, no one would blacklist ccuuuum. They'd blacklist slurs, or degrading words. Maybe someone enjoys content where characters are called sluts, but dislike the word whore. Or they can't take their scat seriously when someone says "poo" or "poop".

And while, yes, it WOULD be easier to create tags for those text ideas, we're discussing, here, a new feature on the website that currently doesn't exist.

I'm well aware of Palcomix's infamy, I was just using the same example you provided

I still believe that for the slur/common trigger words usecase, a "regular" tag should be used instead. There's quite a few of them, there's too much that can go "wrong" with the contents of the transcription field (user spelling differences, or the image itself could be obfuscating the words by writing stuff like "f*ck", "f_ck", etc)

For this particular case, it is too "important" and too much can go wrong with it that depending solely on transcript content wouldn't be effective. A good solution would be "aliasing" the transliterated content (transliteration:fuck implying profanity for example) so we can depend on user input, while having the transliteration content automatically tagging the most common cases

For the other usecases that you presented (Like blacklisting "poop"), it can be done (in the same way that we can blacklist/query for description text) but for the same reasons listed above it might not be a 100% effective solution. It can be implemented but it's usefulness could be easily compromised

savageorange said:
Regexp would allow more compact formulation and less false positives, but isn't the amount of cases where that would actually make a difference pretty small?

Oh, definitely

Everything we are talking about in this thread by definition is only applied to a fraction of the site's content

I'm not arguing against any of the ideas, I'm just presenting cases where the implementation might not work 100% as expected

Okay, time to be real about this.

It's a great idea, but I can't come up with a way to actually implement something like this that makes sense in the search system, because the way that the field behaves is not in line with how the rest of the search works. It looks like we want a full text search, but with multiple
words, which presents a lot of unique problems for a metatag. Would also have to deal with spacing and other mangling issues that the current search format enforces.

Oh, it's highly unlikely that it would be a field that you could blacklist, just because it would mean having to send the whole thing to the client at every turn.

KiraNoot said:
Okay, time to be real about this.

It's a great idea, but I can't come up with a way to actually implement something like this that makes sense in the search system, because the way that the field behaves is not in line with how the rest of the search works. It looks like we want a full text search, but with multiple
words, which presents a lot of unique problems for a metatag. Would also have to deal with spacing and other mangling issues that the current search format enforces.

Oh, it's highly unlikely that it would be a field that you could blacklist, just because it would mean having to send the whole thing to the client at every turn.

Are you sure? In what way would the system we were talking about be different from the one that we already have in the description field? (Aside from the blacklist thing, as I'm not sure if blacklisting works with descriptions either)

KiraNoot said:
Okay, time to be real about this.

It's a great idea, but I can't come up with a way to actually implement something like this that makes sense in the search system, because the way that the field behaves is not in line with how the rest of the search works. It looks like we want a full text search, but with multiple
words, which presents a lot of unique problems for a metatag. Would also have to deal with spacing and other mangling issues that the current search format enforces.

Oh, it's highly unlikely that it would be a field that you could blacklist, just because it would mean having to send the whole thing to the client at every turn.

Is it impossible to have %20 work as a maybe-placeholder for spaces in Source/Transcript field ("maybe" meaning it searches for both spaces and for actually %20)?

I had system similar to comment searching in mind, so if you want to search "hey kid, wanna yiff" meme but not sure if they used , you could wildcard it.
https://e621.net/comment/search?query=kid*wanna+yiff&results=exact

Mairo said:
I had system similar to comment searching in mind, so if you want to search "hey kid, wanna yiff" meme but not sure if they used , you could wildcard it.
https://e621.net/comment/search?query=kid*wanna+yiff&results=exact

Does the + work as a "single character" wildcard in the current system?

KiraNoot said:
Okay, time to be real about this.

It's a great idea, but I can't come up with a way to actually implement something like this that makes sense in the search system, because the way that the field behaves is not in line with how the rest of the search works. It looks like we want a full text search, but with multiple
words, which presents a lot of unique problems for a metatag. Would also have to deal with spacing and other mangling issues that the current search format enforces.

Like mabit said, we already have the description: and notes: metatags. Why is this any different?

But, anyway, transcript search doesn't necessarily have to integrate with tag search. A standalone transcript search feature, like we have for searching comments, would still go a long way.

Performance permitting, a two-stage "advanced search" would work, too. First find posts according to the normal tag search rules, and then filter those results further based on their transcript contents.

At least from what I've seen of image transcription things that have weird fonts, handwriting, cursive, a lot of clutter, and things like NMNY's avatar tend to give results that aren't great. A way for it to automatically flag possibly bad transcriptions for human review (maybe a filter that checks transcriptions for weird characters, misspellings, and nonsense phrases) would probably be helpful.

Easier implementation: Use the existing notes system, but add a transcribed or transcription tag, short for transcribed_english, to any images that have notes that are not for the purpose of correcting spelling errors or translating from another language. When the post loads, hide the notes by default if the tag is present. Clicking will show the notes.

Alternative: Use the notes system, but add a dropdown to the note creation box that lets you select a type for each note. Transcription, Translation, Correction, and any other types. When the post loads, all of the notes except for Transcription type notes will be visible. Clicking the image twice, once to hide the notes and another to show them again, will show all notes including the Transcription type.

regsmutt said:
At least from what I've seen of image transcription things that have weird fonts, handwriting, cursive, a lot of clutter, and things like NMNY's avatar tend to give results that aren't great. A way for it to automatically flag possibly bad transcriptions for human review (maybe a filter that checks transcriptions for weird characters, misspellings, and nonsense phrases) would probably be helpful.

This thread is (mostly) talking about user generated transcriptions, check up there

KiraNoot said:
I can't come up with a way to actually implement something like this

1. Add a new column "transcribed_text" to the "posts" datatable which contains exact transcribed text
2. Add a new inverted index on the "transcribed_text" column, which does word stemming and stuff to the words and puts them in the index.
3. Add a new search option text:"word_1 word_2 ... word_n" which searches this index and returns posts which contain all of searched words.

Delian said:
1. Add a new column "transcribed_text" to the "posts" datatable which contains exact transcribed text
2. Add a new inverted index on the "transcribed_text" column, which does word stemming and stuff to the words and puts them in the index.
3. Add a new search option text:"word_1 word_2 ... word_n" which searches this index and returns posts which contain all of searched words.

Yes, that's the short of it. :P

With all due respect, how many uploaders would actually be bothered with this when it's more than a couple of sentences? I mean, assuming you're not gonna incorporate some advanced symbbol-recognition program running through every single post that gets uploaded, most uploaders would have to this manually.

MyNameIsOver20charac said:
With all due respect, how many uploaders would actually be bothered with this when it's more than a couple of sentences? I mean, assuming you're not gonna incorporate some advanced symbbol-recognition program running through every single post that gets uploaded, most uploaders would have to this manually.

Much as with translations or tagging projects: a specific subsection that decides that This Matters To Them, and they decide to perform that task. *shrugs*

Furrin_Gok said:
Is it impossible to have %20 work as a maybe-placeholder for spaces in Source/Transcript field ("maybe" meaning it searches for both spaces and for actually %20)?

Unless I'm missing something, we probably won't want to search for the character _. So that's another possible placeholder.

Though ideally, I'd like the query parser to understand "this kind of" quoting, since that is a generalized solution which can apply to any 'reserved/special character' situation, not only spaces and not only transcript:.

+1, especially in coordination with the notes system. We would probably need both direct transcription and corrected transcription fields for example post #269569, most users will correct it in their heads and search for that.

<I just want to mainly propose a transcript feature for dialogue that way we can search for dialogue on e621>
This is what I was going to start in my thread
but I decided to look if this was already started.
thankfully after flipping through pages and pages of forums I found this.
I still think this should totally be a feature.
have the higher-ups looked at this yet?

I still like the suggestions I posted above. Give notes a type in the database, then use the notes system to transcribe. Hide transcription notes by default, show other types.

Automation of this work would be great, since there are hundreds of thousands of images with text in them.

I'm going to bump this.

Couple things has changed in these years, now we are on new codebase on the website and have gotten a lot more new features and improvements.
Another one is that from what I recall, e6AI got new field for posts to disclose prompts that were used to generate the images, basically meaning that introducing new input fields on posts shouldn't be that hard thing to implement now.

lance_armstrong said:
I still like the suggestions I posted above. Give notes a type in the database, then use the notes system to transcribe. Hide transcription notes by default, show other types.

Automation of this work would be great, since there are hundreds of thousands of images with text in them.

The problem with this is animations and especially webm and flash posts as these do not support notes and even if they do/did, there's data over time.
Altough I have been having this mental image of this data being submitted in .VTT format, so that it would just work directly and natively with the video files when they are pointed out to fetch the subtitle data from the field. Of course invalid data would just not show up at all.