Topic: Idem's Sourcing Suite - A collection of tools to make sourcing easier

Posted under e621 Tools and Applications

I'm back at it again, with another project to throw to the wind, and have nothing happen.

Idem's Sourcing Suite

Install

Examples Currently not accurate

Github

You must have some user script manager. Greasy Fork has a good list. (I use tampermonkey)

What is this?

This is a bundle of tools designed to make sourcing posts and images easy for e621. There are three main parts:

  • on-site plans
  • image-data comparison
  • post BVASer Currently broke

on-site plans

The on-site plans include utilities for uploading and verifying the source of post. It includes three parts: a convenient link to upload an image to e621, an automatic image hasher that checks if an image was uploaded to e621, and a button to copy a posts' description.

image-data comparison

The image-data comparison tool is something that could be used to find the subtle differences between image files. It is most useful for checking if there are unnoticeable compression artifacts in an image.

post BVASer Currently broken

The post BVASer is a recreation of a tool originally made by Munkelzahn. It will replace a post on e621 with a new image that has superior quality. It will copy all information from the old post into a new one.

Why doesn't it support site X?

Send me a suggestion, either on github or on e621 and I will probably add it.

I have a great idea!

Create an issue, or even write the code yourself. I like great ideas and I like making great things.

What about these permissions? Do you really need send requests to ANY URL?

No I don't, and I wish I didn't need those permissions. I feel the less you need to trust me the better. The only reason I request access to any site for this extension is so the image comparison and post BVASer can function. This is because your web browser is concerned about XSS attacks (and it should be!) so it does not allow me to get values from the canvas I draw the images on, and it does not let me get the md5sum of those images.

I really don't like the way you do X

That's a suggestion. See above.

Who actually uses this?

I do. Mairo does (Thanks, blip #90217). And now hopefully more people.

Updated

Reserved. Is there a post length limit?

Admin edit:
50,000 characters

Admin edit:
pett

Updated by anonymous

I run a private Shimmie booru that I'd love to be able to quickly look MD5 Hashsums up quickly. Does your on-site-md5-hasher work on Shimmie boorus, and if so, how do I enable?

Updated by anonymous

Update:

  • Code is more maintainable
  • Pixiv now works again
  • Added a copy description button on sites to copy the description
  • BVAS bug fixes and facelift

It might not look like a lot, but this was a big update and took some time to do.

21-Shades-of-Faux-Pa said:
I run a private Shimmie booru that I'd love to be able to quickly look MD5 Hashsums up quickly. Does your on-site-md5-hasher work on Shimmie boorus, and if so, how do I enable?

After communicating over discord we had discovered that this feature was not required. If in the future someone still wants this. I can implement it.

Updated by anonymous

I really enjoy your bvas tool, but is there anyway for me yo upload files directly from my system instead of providing a url?

Updated by anonymous

crusty_fire said:
I really enjoy your bvas tool, but is there anyway for me yo upload files directly from my system instead of providing a url?

Thank you. I'm not sure if there is a way to do this through a userscript, but I will look into it.

Also I know that version 1.00013 does not work on Firefox and made a quick change to attempt to fix that (1.00014). I haven't tested that change, but I should be able to in about five hours.

Updated by anonymous

Do you compare that MD5 to e6? I was thinking you should show like the same text, but in red color if it's already uploaded. Maybe green for OK.

Updated by anonymous

AoBird said:
Do you compare that MD5 to e6? I was thinking you should show like the same text, but in red color if it's already uploaded. Maybe green for OK.

It does not compare the md5. I figured if there was a collision e621 would throw an error.

Changing the color based on the md5 existing on e621 is a good idea though. I will try and put that in the next version.

Updated by anonymous

Idem said:
It does not compare the md5. I figured if there was a collision e621 would throw an error.

Changing the color based on the md5 existing on e621 is a good idea though. I will try and put that in the next version.

It will, but why waste time trying when you can check beforehand.

Updated by anonymous

v1.00017

  • Added red text to the new image md5 if image is already uploaded to e621.
  • Added an option for direct image uploads
  • Fixed the delete option. Now will attempt to delete the post (only works if you have permission to delete posts)
  • Fixed Copy Description on inkbunny pages

I accidentally uploaded the previous version (1.00014) with the name of my local testing script. I have no idea how this will effect upgrading the script.

At some point in the future, I will create an update to the original post and the github with updated pictures/videos on how the tools function.

Updated by anonymous

Idem said:

Post BVASer

This is a reinterpretation of forum #87369, a tool originally made by Munkelzahn. I had received some suggestions for how to improve this tool, and so I added those. On this page you can input a currently known e621 post, the data from that post is then loaded. After you can supply a new image as well as editing tags, sources, and other things. Finally you can upload this post, flag the old one as inferior and do a bunch of things automatically. This tool has worked well for the about ten uploads I have used it for. There are some settings, but you must edit the script for these.

I've just used it for the first time, and it works great.
Had to install Tampermonkey, though.
(bug report on Github)

Updated by anonymous

Some general feedback after using this script for a couple weeks:

Overall, this is a good quality of life improvement. I chiefly use the image_compare functionality for Boolean comparison between images of the same resolution. The upload_bvas page is a welcome addition, but difficult to merge into my old workflow due to established muscle memory.

For more specific feedback, I have filed several GitHub issues pertaining to user experience. Thanks for all the hard work!

Updated by anonymous

Strongbird said:
Some general feedback after using this script for a couple weeks:

Overall, this is a good quality of life improvement. I chiefly use the image_compare functionality for Boolean comparison between images of the same resolution. The upload_bvas page is a welcome addition, but difficult to merge into my old workflow due to established muscle memory.

For more specific feedback, I have filed several GitHub issues pertaining to user experience. Thanks for all the hard work!

Thanks for the feedback. Github is the best place for logging issues and feedback because then I can not forget them.

Something I've been thinking about is switching this over to a browser extension instead of a userscript. I've had a little experience with writing extensions, but I don't want to make another because it feels too grand for what I'm making. Though a extensions would allow for a better development environment.

I've got so many projects and many ideas for more. I fear I've spread myself thin.

Updated by anonymous

Well I wasn't really hoping to push this live now, but the old one is broke so I don't have much of a choice.

Hello! I've been trying to work on this in the past and make it better, real life got in the way of devoting myself to coding for a furry site full time and I had to dial back on what I did.

The script is now at v30 (because I said so) and is updated on github, the old links will no longer point to the correct place to auto update (sorry) so hopefully you find this forum. Basically the old version became increasingly difficult to maintain which necessitated this rewrite. All the core functionality is there except for the BVAS tool which I know a lot of you really enjoyed (I'm sorry mairo, I'm working on it). The reason fixing the BVAS is going to take a while is because I have to update my code to this new API just like everyone else.

One thing that people kept asking for is a page for a list of the extensions. You can visit here (won't work if you don't have the script installed) and view a list of all the pages and other things. Also a note to developers, go ahead and use the file I've stashed away in there if you want your stuff to appear on this page also. I'd hope for at least a little uniformity, but I'm not the best person to bring it about.

We'll just wanted to say that this tool is updated and should work (mostly) the same way as it has in the past.

Hi Idem!

I hope you're still working on ISS, because I have a feature request ^-^

ISS inserts an "Upload to e621" link on every FA view page, like https://www.furaffinity.net/view/45134129/
But the description on this FA view page is too long, so clicking on "Upload to e621" leads to an e621 error page:
414 Request-URI Too Large

This can't be fixed: https://e621.net/forum_topics/31783?page=1#forum_post_323866

So I'd like to suggest that ISS also add an "Upload to e621 without description" link to every FA view page.
I can then click the "Copy Description" button and manually paste the description into the e621 upload form.

Or:
There's only an "Upload to e621" link.
But ISS checks if the URL length is ok.
If the URL is too long, it automatically copies the description to the clipboard, and displays a warning message via notification.

Updated

I'm using this now, and I've found it quite useful

I've gone and forked it and fixed that most recently mentioned issue, if you want to fork it yourself and add in a fix (as I've changed a few other things as well)

It checks for links >8000 (as mentioned, the limit is 8192 but it's better safe than sorry) and the strips the description if present, then the tags if present and the link is still more than 8000 characters

Deviantart doesn't seem to be working right now or in the past few days. I tried seeing if it was on my end and I think DA changed something that messed with this.

millcore said:
Deviantart doesn't seem to be working right now or in the past few days. I tried seeing if it was on my end and I think DA changed something that messed with this.

DA hasn't been working for me for weeks now, either. Considering that it's also been a long time since this was updated, I don't believe this will ever be fixed.

kora_viridian said:
I dinked with it a little bit just now, but I'm running into an error that I think is happening because I'm not logged in to dA. (I don't even have a dA account.) I think the error is happening when ISS tries to collect all the metadata on the dA post, and dA says no, because I'm not logged in.

I never tried to use ISS before it broke, though, so maybe this is the same behavior ISS has always had when you weren't logged in to dA.

No, I'm logged into a DA account, and it's still not working for me there. I even tried logging out and clearing cookies before logging back in once before. No dice.

@kora_viridian oh my gosh thank you so much!

If anyone finds this forum after those links have expired I'll be saving them so poke me xD
It's just so useful to know almost right away is an image is on e6 or not in galleries!

I think the sourcing suite is as good as dead now for anything other than easy uploading; the recent switch to Cloudflare is preventing the MD5 check from working properly.

alien_fluff said:
No, I'm logged into a DA account, and it's still not working for me there. I even tried logging out and clearing cookies before logging back in once before. No dice.

same for me here, not working on DA

Hmm, was interested in new tools like this. I should probably write one that lets me archive sources easier.

Twitter has recently changed it's domains to x.com/* so the script is now broken for twitter

Is there an alternative to this? Trying to learn how to fix it and/or add new sites is a daunting (and possibly not practical) task. But this tool has been essential to me.

I believe Twitter/X further changed the URLs so just going to the twitter.com version of a post (which is still there) doesn't work anymore.

  • 1