Well hello there :3
I've been searching for something to download posts based on tags, but since I didn't find anything, I just coded one myself!

It's coded in Java, so that's definetly recquired.
It takes some time to gather the posts' ids, based on how many pages you want to download because it gathers them from the HTML code.

The programm automatically generates a thread for each of your CPU cores, optimizing the speed.

If you encounter any bugs, please tell me!

Screenshot

UPDATE v.1.1:

Download HERE

~NeBuR

You don't need to do make it core aware, because every modern core should be able to treat multiple threads/downloads at once.

filesDledD should be either have a synchronized setter or rather use java.util.concurrent.atomic.AtomicInteger instead of a normal Integer.

This test class will quickly show you what happens if you don't synchronize:
http://pastebin.com/ALs1SRtQ

Why not use the API instead of scraping the HTML?

I'm sure I know a few of these words...

Rainbow_Dash said:
I'm sure I know a few of these words...

Nope! It aint me! Like wyvrn said, I would use the Pool API (which is what I use on the script)

I can PM you the fragment that does that, if you like.

So, if anyone could help me figure out how to work with the API, I'd be grateful.

https://e621.net/post/index.xml?tags=mlp

only gives me 45 posts.

https://e621.net/post/index.xml?tags=mlp?page=2

gives me nothing and

https://e621.net/post/index.xml?page=2?tags=mlp?page=2

gives me every post ever with an offset of 45 :I
Is there any way to combine these two requests?

Or is there a workaround?

Parsing the HTML seems to be the only way.

Thanks in advance, NeBuR

EDIT: I found a major flaw in my coding: I didn't realize that the post's static-file-url was embedded in the index's HTML code. Currently I am changing the code to accomadate this.

You need to separate your parameters with & not ?

https://e621.net/post/index.xml?tags=mlp&page=2

Gives you what you're looking for.

Wyvrn said:
You need to separate your parameters with & not ?

https://e621.net/post/index.xml?tags=mlp&page=2

Gives you what you're looking for.

Ah, thanks. Back to changing the code from HTML to XML, I guess.

Updated!

So I'm trying to use it right now and it systematically finds 0 post no matter what tags I use.
It's weird, it worked like a charm when I used it a month ago.

Phew guys, it's been a loooooong time. I've finally gotten around to update my downloader!

UPDATE v1.2

Yup, so that's about it.

Grab it here:
MEGA
MediaFire

Feedback is appreciated!

Also, I'm terribly sorry for my lack of updates or rather of even a single update.
I didn't even know people used my tool.

~NeBuR

my client can do that since its there lol

I'm shocked you didn't come across my downloader.

I know this is old now, buuuut....

https://static1.e621.net/data/a1/1f/a11f6306c142ba69d261788d993475fc.jpg
59
5
<?xml version
"1.0" encoding
"UTF-8"?><posts count
"4390" offset
"4425"> </posts>
4390
Added a file to Thread 1
Added a file to Thread 2
Added a file to Thread 3
Exception in thread "AWT-EventQueue-0" java.lang.IndexOutOfBoundsException: Index: 4, Size: 4
        at java.util.LinkedList.checkElementIndex(Unknown Source)
        at java.util.LinkedList.get(Unknown Source)
        at e621downloader.Gui.startButtonActionPerformed(Gui.java:329)
        at e621downloader.Gui.access$2(Gui.java:253)
        at e621downloader.Gui$3.actionPerformed(Gui.java:124)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$500(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)

I am unable to download it. Is it because I'm on a Mac?

Lite3000 said:
I am unable to download it. Is it because I'm on a Mac?

I'm on a Mac and it downloads fine for me in Safari/Chrome/Firefox. When I run it it gives me this, which tells you what you need to run it.

It's possible that an antivirus program or the like is blocking it.

Edit: I didn't realize you meant after running the program. I wouldn't be surprised if the API changed since this was released, so I doubt it will work without an update.

For me it gets to being able to download and saying "Beginning download..." but it doesn't actually download, I have it excluded by my AV and allowed in firewall but it just won't download from the website

Sadly it doesn't work for me, i'm running Windows 10 64-bit

Same problem as Zaryss, i get "Beginning download" and then nothing, no CPU activity whatsoever from the program

Doesn't work for me on Windows 10 either, nothing happens.

I'm going to try using it on a linux partition to see if it's a Windows 10 bug

Edit: Yeah it's not working here, either.
Might be because of an API change they said they did.

not working for me either, windows 7 64-bit, checked fire walls and all that

Azarath~Azzy said:

Oops

The program launches and all and i give it simple tags such as "sergal" or "aaron_(artist)" and it returns with:

"Fetching posts from e621...
Total of 0 posts found.
Beginning download..."

'beginning download'
and then it just sits there forever and does nothing

win7-64bit latest version of java at the time comment is made

edit: oops i just saw the update lower in the thread. sorry!

Well, I got the lowest update in the thread and i put some simple tags in as before.
when i press 'go' it says

"Fetching posts from e621..."

and then just sits there doing nothing other than taking up RAM on my computer
i thought of pretty much everything that could have been stopping it
-odd characters in tags
-too many tags
-my firewall
-weird page counts
-pretty much everything

nothing seems to work. the weird thing is that it doesn't crash and it never says "program is not responding" like any other would in a situation like this

( halp plz? :( )

Probably Queuing images because the creator didn't use multi threads, execution services, or any from of concurrency.