Topic: Tool to convert parent chains to pools

Posted under e621 Tools and Applications

Long chains are quite annoying to fix, so I decided to create a simple tool to do it.

I don't want to associate my Github with e621, so I'll just post the code here:

Code
import json
import os.path
import requests as requests
from ratelimit import limits, sleep_and_retry


@sleep_and_retry
@limits(calls=1, period=1)
def api_limit():
    pass


def get_login() -> tuple[str, str]:
    if os.path.isfile("login.json"):
        with open("login.json", "r") as file:
            data = json.load(file)
            return data["username"], data["key"]
    username = input("Username: ")
    key = input("API key: ")
    if input("Save login (y/n): ") == "y":
        with open("login.json", "w") as file:
            json.dump({"username": username, "key": key}, file)
    return username, key


def get_page(session: requests.Session, post_id: str) -> dict:
    api_limit()
    response = session.get(f"https://e621.net/posts/{post_id}.json")
    if not response:
        print(f"Unable to get {post_id}: Status {response.status_code}, response {response.text}")
        exit(1)
    post = response.json()["post"]
    return {"id": post_id, "parent": post["relationships"]["parent_id"], "pools": post["pools"]}


def get_chain(session: requests.Session, start: str) -> list:
    print("Getting posts")
    post_id = start
    result = []
    while post_id is not None:
        result.append(get_page(session, post_id))
        print(f"Got {post_id}, pools: {result[-1]['pools']}")
        post_id = result[-1]["parent"]
    result.reverse()
    return result


def get_category():
    while True:
        response = input("category (series/collection): ")
        if response == "series" or response == "s":
            return "series"
        if response == "collection" or response == "c":
            return "collection"
        print("Invalid response")


def create_pool(session: requests.Session, chain: list):
    payload = dict()
    payload["pool[name]"] = input("name: ")
    payload["pool[description]"] = input("description: ")
    payload["pool[category]"] = get_category()
    payload["pool[is_active]"] = "true"
    payload["pool[post_ids]"] = " ".join([str(post["id"]) for post in chain])
    api_limit()
    response = session.post("https://e621.net/pools.json", data=payload)
    if response:
        print(f"Created pool: https://e621.net/pools/{response.json()['id']}")
    else:
        print(f"Failed to create pool: Status {response.status_code}, response {response.text}")


def remove_parents(session: requests.Session, chain: list):
    for post in chain[1:]:
        payload = dict()
        payload["post[parent_id]"] = ""
        payload["post[old_parent_id]"] = str(post["parent"])
        payload["post[edit_reason]"] = "Convert chain to pool (automatic update)"
        api_limit()
        response = session.patch(f"https://e621.net/posts/{post['id']}.json", data=payload)
        if response:
            print(f"Removed parent from post {post['id']}")
        else:
            print(f"Failed to remove parent from post {post['id']}: Status {response.status_code}, response {response.text}")


def main():
    login = get_login()
    start = input("ID of last post in chain: ")
    with requests.Session() as session:
        session.auth = login
        session.headers.update({"User-Agent": "ChainToPool/1.0 (by SCTH)"})
        chain = get_chain(session, start)
        if input("Create pool (y/n): ") == "y":
            create_pool(session, chain)
        if input("Remove parents from chain (y/n): ") == "y":
            remove_parents(session, chain)


if __name__ == '__main__':
    main()

I don't use Python much, but it is quite convenient for small things like this.

Note that you can't create more than two pools per hour, unless you're a janitor. Main use is just removing parents after a pool has been created.

Found a very odd post tree, starting at post #804034. Seems to be made up of multiple pools on IB, though potentially all one commission (or at least same commissioner)
Any suggestions what to do with it?

List of all the posts in it
804034
└── 805199
    └── 810075
        ├── 820373
        ├── 823457
        │   └── 826198
        │       ├── 836150
        │       │   └── 837626
        │       │       └── 841256
        │       │           └── 843983
        │       │               └── 848483
        │       │                   └── 859762
        │       │                       └── 861821
        │       │                           └── 863266
        │       │                               └── 864734
        │       └── 880351
        │           └── 885483
        │               ├── 895391
        │               └── 897532
        │                   ├── 901080
        │                   └── 905829
        │                       ├── 906744
        │                       │   └── 917588
        │                       │       └── 922286
        │                       └── 910073
        │                           └── 915656
        │                               └── 937622
        │                                   └── 947714
        │                                       └── 980574
        │                                           └── 1221162
        │                                               └── 1223126
        │                                                   └── 1286083
        └── 843166
            └── 894438
                └── 898088

(Updated as I was missing a branch. Might have missed others too)

Updated

kora_viridian said:
I have no idea if this is the "right" way to do it, and I haven't looked at all the posts, but if this really is all one story that branches like that, then...

It's definitely not all one story, and looking a bit more closely it's not even all one commissioner like I originally thought. All they have in common is the artist, and mostly the same set of characters.
They should be split up into pools, but there's enough separate parts that a janitor should probably do it (members can only create 2 pools per hour).

scth said:
Found a very odd post tree, starting at post #804034. Seems to be made up of multiple pools on IB, though potentially all one commission (or at least same commissioner)
Any suggestions what to do with it?

List of all the posts in it
804034
└── 805199
    └── 810075
        ├── 820373
        ├── 823457
        │   └── 826198
        │       ├── 836150
        │       │   └── 837626
        │       │       └── 841256
        │       │           └── 843983
        │       │               └── 848483
        │       │                   └── 859762
        │       │                       └── 861821
        │       │                           └── 863266
        │       │                               └── 864734
        │       └── 880351
        │           └── 885483
        │               ├── 895391
        │               └── 897532
        │                   ├── 901080
        │                   └── 905829
        │                       ├── 906744
        │                       │   └── 917588
        │                       │       └── 922286
        │                       └── 910073
        │                           └── 915656
        │                               └── 937622
        │                                   └── 947714
        │                                       └── 980574
        │                                           └── 1221162
        │                                               └── 1223126
        │                                                   └── 1286083
        └── 843166
            └── 894438
                └── 898088

(Updated as I was missing a branch. Might have missed others too)

Derp.

smudge_proof said:
Derp.

This sounds like a case where a "collection" style pool is the solution, with parent/child relationships remaining only for directly related images.

I've been told in the past parent-child relationships shouldn't have more than three or four max images in most cases, and pools are the answer when things get larger than that, and they should be set as siblings to a single parent rather than being daisy-chained.

The topic here leaves unstated the standard of not daisy chaining parent-child relationships, which is what the tool is about.

The only potential issue of attaching things as siblings to one parent is the order of them is at the mercy of when they were uploaded. Pools solve that though.

  • 1