I've been coding a simple pool downloader in C#, getting pool json works fine but when fetching the individual post jsons all the fields are null.

I've seen other threads with similar questions but I don't think I need to use my API key just to get a direct link and download it.

Below is a code snippet of when it attempts to download the picture:

client.Headers.Clear();
client.Headers.Add("user-agent", "PoolDownloaderNET/0.01 (by NotVila on e621)");

string jsonFile = client.DownloadString($"https://e621.net/posts/{postIDs[i]}.json");

var File = JsonSerializer.Deserialize<File>(jsonFile);
client.DownloadFile($"{posts.url}", $"./{postID}.{posts.ext}");

Below is a snippet of the fields it fetches from the json

internal class File
{
public string url { get; set; }
public string ext { get; set; }
}

I've been trying to figure how to make this work for the past couple of days, thanks in advance :)

It means the images are on global blacklist, you need to authenticate to get the urls in that case.

aobird said:
It means the images are on global blacklist, you need to authenticate to get the urls in that case.

Tried it on different pools and even made sure to check if they where blacklisted and it still returns null, with blacklisted pools it crashes when trying to fetch the pool json.

notvila said:
Tried it on different pools and even made sure to check if they where blacklisted and it still returns null, with blacklisted pools it crashes when trying to fetch the pool json.

All fields are null? Are you respecting the rate limit? You should use https://e621.net/posts.json?tags=id:{PostIDs} when getting multiple images.

Or you could search by pool like https://e621.net/posts.json?tags=pool:23456

aobird said:
All fields are null?

All fields are null: https://i.ibb.co/KwVKWnD/Untitled.png
Fetching the pool json works fine though: https://i.ibb.co/hYM1L5P/Untitled2.png

aobird said:
Are you respecting the rate limit?

I know the snipped I included didn't show any pauses but I did read through the API documentation beforehand and I have already included a "Thread.Sleep(1000);" (1 second pause)

aobird said:
You should use https://e621.net/posts.json?tags=id:{PostIDs} when getting multiple images.

Or you could search by pool like https://e621.net/posts.json?tags=pool:23456

It seems that https://e621.net/posts.json?tags=id:{PostIDs} and https://e621.net/posts/{PostIDs}.json return the exact same response, I'll try the former and see if it works.

EDIT: tried the variant link above for the post json but it still doesn't work, the json seems to be fetced correctly but the url is still null: https://i.ibb.co/Sfcs2wR/Untitled3.png

kora_viridian said:
I don't know a lot of C#, but I feel like there's something missing between these two lines:

4 var File = JsonSerializer.Deserialize<File>(jsonFile);
5 client.DownloadFile($"{posts.url}", $"./{postID}.{posts.ext}");

Line 4 is trying to deserialize the JSON string you got from the first download, and store the results in File, which is OK.

However, I feel like you need to refer to File again somehow when you try to do the second download on line 5. I don't think your code will know what posts.url or posts.ext are without some reference to File. I'm not sure of the exact syntax, but you probably need to say something like File->posts.url and File->posts.ext instead.

Yeah that was an error from my part, sorry. I hastily made this thread without realising the mistake in the code, you can find a correct snippet below:

client.Headers.Clear();
client.Headers.Add("user-agent", "PoolDownloaderNET/0.01 (by NotVila on e621)");

string jsonFile = client.DownloadString($"https://e621.net/posts.json?tags=id:{postIDs[i]}"); //Get json file from the post id array in the pool json

var FileUrl = JsonSerializer.Deserialize<File>(jsonFile); //Deserialize json as "FileUrl"

Console.WriteLine($"https://e621.net/posts.json?tags=id:{postIDs[i]}");

client.DownloadFile($"{FileUrl.url}", $"{num}.{FileUrl.ext}"); //Download file url and save as post number
num++;

Thread.Sleep(1000);

In the previous code I forgot to change the "posts" to "File" among other things.

kora_viridian said:
2) it implies some fairly scary things about how C# handles JSON. I can only make suggestions about 1), though. :D

C# can be very crappy sometimes, a lot of times everything you do seems to not work for no apparent reason, I guess I should've learned java instead :/

kora_viridian said:
Finally, I'm pretty sure this isn't the problem, but once you do the client.Headers.Add, the added header persists over all the calls you make with that client, right?

Yeah the header is persistent as I apply it every time I fetch a post json.

Thanks to both of you for being so helpful, never thought I'd get replies so fast! :)
(I would've most likely been downvoted to oblivion and gotten not so helpful replies on stack overflow :P)

When deserializing JSON, you need to follow the model of the response exactly. The file info is inside of the post info, so you can't just use only File expecting the parser to know it where to look for it.

There is also a difference in the responses between using posts/{id}.json and posts.json?tags?id={id}. The former will return a single post ( "post":{...} ), while the latter will return a list of posts ( "posts": [ ... ]).

As mentioned before, you can also use the tag version to get multiple posts at once by giving it a comma separated list of post ids (up to 100 I believe). You may need to set the limit in the query string if you do want to get 100 posts at once, the default may be lower than that. Alternatively, use page in the query string until you get everything. Doing it this way will require much fewer api calls.

In any case, try deserializing to this object.

public class PostResponse
{
	public class File
	{
		public string url { get; set; }
		public string ext { get; set; }
	}
	
	public class Post
	{
		public File file { get; set; } = new File();
	}
	
	// posts/{id}.json 
	public Post post { get; set; } = new Post();
	
	// posts.json?tags=id:{ids}
	public List<Post> posts { get; set; } = new List<Post>();
	
	
}

And use it this way

var response = JsonSerializer.Deserialize<PostResponse>(jsonFile); 

// for posts/{id}.json
var file = response.post.file;

// for posts.json?tags=id:{ids}
foreach(var post in response.posts)
{
	var file = post.file;
}

You are not deserializing properly I would guess, https://i.ibb.co/Sfcs2wR/Untitled3.png shows some data, but you have a 0 there.
Look at the json response first then at your deserialization object.