This is a thread for those interested in downloading NSFW content for local consumption.Here we share tools, scripts and tips on how to download the content and organize it. >How do i find the best sites?https://everythingmoe.com/ (click on NSFW)Some tools:>gallery-dlhttps://github.com/mikf/gallery-dl>Grabberhttps://github.com/Bionus/imgbrd-grabber>yt-dlphttps://github.com/yt-dlp/yt-dlp>why not make this thread on /g/?/g/ had a series thread named "coom/g/". But the janitors delete any attempt on making this now.I start by asking a question: Mangayomi or Suwayomi for desktop tachiyomi-based reader?
>>8754302Anything for mass conversion of webp into civilized formats?
>>8754570it's trivial to script tho
>>8754302Does anyone know of a way to grab Irodori's third-party titles for offline viewing? It's for those doujins/manga that requires you to view it on their website instead of downloading a DRM-free copy.>>8754570You can probably do it with ImageMagick. I know webp is a supported format so it shouldn't be an issue.
Gotta plug Hydrus Network here, probably the best scraping and tagging solution for coom (posting image to keep it /h/)https://hydrusnetwork.github.io/hydrus/index.html
>>8754570Xnconvert if you want a GUIA script that uses FFMPEG or ImageMagick if you want a CLI
>>8754653>IrodoriIf gallery-dl doesn't have it, you're unlikely to find it anywhere else. You have to write a scrapper for a unsupported site. I used gemini-cli to write a scrapper. It smashes through python.
>>8754302I've been using Suwayomi. You can also use Suwayomi as a server so you can connect it to Mihon/Tachiyomi on your phone. Also if you don't like the built-in Suwayomi frontend then you can use others.
>>8755110Is there a difference between using Gemini-CLI versus just using Gemini straight from Google's website?
>>8755490Gemini-CLI is a agent. It runs on the console and it's better for programming. You get 1000 daily requests (i hardly go over 100 in 1-2 hours).>create a folder>copy some examples/code/repositories it can use as reference to that folder>write a GEMINI.MD file with general instructions>run and tell it to follow GEMINI file instructions
Ever since bakabt i haven't downloaded hentai whats are the best options these days
I want to download stuff from the following sites:Rule34xxxDeviantArtKemonoHitomi.laEx-HentaiPixivSince Rule34xxx is a booru, i can just use Hydrus Network. But what about the others?Must contain the following:>Process a Url list one by oneSo i can just keep a list of URLs and run a script to fetch them.>Skip already downloaded files/galleries by checking a sqlite dbA sqlite db that stores the metadata from the galleries/files, such as tags, artists, etc. So that i could use it to rename the folders/files later by using a script.>Stop fetching more if it skips X amount of galleries/filese.g. if i search a tag in Hitomi.la, and it downloads thousands of pages, next time it process that i don't want it to go through thousands of pages again.>A GUISomething like hydrus that keep track of tags + thumbnails.>Customizable folder/file naminge.g. [Artist]/ [Artist] (Release Date) Gallery Title [Language] (Site Gallery ID)/ [Artist] (Release Date) Gallery Title [Language] (Site Gallery ID) (1 of 20).jpg [Artist] (Release Date) Gallery Title [Language] (Site Gallery ID) (2 of 20).jpg I was thinking about Suwayomi for Hitomi.la, Ex-Hentai and Kemono. It has a GUI with thumbnails and tags, and i can read it from PC and phone. But it would be a pain to fetch new content since i would need to add it manually.
>>8755989sukebei.nyaa.si
>>8755477I came to the same conclusion after using both. Suwayomi came out on top by ease of set up and usage.
>>8754302https://github.com/ififfy/flipflip
>>8755994After some researching and asking around i found that gallery-dl is the perfect tool for the job:>skips already downloaded files>dumps json with the metadata (no need to download the image/video files, it can skip them)>aborts fetching if it skips X itemsI just need to download the metadata, convert it into a sqlite database with a script, then make another script to query the database and build the gallery-dl command for downloading. Then make another script to create a Suwayomi/Tachiyomi xml file for each gallery, and now trere's a UI to read things from! It's more complicated than just downloading, but being able to use a database with sql queries is worth it. I post the scripts here later.
>>8755928I tried that but eventually it figured out the site was blocked by Cloudflare and it couldn't login to my account. It straight up just said it doesn't know how to bypass it. Well, I guess we're SOL until someone more competent takes over. Thanks for the help anyway! Didn't know such a thing existed and it's pretty cool.
Do you guys know any slideshow software or something that can help me pull out images at random?I don't have so few images saved that I could just remember every single one of them, so I'd like some way of surprising myself with thing I've forgotten.
>>8757460 >>8756846
>>87570712 things you can do:1. Tell it to make the scrapper using selenium in python, and to use cookies to try to bypass cloudflare.2. Tell it to make a violentmonkey script to scrape the current page you're viewing. Since you're logged in and can view the content, it should be easier. You may need to dump the html files for that page (right click > save page as..) in the project directory, so that it may use them as a example.
>>8756880gallery-dl saves metadata in a retarded way because it re-downloads the same metadata.
>>8757522Damn, it turns out clicking random 4chan links with no description is actually pretty helpful.
i need something to download all my pixiv bookmakrs, cuz of the threat of the recent changescan anyone help?
>>8756846thanks
>>8755989>bakabtthat's a name i haven't heard in a long time
>>8754570https://github.com/JacobDev1/xl-converteryou can obviously also save size by using lossless jpeg xl with this
tks for the links Do you know where I can find mirrors from vip and secrethentai sites ? Went to the everthingmoe and they don't list themThey were taken down a while ago and they don't seem to have any more valid sitesOr at least any site that mirrored or dumped their older posts Pure L hentai has newer stuff, but the older stuff that appeared only on those sites that vanished and could not find any real sites Some crap domains with terms like that only open scam ads and dating shitTks
>>8759195Gallery-dl
>>8762125>secrethentaiDo you mean ex-hentai? Because that's where everything that gets pruned from e-hentai goes. There's also hitomi.la, which just clones whatever is posted on ex-hentai.
>>8762429Actually not from those, they used to look like this Found some stuff similar but it was a site with paid password and crap/less content and the other one had some premium file hosting stuff, but mostly posts had a lot of images They were like that
>>8754791thanks
.
>>8754570Brother this is so trivial that even an LLM can get it right on the first try.
I'm using HDoujin DL, and I managed to get it to work for sadpanda, but for some reason it won't download from Mangadex or any site like that.What am I doing wrong?
>>8754302>GrabberIt's a shame Rule34xxx stopped working with Grabber recently
>>8764797Must be a bug. Make a bug report or use something else like Suwayomi.
...
>>8766543>recentlyIt has been this way for months now. The tag database is broken and the faggot bionus is too lazy to generate a api key to get it working again.
>>8768379>>8770690>nobody actually uses this thread
I appreciate this thread.
>>8754302hey if anyone is interested in hosting their collection locally check out https://github.com/stashapp/stash
>>8754570Imagemagick is the way
Just want to shill Lanraragi here as I can't find anything even remotely close for local doujin management.
Any AI that can select/move/sort images based on my tastes? It will take a long time for me to sort stuff i got online. It would be great if there was something that i could "feed" the things i sorted already, and it would go through the unsorted things, and move the ones that fit my tastes based on what i "fed" it.>but what about tags?A image having a tag i like doesn't automatically mean it's good.
>>8754302https://github.com/AldebaraanMKII/Gallery-Downloader-Scripts
>>8772710Any chance you use hentoid and know how to import the tags through its .json file to lanraragi?
>>8754302https://theindex.moe/https://wotaku.wiki/
>>8776046I use the jsons thathttps://github.com/ccloli/E-Hentai-Downloaderadds to archives if you check the box.Might have to look into Hentag.
>>8777310I have thousands of doujins from diff sources so i doubt hentag will help me, hope this "lanraragi connector" is a solution to my prob but i have no idea what it even doeshttps://github.com/avluis/Hentoid/issues/1247
>>8754791Never managed to make it work, always dying on me or just not working at all.
>>8754791>hydrusOnly good for random images webms and memes that you can't effective categorize. It's horrible for archives, because of it's main feature which I still think is the dumbest idea ever.It doesn't respect your folder structure and keeps track of everything in it's bloated database which is extremely prone to breaking whenever you try to update to the next version. Whatever you feed it it'll break it apart and save it randomly in it's own data structure. Lose your db, your files are scattered like the wind.>inb4 but you can exportFuck off with that extra step.Good for memes and single one off images. With manga and other stuff you're better off with lanraragi.>tagging is greatThe PTR is fucking bloated and tag overload is a real thing too many useless tags might as well not tag at all, and enjoy spending days trying to sync up from scratch.It could have been so much more but it's bloated trash imo.
>>8779147I have issues with hydrus to the point where I've decided to make my own alternative, but you've somehow picked the stupidest and easiest problems to solve.>Lose your db, your files are scattered like the wind.Just keep a backup. If you've got enough files that you need software to maintain it, then you should have enough space to keep a backup of the DB.>The PTR is fucking bloatedIt is, Just don't use it then. It's not that hard
>>8779147>Extremely Prone to Breaking.Its SQLite, one of the most stable DB options out there. Extremely Prone to Breaking? just no, you must have a failing drive for that or failing ram. You can kill off Hydrus in mid write without issues, you can kill power. you can really abuse your client without breaking the db. but nothing can help you if your hardware has issues... but then you have bigger issues.But i give you this, Hydrus is not for Comics, Manga or Doujinshi. Use something like Comicrack for that.... along with Lanraragi for Doujinshi, Ubooquiti or Kavita for Comics for web reading perhaps.
>>8755989Damn friend, I don't miss the super strict upload ratios, while I was fighting the good fight with old internet. Gotta admit I do miss the absolutely amazing selection and having them all be seeded was heaven.
>>8759134Fucking lol nice
>>8779147Only tags that matter are Spats, Tomboy, Incest. Fight me. Also I usually throw in tankoubon so it darts straight to completed works.
