Sup /g/Just released the first version of naiad-net, a hobby project that I ended up pushing into a usable state. The tl;dr of it is you add picture folders to it, it indexes them into a sqlite db, and you can tag shit.But I also wanted what the Hydrus network has, in a more lightweight package. So I have a dev server that ingests hydrus ptr updates every night. This way you can also pull tags from the net, in a quicker and only slightly less private way. Basically instead of downloading the entire db, you request buckets via hash prefixes. It's pretty fast this way.Shit's made in rust, optimized for speed, interface is wip but imo right now the basic functionality of 'just put tags on my images' works. It's completely slopped together, and I'm actually surprised I got this far. If anyone cares enough I'll keep adding planned features, like plugins to scrape/autotag via model, but right now only basic functionality is done.
>>109593708tried hydrus once and uninstalled the turd 10m later. explain how your fecal matter is different and what it does better than windows explorer.
>>109593708what gui library is used?
>>109594835Claude webshit
>>109593708A thread died for this. Fuck you.
>>109594825You unzip it, add an image folder, and 5 minutes later you've got hydrus tags for them.>>109594835Svelte>>109594947Yea
Oh right, forgot to put up the github. https://github.com/scoopscoop/naiad-net
>>109594825>filtered by autismo softwarelol
>>109595046>Shit's made in rust, optimized for speed>Sveltelol
>>109595285gave it a 10 seconds look>uses "jpeg_decoder" for thumbnails when "image" with jpeg decoding is in the dependency tree>(picky) uses sync-only ureq in the client when async is already used in the workspaceis this the power of trillion dollar intelligence? lol.a funnier game for someone with more time, is to figure out what parts were "borrowed" from where.
>>109595448It can filter through a 90k image library in seconds, and pull just as many hashes from a repo in minutes, that's good enough for me.
>>109595582i made that comment before seeing you posted the repo.>>109595482 is may last comment.
>>109595448Svelte is not what's doing the actual numbers crunching in this sort of programs, it's just gui library.>>109595582>It can filter through a 90k image library in secondsThat's really bad to be honest. People use Hydrus for collections with tens of millions files and billions of mappings.You gotta optimize it to under 100ms and hopefully it scales sub linearly with the database size.
>>109595968I haven't tried going past 90k to benchmark it even further, though it would be worth trying to duplicate my library a few times to see where it might grind. Like I said, it started as a hobby project so currently I'm just doing whatever features I want to personally use. Next up would be a plugin system that lets you write scrapers/importers or accept tags from autotagging models.>>109595482I mean there might be unused/legacy code in there or suboptimal patterns, but as far as I know I'm the only user and it's not bothering me, so it doesn't make sense to look into it atm.
>>109596150>though it would be worth trying to duplicate my library a few times to see where it might grindhttps://github.com/funmaker/hygenHere is a simple image + tags(sidecar) generator I use to benchmark my project.For the performance tips, I personally do not know how hydrus actually does it, but it archives pretty good performance with sqlite alone. I am using postgresql and use intarray extension. Basically, I store sorted list of post ids for each tag and then do set operations on them. Sorted arrays are very fast for this, you can get unions, intersections and differences in O(n) with simple algorithms. If you use Rust, you might do all of that on the Rust side and even add some smart parallelism to really speed things up. But that if you ever feel like optimizing it all. Just be mindful that if your solution is worth shit, people with millions of images will flood in and constantly bitch about performance.Good luck!
>>109596271Thanks anon>Just be mindful that if your solution is worth shit, people with millions of images will flood in and constantly bitch about performance.Yeah, I get that, but I'm not intending to replace hydrus, in fact somewhere down the line I want the server to push tags with good enough sources (direct booru scrapes or manual tags) to the hydrus PTR, but once again that depends on demand
>>109593708BASED BASED BASED I always wanted something like this. Always felt getting used to hydrus was a waste of time because now everything is fragmented under a trillion tags and easy to lose track of compared to just using subfolders
>>109596880>because now everything is fragmented under a trillion tags and easy to lose track of compared to just using subfoldersYou can use tags like paths. Tags are more general structure.
>>109596880Naiad doesn't move anything, it just creates a list of hashes from your files, creates a thumbnail cache, then whatever tags you have are stored in a sqlite db.
No, thanks.
>>109593708What are you using to tag the images?
>>109597717Currently just manually, but you can pull tags from a hydrus-derived database sitting in a public repo. It doesn't have tags for everything, but at least half of everything I throw at it gets tagged pretty easily with just a download.
