I like using imageboard archives as alternatives to search engines like google, bing, duckduckgo, etc. so I will share what I know## Latest generation archiving solutions>webserver (frontend + search)https://github.com/sky-cake/ayase-quarthttps://ayasequart.org>data archivers (pick one)https://github.com/sky-cake/Ritual/tree/masterhttps://github.com/bibanon/neofuuka-scraperhttps://github.com/sky-cake/neofuuka-scraper-plus-filtershttps://github.com/bbepis/Hayden## Image search (More coming soon)https://archive.4plebs.org/_/image_search/## List of existing archiveshttps://archive.4plebs.org/_/articles/credits/#archivesFrom this link, you'll notice existing archive search offerings continue to decline as datasets grow and hardware becomes more expensive## Other>https://4rchive.org/ was another newer archive but it redirects to some ad page now>https://ayasequart.org/g/thread/105241843#p105241843
This thread is archived here https://ayasequart.org/g/thread/109272347
And at,https://desuarchive.org/g/thread/109272347https://archived.moe/g/thread/109272347https://arch.b4k.dev/g/thread/109272347
Haven't found any other decent alt search engines.
Hello? Archivers?
this is a good topic but i have nothing to say about it
>>109273654Thank you
>>109273690You are always welcome here
Can you spoon feed me, if I wanted an image of tubgirl and google won't provide it, I go to which link to search for it?
>>109275266https://archive.4plebs.org/_/image_search/tubby%20girl/
>>109272347Have a bump for Chihaya
>>109272347Does there exist any existing archives of /wsg/ content including .webm files? I'm trying to track down some stuff from when the community OC stuff was bigger.
>>109272347which GEGL plugin did ya use for this?Glass Metal Marble? It looks better on larger text
>>109276217Good taste. Thank you>>109276845you're right, there are only thumbs here,https://archived.moe/wsg/thread/6188058/#6188058Not sure which archives would have this.>>109276863Hey beaver, how's it going? Idk, it was made quite a while ago when you first started sharing your gimp3 plugin masterpieces. You taught me how to use it in your thread. Thank you, god bless
good thread
>>109272347There's a 300 post thread on /t/ about this exact topic.>>>/t/1153106
>>109278080That one is about the data, and this one seems to mostly cover the software
>>109278080cheers, thanks for sharing that
>>109273548hi
I was going to make the next thread in this general:>/asdiq/ Archiving, storage tech, development, in-depth history/analysis, and questions generalwith>CURRENT EVENT: btdig.com is dead!and picrel, but then I saw this thread.>>109278080That's focused on 4chan. There's other imageboards. Types of imageboards:- Futaba-style imageboards: "chans"- Danbooru-style imageboards: "boorus"
>>109284392Post I was going to make in that hypothetical thread:What are you working on? For me, I'm working on the following. Here's a .torrent file:https://web.archive.org/web/20260716002436/https://eu-west-1.s3.fil.one/antiarchiveorg/3850e42c8449a43e2959db46ad4985ded54408aa.torrent?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=8F92D0X7RS74LK3JIZKJ%2F20260716%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260716T002301Z&X-Amz-Expires=72400&X-Amz-SignedHeaders=host&X-Amz-Signature=565d3ece8ebdbe0882ce6a068914b2c8415a9a60957cbe69d816df661164a22bmagnet:?xt=urn:btih:3850e42c8449a43e2959db46ad4985ded54408aa&xl=964141778368It's 897.92 GiB of this one imageboard; history of it:- ~2014: booru site began.- 2019: site shutdown because booru.org is untrustworthy. Maybe for the best that that site ended and it's data continued in BitTorrent and IPFS...- 2022-11-17: complete with-outlinks WARC of the site shared as a torrent.- 2022-11: WARC uploaded to https://archive.org/details/ ("975,256,044,559 bytes").- 2024: IPFS CID of the 898-GiB folder created.- 2025: WARC deleted off of https://archive.org/details/ by someone other than the uploader- 2026-07-15 UTC: 127.1 GB of it exists under "p" in "$ aws s3 ls s3://antiarchiveorg/ --endpoint-url https://eu-west-1.s3.fil.one" (am adding more to this S3 folder)
I have 1,105,578 full images from a 4chan board. Average size per file = 1.0035 MiB. Latest file is from 2023.Packed version (torrent, 1.058 TiB):>>>/t/1399638Unpacked version of that torrent (IPFS, 1.2 TB):http://149.202.248.209:8080/ipfs/BCIQEDWLVZQAVLLU2MO536QJH474C6GPJNIVSCSA3YZDDV5T37ZI6NCAAttached GIF is one image inside of it.
>>109284803"Interestingly", a .torrent of 1,105,578 non-packed files just ain't gonna work. Most BitTorrent clients will reject a .torrent file which is larger than 100 MB. And even if you change the settings to allow for that, opening up the Contents tab in qBittorrent will make the software use 100 GB of RAM.So basically, you have to pack the files into archive files for torrents which contain ~1 million items. IPFS works fine with however many millions of unpacked files in a folder. Make each subfolder contain 1000 files max, if you can.Here's another image in that one-terabyte set.
>>109272347>I like using imageboard archives as alternatives to search enginesworks OK, as long as the filenames have English words in them. I noticed that 4plebs does a thing where the IMG alt= is an AI-generated description of the image.>>109284392>btdig.com is dead!More info on that: see the following link and its Talk page as of todayhttps://en.wikipedia.org/w/index.php?title=BTDigg&diff=1364087436&oldid=1359567370
>>109284803Another useful thing with this set: it contains 4chan images which aren't in any of the 4chan archive HTTP websites! I found multiple so far. Maybe I'll post some old ones ITT. So REPLIES to this post might be images in that set.>>109284847>4plebs does a thing where the IMG alt= is an AI-generated description of the image.But is that searchable?
>>109284887Found this Chris Hansen pic named "1396015722551.jpg". It's 404'd at the following link (as of writing this), but after posting it right now it may show up as alive in Desuarchive. As long as the current 4chan software doesn't edit this image file (as in, remove metadata and so on).https://desuarchive.org/_/search/image/YzJnSzqLneWIsm_2jZEGbA
>>109284981>As long as the current 4chan software doesn't edit this image file (as in, remove metadata and so on).It did edit it. It's not hashhttps://desuarchive.org/_/search/image/YzJnSzqLneWIsm_2jZEGbAbut insteadhttps://desuarchive.org/_/search/image/n1qg-FgFkDDCJTgCc472lQHere's the unmodified JPG file:next URLYou can verify that it's the original hash by running this (same code 4chan archive sites run but in Bash):$ curl -k https://amelaz.space/raw/EyP6bvnu7FYb4TlK9RHUN-3YDMneeppwNiOH4ujZVB8 | md5sum | sed "s/ .*//g" | xxd -ps -r - | base64 - | sed "s/\//_/g" | sed "s/+/-/g"But hey, even with a different hash, it still restored one dead Desuarchive image.
>>109285029So using this torrent >>109284803 I could restore many dead 4chan images in Desuarchive (ones that return a 404 error but will return an alive status if I post the image here).I could do that, but would anyone be interested in such posts?
>>109284803I'm glad that there's some BitTorrent peers on that 4chan image collection, probably no IPFS peers with any significant amount of it.Decentralization AND distribution makes me think of something. Glowies know that individuals can easily be managed, but groups of people can change history. The Digital Nomad Guy talked about such potential big changes to history in his video "Why America Stopped Gathering" athttps://www.youtube.com/watch?v=3tYKw8OxRAgBTW, a gateway temporarily glitched when try to display a folder in said collection:>https://archive.is/H09wb = https://mintnho.store/raw/JRUixsjvwzgC6f9CI8BW34v4vDa9JmQxdriUvACs0qc>internalWebError: open /var/www/clients/client1/web1/home/dev/.ipfs/blocks/GX/[...].data: too many open files
>>109278080>/t/ threadIf things go well, a new /gif/ monthly release will happen within the next 27 days.(For this one thing, I need to complete approximately 33 GB per day: 3 days past, 135.5 GB "done", about 800 GB todo = only 1 day ahead of the curve.)In the meantime, I could share some imageboard things that I have in the form of archives.
>>109285265>4chan images>no IPFS peers with any significant amount of it.If anyone wants, they can download the .car files from the torrent, import them into their IPFS node(s), then keep the daemon running for months. I used to have a mass storage ipfs node running for year(s). However, that HDD is failing, so now I only have that ipfs peerID (and connected Internet-public services) running via a RAM drive in GNU/Linux.First 4 lines of my init file for when I restart the computer (or have a power outage):>sudo mkdir /mnt/ipfs>sudo mount -t tmpfs -o size=3G tmpfs /mnt/ipfs>export IPFS_PATH=/mnt/ipfs; ipfs init>cp ~/Documents/ipfs-init-config /mnt/ipfs/configSucks because it's only 3 GB in size, great because it's the fastest hardware-to-network speed possible. RAM is faster than NVMe. And of course RAM cost so much nowadays. Luckily, I bought my 32 GB of memory one to three years ago (I wanted 64 to 128 GB but settled with 2 16-GB DDR5 sticks).
Right now:>https://web.archive.org/save/...>The capture will start in ~41 minutes because our service is currently overloaded. You may close your browser window and the page will still be saved.I wish it was around a 3 minute wait time, not that long. According to duck.ai, S3's &X-Amz-Expires=72400 is in seconds. 72400 seconds = more than 20 hours.>>109272347>existing archive search offerings continue to decline as datasets grow and hardware becomes more expensiveReposting text and picrel:desuarchive.org was using more than 100 GB of RAM for their search feature. Shows that their web/database software is inefficient or something. MOTD:>The search engine usage exceeds the 128GB RAM a single server provides. It is paused while we look for solutions. Donations to the archive would be appreciated to help fund our server hardware & storage drives. We are looking for developers to help build new software and archives, discuss here.
7chan, at least as it is now in current year 2026, is the like the damn Reddit of the chan world.Attached pic is a screenshot of a 7chan ban from 2014 due to "not respecting chan culture". The photo or image macro in this screenshot is from rotate.php: it would rotate through some "you are banned" images, such as this other one (filename "rotate.php.png"):https://archive.is/2026.07.16-041231/https://ario5.0x0.io.vn/raw/4P0LXckW6SERMtG7Si8Qcryge1aU7JXgZBfuAdE9UNI
