This is a general which is focused on archiving, but also interested in other related topics.Storage technology and file sharing:Hardware, software, services, shadow libraries, backups, home server, and networks such as tape drives, HDDs, file systems, archive.today, IPFS, Arweave, BitTorrent, etc.Development:E.g.: web archiving is much harder in 2026 compared to 2016. Too many websites are walled off by systems such as Cl0udflare, making it impossible for services such as archive.is to capture their webpages. That's a big chunk of important data that easily disappears with no web archive captures. We have to develop solutions to this, such as using the SingleFile extension and other stuff.In-depth history:E.g.: important web history events and future events such as sites closing, or get into the "minutia and trivia" about the history of websites and all the little changes.Analysis:E.g.: analyzing files and folders that you obtained from scraping or data hoarding, or, what you're sad was lost and not archiving, what you're glad was archived.Questions:Ask whatever questions about any of this.Previous>>108914628
Inspirations for this general:/dhg/ - Data Hoarding General>Links>Rentry: https://rentry.org/dhg>>What is /dhg/>In this thread we discuss and create technology and software for data-hoarding, archiving, scripts, and more.>>gallery-dl - scrape images, manga, videos and more from many websites>https://github.com/mikf/gallery-dl>>Hydrus Network>https://hydrusnetwork.github.io/hydrus/>>Stash>https://github.com/stashapp/stash>>SmartImage>https://github.com/Decimation/SmartImage/dapp/ P2P Decentralized Applications General>Share your favourite dapps here.>>Examples:>>brig https://brig.readthedocs.io/>ipfs https://ipfs.io/>ZeroNet https://zeronet.io/>Arweave https://github.com/ArweaveTeam/arweave>Gitopia https://gitopia.org/>BitTorrent>>Leave your suggestions below.>>These components collectively make up the future internet known as web3./dshag/ thread>Data scraping, hoarding and analytics general thread.>What are you scraping, hoarding or analyzing frens? Also post some pics so I can post them from next time, anime also works/AAD/ - Archiving And Donating computer resources general>https://desuarchive.org/g/thread/108890811/
In the previous thread, I was wonder about a current-day localhost to Internet tunnel which doesn't suck. I wanted to use it for web archiving. I found one!!!How it ranks:- Cl0udflare Tunnel (formerly Argo Tunnel): sucks the most, have to put in credit card info for their free tier- ngrok: sucks less, have to make an account for anything to happen; the free tier has basically unremovable verification walls put up before accessing any of the target webpages- https://loca.lt/ : OK. Same as ngrok except no account is needed- https://localhost.run/ : the best! Only possible problem is subdomains rotate every 5 to 15 minutes, but I don't care about that.
Positive:catbox.moe was recently unexcluded from web.archive.orgNegative:IA still sucks and is very untrustworthy. 2051 websites are still excluded, and archive.org/details/ have done multiple idiotic mass deletions.
>>109032656>video, androidFull version of OP video with audio:https://web.archive.org/web/20260303103254/https://chanii.ddns.net/b/src/1772394666845.mp4It's a cool Cleon scene from some sci-fi TV show:"How old are you?I have told you: just over 18,000 years.You're a liar. Nothing lasts that long."
>>109032731One of the commands I run:$ ssh -R 80:localhost:2016 nokey@localhost.runI use IPFS and ipwb to make a <s>WARC replay</s> webpage capture show up here:https://archive.is/https://silver-bear-41.gw.ipfs.ninja/ipfs/*https://web.archive.org/web/20260612002528/https://80cc6707802495.lhr.life/memento/20260612001835/https://rule34.xxx/index.php?page=post&s=view&id=6315729rule34.xxx is a website which is excluded from web.archive.org, due to artists being pricks or something.
People do so much bedrotting and doomscrolling: just consuming content from social media and stuff. They lie in bed on their mobile devices, and most people don't archive what they see.Like a month ago, I found out that the social media platform Threads is more archive-friendly than X (formerly Twitter). Years ago, Twitter was archive-friendly and alt-front-end-friendly (not anymore).Reminds me, I was talking to some guy IRL and we agreed that:- Mobile devices like smartphones and tablets = entertainment or amusement devices, locked-down toys which aren't real computers which can do work- Laptops = a compromise between mobile devices and desktop computers, can do real computational work- Desktop computers = most productive computers, best type of computer for doing important thingsThe more portable the computer is, the less powerful it is (both in terms of its own ability and what people can use it for).>>109032731Version with a screenshot in the locket.
