There is the concrete possibility that the internet will go down soon due to some AI fuckery... are you ready to browse the archives offline?>there's already code for it on githubAlright but I want to do it myself from scratch, as an exercise, with the functionalities that I desire. I will have an AI optimize it later.My homemade Python/HTML reader is starting to look good, now you can even click hyperlinks to previous posts.The colors are not definitive, still experimenting with that.
I'm navigating the real Matrix.
Some free code for smart frens who will know how to use it:
If you were here when I posted my other threads you know where to find more info.If you're lurking leave a bump so more frens can see how to access the archives in case of total internet blackout.FUCK GLOWNIGGERS
The text archive of all /pol/ posts archived is only around 100 GB. But what is the fun with only text?Almost all the stuff posted here gets context from the images that are posted alongside.Good effort though, even text alone is sufficient to read for a lifetime.
>>545078340There's also archives of the thumbnails and full pictures.You can create a full 4plebs clone if you want.
As a suggestion, maybe try beautiful soup, anon.https://pypi.org/project/beautifulsoup4/
>>545078394Yes you can also directly look at the source code of the page.As I said, work in progress. I want (YOU) to build your own version for when SHTF.It works on archived .csv's for every board, maybe with a couple small changes.
>>545078585I just use kurobaex. No need to reinvent the wheel like a black man
>>545078649That cannot read .csv archives offline tho.That's the point of it all.
>>545078340>But what is the fun with only text?I use it to cross reference current events with what pol has talked about in the past.
>>545078206It's not AI fuckery that's the plan they want to bring it down or make it unusable so people will get baited for ID only secure internet where tracking is going to be even greater.
Thread theme:bitchute.com/video/5HAM2vuMsOpO
>>545078846Exactly. You can do even more (statistical analysis etc.) with some small code adjustments.
What 4chan archives don’t have viruses and shit on them or malware? That used to be an issue so I stopped using them
>>545081093No if you get the internet archive csv's.It's literally all text you can have an AI scan it for malware but I highly doubt there's any.
>>545081137Maybe I was doing something wrong but I would just google and go to one of rhe websites and that’s where I ran into issues years back. Maybe it was just pop ups or something getting by mu blocker I can’t remember but it was an issue at the time
Thread theme 2:bitchute.com/video/Dsfx0F2OYiXJ
Good I managed to remove excess spacing (and leaving plebbit spacing when the poster willingly did it).
What if the file is bigger than your PC's RAM?Split it in smaller files and load individually!
Could you stupid nigger not continue spam about the weather insteadLiterally kys
>>545084755Heatwave season is over.FUCK ALL GLOWNIGGERS ALL YEAR LONG
bump
A random gem I found. Really makes you think.
This is the maximum I managed to load in my 15 GB RAM at the moment.~1 billion posts from 2013 /pol/, 233678 OP'sWhen RAM is not enough just split the files.
Current load goes from December 2013 to August 2014.For more use the splitted files.
The word nigger was used 241'937 times in about 9 months.There are more mentions of the word "nigger" than unique threads (these are 233678).
Next step is to get these lil niggers to appear too
With just 289 lines of code you can already have a somewhat similar clone of /pol/ that you can browse offline, plus some other functions (search, split, output files, etc.)
It is also possible to mass export threads in HTML.
>>545078206>Reading DARPA algoslop offlinelmao @ your life
Come on, help me a bit with the BVMPING
>>545089313KYS
>>545089524Keep your panty hose on.
>>545078206digital is the way
>>545078373wow>>545078846give an example of this, as in an event and how exactly you cross referenced to it
>>545090798backing up ebooks is a way better use of time.
>>545091013I have that category see the 6th entryhere is another collectionREACTION FOLDERhttp://drive.google.com/drive/folders/1pDVj-pkOjbmDhfgaVjGCWfwyzIPmfn04
>>545091204this one is also good to understand what's going on in glowie heads 1/2
2/2
>>545091691Based, thanks fren
More thread music:odysee.com/@dissidentmusic:4/mr-bond-schlomo:d
>>545091770no problemo
What are you all saving outside OPs project?I'm saving how-to YT videos, books on gardening, and military manuals.
