Swiss anon strikes again!There is the concrete possibility that the internet will go down soon due to war or some AI fuckery... are you ready to browse the archives offline?>there's already more complicated code for it on github so you can run it on a serverAlright but I want to do it myself from scratch, as an exercise, with the functionalities that I desire and as simple as possible, less than 500 lines of code.My homemade Python/HTML reader is starting to look good, I finally managed to get the correct reverse reply links with a bit of vibecoding help.These little niggers are hard to implement if you never tried a similar task beforehand.The colors are not definitive, still experimenting with that.I want (YOU) to try building a similar reader for yourself, you can find some code screenshots here and in previous thread as a help.
I've been also adding some new functions, since the picture in the OP is very similar to iteration 4.Have a look at this new part, it ranks posts according to the number of replies, and also labels them to distinguish OP posts from standard replies.
>>545306045For now this is the absolute top 1000 in the first 1000 threads from 2013.It's still a bit slow when you start asking for more than 10k threads.Top replies are usually either high IQ posts, controversial/ bait or similar.Interesting nonetheless.
This is come of the code I used. It's partially vibecoded of course, but it's incredible what you can do with LLMs.
Fake and jewish
>>545306270OY VEYLook who has joined the chat!
Bump for interest
>pay more attention to American bot posts with this one little trick to boost visibility of American bot threads and whitelisted threads and posts boosted by amerimutt bot farmsyeah thats exactly what we needed
>>545306320Thanks fren.You can also verify on 4plebs the correctness of what my scripts are turning to HTML by searching for the post number.For the next experiment I'm ranking all posts in the first 10k threads from 2013.Let's see how long does it take, surely I will optimize it to go faster later.1 iteration in the main loop = 1 thread.
>>545306672KYS KIKE / GLOWIE
Post #4 for instance is about theodicy, and it is high IQ and controversial since it triggers abrahamic cultists.This ranking system allows you to see through the noise of the matrix.
Help keep the thread bumped frens, this is good prepping material for what's coming.My next top 1000 has already ranked posts in 3500 threads from december 2013, still going steadily.
KEYWORDS:off-grid bug-out-bag shtf stockpile bushcraft self-reliance water-purification rationing first-aid bunker edc freeze-drying ham-radio geiger-counter generator survivalism foraging tactical-knife permaculture perimeter-security emp barter-system canned-goods topographic-map solar-arrays ferro-rod gas-mask hydroponics fuel-stabilization tarp compass cache grid-down conservatism sovereignism nationalism libertarianism patriotism traditionalism capitalism deregulation small-government law-and-order border-security cultural-heritage meritocracy traditional-family federalism anti-communism free-enterprise second-amendment euroscepticism populism flat-tax subsidiarity protectionism private-property judeo-christian-values anti-woke social-stability executive-strength religious-freedom individual-responsibility fiscal-conservatism constitutionalism laissez-faire algorithm repository debugging framework api compiler database open-source backend frontend variable loop syntax cryptography devops refactoring ide array function git cloud-computing scripting asynchronous machine-learning cybersecurity object-oriented deployment polymorphism markdown tech-stack query recursion concurrency data-structures python NSDAP NS national life history strategy biological determinism polymorphism SNP haplogroup wehrwolf warwolf
Neato
Almost there.It still takes ~1h to go through 10k threads with all their posts, there's room for speed improvement, but it works.
MORE KEYWORDS:hitler nazism reich swastika fascism germany wehrmacht gestapo ss sa holocaust antisemitism propaganda goebbels himmler aryan totalitarianism dictatorship fuehrer militarism nationalism autarky lebensraum anschluss munich versailles weimar mein kampf reichstag concentration camp ghetto deportation genocide eugenics racial purity youth hitlerjugend censorship total war axis allies stalingrad berlin nuremberg trials laws kristallnacht pogrom sstotenkopfverbände einsatzgruppen final solution auschwitz treblinka dachau buchenwald totalitarian ideology white rose stauffenberg putsch dictator repressive regime autocracy gleichschaltung lebensborn volk volkssturm luftwaffe kriegsmarine blitzkrieg barbarossa occupation annexation collaboration partisan liberation armistice surrender bunker denazification postwar tribunal complicity memorial fascist authoritarian militant baeumlerThis is a bit of spam so that anons can find this thread in the archives. The first list is a bit gay because it was made by (((Google AI)))national socialism folk people's community fatherland homeland blood soil living space master race race racial hygiene nordic aryan ancestry clan family motherhood large families hereditary health eugenics selection breeding order discipline duty loyalty honor obedience comradeship community solidarity self-sacrifice heroism martial spirit soldierly front-line spirit leader principle allegiance authority realm greater germany pan-germanism self-sufficiency work labor service reconstruction peasantry farming craftsmanship middle class socialism national economy thrift cleanliness virtue custom morality faith myth symbolism swastika runes flag banner anthem youth sport health nature future
>>545305993The OP posts are rarely the most interesting themselves, but serve as a jumping off point for better responses. Engagement is not quality, just one variable to account for.Do keep in mind also that before LLMs we had plenty of organized and dedicated shill teams here as well.
>>545307415bump
>>545308917What I'm seeing in the top lists is that obvious shills and bots are usually recognized and don't receive that many replies, unless the topic is extremely controversial, which interests me.>The OP posts are rarely the most interesting themselves, but serve as a jumping off point for better responses.That's why I put separate labels for OP posts and reply posts in my lists.When the 10k run is over I will post the ranking on catbox so you can play with it.
>>545309072Release the code already. I will review it.Otherwise, these threads are more like Plato's cave than useful reality.
I don't get it, what's the point?
Alright the top 1000 posts ranked by number of replies in the first 10k threads from 2013 are ready.LINK to the HTML:https://files.catbox.moe/6x83wm.zipHappy reading!
>>545309978Read this >>545310526 and if you're not a retard you will understand.
First ranked is the sticky when Nelson Mandela died, kek.
Help with the bumps frens I can't do all by myself.
>>545311814Here you go
>>545312146>triple slash IDinteresting... checked!
Is everyone still playing hide and seek?HellooooI want to make the code faster, 1h for 10k threads is too slow. I don't want to complicate it too much either.Let's try vibecodemaxxing
>>545314229why overload it with 10k threads? just make it pullup 10 times at a time everytime you scroll down in it everytime it goes under <5 pullup another 10 threads
>>545314442Eeh the ranking is quite compute intensive because it must check every single post against every other post in the 10k threads.It's already quite optimized as it only passes one post once but I'm trying to see if the LLMs have some nice ideas to improve it even more.
>>545309918One bread crumb at a time anon... I don't want it to be too easy for glowies and retards.Once it's optimized I will release some parts as text so you can put it into Spyder or whatever IDE you have.
>>545314229You're working in Python and gave constraints like to be simple and under 500 lines. Don't expect speed.Just ask your clanker to profile the bottlenecks and give you strategies around them. And switch to something that's not interpreted and allows proper parallelism/concurrency.In heavy bulk jobs finding the right things can make the difference between finishing in 2 days vs. 2 billion years.
Also... if there's any richfags among you, it would be nice to be paid some money... to get the full package before anyone else I mean.I think I'm the type of person you should try to empower in our eternal fight against marxism.
>>545315095Thanks, yeah that's what I'm trying to do.
Bumperino?I must also find a way to load the csv in chunks instead than 6 GB at a time.
I always just downloaded threads that I found interesting over the years with a simple click that gathers it into a single html file.I would dl everything going back years if it were convenient and everything gathered in a single file.I can't justify it if it's around 1tb or so either
>>545315167you are an incompetent retard who fails at trivial web scraping.maybe get some real skills before asking for money
>>545305993neat, finally my posts will be recognized for the brilliance they are
>>545318077Do you want to scrape everything from scratch?Maybe I'm not the retarded one here.Also you cannot scrape stuff deleted 10 years ago.NIGGER
>>545318170Nothing was lost after 2013. High IQ posts can be multidimentionally retrieved.
>>545318964anything that increases the visibility of my posts in the archives is a boon to me, doing god's work frendo
>>545317657That's one way to do it, but I can access EVERYTHING now.Except the pictures, these can come later, there's archives of thumbnails and pictures too.
>>545318932you of course are not aware that 4chan is archived at at least 10 separate places yes?for free?scraped by people who actually dont have to ask sloppa for help with fucking reply links jesus christ
>>545305993>And the library starts to come onlineI love you guise
>>545319376Who cares, if the internet goes offline these places will be gone.Mine is the only simple solution for who wants to browse the archives locally without spending time configuring Fuuka etc.
>>545312222CHECKED
Alright the LLM managed to reduce elaboration time to ~30 seconds for 10k threads.Impressive.
>>545305993Interesting thread fren. Have a bump.>>545306270Glownigger spotted.>>545306672Glownigger spotted.
>>545320733Thanks mate.This is how we bypass jewish censorship and win our European nations back.
It's so fast now.We have a winrar for the first 100k posts, lmfao:
I can process and rank billions of posts in minutes now.Wow, I didn't know my shitty laptop was able to do this.
>>545305993bumpo
>>545305993digital is the wayget the pdfs before internet is censored
