Web Scraping GeneralRevival editionFAQ: https://rentry.org/scrapists> Captcha serviceshttps://2captcha.com/https://www.capsolver.com/https://anti-captcha.com/> Proxieshttps://hproxy.com/ (no blacklist) (recommended, owned by friend of /wsg/)https://infiniteproxies.com/ (no blacklist)https://www.thunderproxies.com/http://proxies.fo/ (not recommended)> Network analysishttps://mitmproxy.org/https://portswigger.net/burp> Scraping toolshttps://beautiful-soup-4.readthedocs.io/en/latest/https://www.selenium.dev/documentation/https://playwright.dev/docs/codegenhttps://github.com/lwthiker/curl-impersonatehttps://github.com/yifeikong/curl_cffihttps://github.com/mikf/gallery-dlhttps://github.com/yt-dlp/yt-dlp> Cool projects by members of our communitydoubledouble.top / lucida.to - Free music scraped from spotifykemono.cr - Kemonoparty for fanbox/fantia/subscribestartv.weboasis.app - Falcon, a goy invite-only pirate streaming service that scrapes video streams from multiple sources
how do you save the original images on j18/jlist/hmarket?https://desuarchive.org/g/thread/107515771/#q107516090_2
>>107516932is it even possible to still scrape what you want today with most sites "protecting against bots" with cloudflare's stupid shit or other similar "checking your browser" things?
thank you for posting this>t. longtime scrapmaker first time reader
>>107521215works for me
>>107521307but unironically
>>107516932You should unironically be on Awoo to discuss scraping.
>>107521215It has more to do with the IP you're using than anything else
Is it possible to use some scraper for stock market data?Every single fucking API either close down the free version or gimp it beyond reason so you need to buy premium.
what exactly do you scrape? anime tiddies?
>>107516932https://x.com/fireplacegg/status/1996265758867992684Not gonna lie this is the reason why I want to scale my scraping
>>107516969hey anon check desuarchive
>>107521869this post is an ad
Oh is this how the retards on /aicg/ learned to scrape. Finally.
>>107516969you again...didn't someone solve your problem in the other thread ?also are you the poor guy or the one offering 1xmr ?
>>107521668bump for this
>>107521215No but this schizo will keep spamming his thread pretending it's still possible for the next decades. Scraping hasn't been possible for at least 5 years.
>>107525968do you seriously think shit cloudflare turnstile and anubis stopped us from scraping or even made it noticeably more difficult ?The only bump in the road are captchas but its easier for us to solve them using indians or AIs than it is for you when you try to post on an unusable imageboard.
>>107525968>laugh in residential proxies
>>107526298Greetings fellow chad scraper>laugh in curl_cffi
>>107524781yeahhttps://desuarchive.org/g/thread/107515771/#q107516090_14
>>107526438why did the fag janny delete the old thread ?
>>107526452idk some people r being schizo i guess
>>107526452they hate actual tech.
>>107526438i get this error on cunny doujins.https://pastebin.com/dDEQ9Rpn
>>107523646No it's NOT
>>107516932dumb idiot OP killing the original discord because he wanted to act be a script kiddy
>>107525968that's the feeling i'm getting. if i can't even visit a website without it "checking my browser", what chance does a scraper have?even if there's a bot that uses an open firefox instance and simulates mouse movement/scrolling and delays clicks 1-5 seconds randomly, how long will that last?
>>107526939no one likes discord.
>>107527500yeah because this thread has had fascinating, valuable discussion so far. retard.
>>107516932can you even not innawoods without a mobile phone number nowadays?
>>107526775Just read nigga, it means the name[id] doesn't exist. So it's either using another id for your doujin or there is an additional logic your code isn't handling for this category
