/g/ - /wsg/ - Web Scraping General - Technology

Anonymous

/wsg/ - Web Scraping General 06/27/24(Thu)14:26:25 No.101177514

File: scraper.png (1.62 MB, 1892x2142)

/wsg/ - Web Scraping General Anonymous 06/27/24(Thu)14:26:25 No.101177514 Archived

Web Scraping General

Anti-kike revolution edition

QOTD: What's your greatest scraping achievement?

FAQ: rentry co/t6237g7x

> Captcha services
https://2captcha.com/
https://www.capsolver.com/
https://anti-captcha.com/

> Proxies
https://hproxy.com/ (no blacklist) (recommended, owned by friend of /wsg/)
https://infiniteproxies.com/ (no blacklist)
https://www.thunderproxies.com/
http://proxies.fo/ (not recommended)

> Network analysis
https://mitmproxy.org/
https://portswigger.net/burp

> Scraping tools
https://beautiful-soup-4.readthedocs.io/en/latest/
https://www.selenium.dev/documentation/
https://playwright.dev/docs/codegen
https://github.com/lwthiker/curl-impersonate
https://github.com/yifeikong/curl_cffi

Official Telegram: @scrapists
Last thread: >>101151145

Anonymous
06/27/24(Thu)14:30:15 No.101177581

Anonymous 06/27/24(Thu)14:30:15 No.101177581

>>101177514
bunp

Anonymous
06/27/24(Thu)14:33:16 No.101177614

Anonymous 06/27/24(Thu)14:33:16 No.101177614

>>101177514
faggot shill thread

Anonymous
06/27/24(Thu)15:43:56 No.101178587

Anonymous 06/27/24(Thu)15:43:56 No.101178587

>>101177614
Seethe API cuck

Anonymous
06/27/24(Thu)18:43:41 No.101180846

Anonymous 06/27/24(Thu)18:43:41 No.101180846

'mp

Anonymous
06/27/24(Thu)22:11:46 No.101183106

Anonymous 06/27/24(Thu)22:11:46 No.101183106

'mp

Anonymous
06/28/24(Fri)01:31:59 No.101184623

Anonymous 06/28/24(Fri)01:31:59 No.101184623

'mp

Anonymous
06/28/24(Fri)01:49:35 No.101184724

Anonymous 06/28/24(Fri)01:49:35 No.101184724

Any good iPhone emulators? I need to intercept some app's requests and it's a mess on Android

Anonymous
06/28/24(Fri)04:10:27 No.101185565

Anonymous 06/28/24(Fri)04:10:27 No.101185565

>>101184724
https://www.reddit.com/r/EmulationOniOS/comments/1c5kxyo/available_ios_emulators_megathread/

Can't vouch for any of these, they all seem trash desu.

Anonymous
06/28/24(Fri)07:20:34 No.101187013

Anonymous 06/28/24(Fri)07:20:34 No.101187013

>>101178587
Nobody can take your thread seriously. Honestly have you ever scraped anything in your life? I can count the instances I've been forced to parse html on one hand. Almost everything has an API, that's literally how the sites and apps work. Most of the time there's no authentication, you can get all the data not just what's rendered, return more data at once, and so on.