/g/ - total dad that i need this for local ai what i ca - Technology

Anonymous

11/09/26(Mon)09:24:14 No.110281849

File: guguklgufykklghk;ljk.jpg (100 KB, 1000x1000)

100 KB JPG

Anonymous 11/09/26(Mon)09:24:14 No.110281849 Archived

total dad that i need this for local ai
what i can actually run on 48gb unified memory?
Gemma 4 31B?

Anonymous
11/09/26(Mon)09:25:59 No.110281863

Anonymous 11/09/26(Mon)09:25:59 No.110281863

>>110281849
I never thought about running local models on my macbook although I have 64gb m1 max. Should I try it?

Anonymous
11/09/26(Mon)09:28:28 No.110281882

Anonymous 11/09/26(Mon)09:28:28 No.110281882

>>110281849
These things are not good at dissipating heat

Anonymous
11/09/26(Mon)09:30:16 No.110281897

Anonymous 11/09/26(Mon)09:30:16 No.110281897

>>110281882
well i dont intend to run it 24/7 just ask it stuff from time to time and maybe code a bit

Anonymous
11/09/26(Mon)09:52:24 No.110282036

Anonymous 11/09/26(Mon)09:52:24 No.110282036

>>110281863
You can run qwen 27b on 16gb, you won't notice the difference compared with Claude and you can run it virtually forever. Go for it bro.

Anonymous
11/09/26(Mon)10:07:34 No.110282126

Anonymous 11/09/26(Mon)10:07:34 No.110282126

>>110281849
>total dad
What?

Anonymous
11/09/26(Mon)11:03:58 No.110282529

Anonymous 11/09/26(Mon)11:03:58 No.110282529

>>110282126
*Told

Anonymous
11/09/26(Mon)11:30:28 No.110282775

Anonymous 11/09/26(Mon)11:30:28 No.110282775

File: IMG_ufc97c.jpg (49 KB, 472x472)

49 KB JPG

>>110282036
>qwen 27b
>same as claude

Anonymous
11/09/26(Mon)11:42:25 No.110282891

Anonymous 11/09/26(Mon)11:42:25 No.110282891

File: 1794185565551625.jpg (240 KB, 984x1599)

240 KB JPG

Anonymous
11/09/26(Mon)12:38:26 No.110283343

Anonymous 11/09/26(Mon)12:38:26 No.110283343

File: z-image-turbo_00102_.png (1.22 MB, 1024x1024)

1.22 MB PNG

>>110281849

Anonymous
11/09/26(Mon)12:38:59 No.110283351

Anonymous 11/09/26(Mon)12:38:59 No.110283351

File: hedoesntknow.png (110 KB, 498x408)

110 KB PNG

>>110282036

Anonymous
11/09/26(Mon)12:40:29 No.110283366

Anonymous 11/09/26(Mon)12:40:29 No.110283366

>>110282036
>You can run qwen 27b on 16gb,
You cannot. I have a 16gb card and its not capable.

Anonymous
11/09/26(Mon)13:22:41 No.110283722

Anonymous 11/09/26(Mon)13:22:41 No.110283722

File: 64753625456875743.png (27 KB, 1226x176)

27 KB PNG

>>110281849
>https://github.com/drumih/turbo-fieldfare

I am on 16gb m6 mini and I can run pic. related
Mind you I made my own system based on this github and bunch of other frameworks, but similar principle. (I am running it from external drive so I won't damage my internal one thunderbolt 4)

Anonymous
11/09/26(Mon)13:36:06 No.110283843

Anonymous 11/09/26(Mon)13:36:06 No.110283843

>>110283722
only possible using 2bit quantization but even then it is around 13-16gb ram use so you can't even open a browser at the same time

Anonymous
11/09/26(Mon)15:43:14 No.110284942

Anonymous 11/09/26(Mon)15:43:14 No.110284942

File: qwen_27b_16gb.png (95 KB, 1888x250)

95 KB PNG

>>110283366
Maybe that Anon is using a setup similar to pic related

Anonymous
11/09/26(Mon)16:06:12 No.110285164

Anonymous 11/09/26(Mon)16:06:12 No.110285164

>>110284942
How do they get 128k context? I can literally "only" get 64k context on my 20 GB 7900XT without loading anything to system RAM.

Anonymous
11/09/26(Mon)21:22:16 No.110287451

Anonymous 11/09/26(Mon)21:22:16 No.110287451

I run Qwen 3.8 flash 125b on 4070 12gib and 64gb ram.