File: (make-lisp-great-again).png (1.15 MB, 1600x1200)

1.15 MB PNG

Nice
https://github.com/lem-project/lem/blob/main/extensions/coalton-mode/coalton-mode.lisp

> new thread
fantastic and beautiful

emakkk

>>106994327
>Coalton and LEM
based

>common lisp being slow for reading stdin
lmaoing
imagine having to write IO and buffer logic just to process some logs. Kudos on the monkey that posted that on the previous jej

File: 1759117867867997.png (25 KB, 1021x253)

25 KB PNG

bruh

Again asking for help for this:
>>106987033

Some anon in last thread managed to get it working but we don't understand why it is not working for me

>>106995201
Don't worry I'll mantain it when I learn enough Lem

>>106995377
it just killed all my expectations, someone who thinks webview is a good substitute for anything just cant make good things

File: Screenshot From 2025-10-2(...).png (109 KB, 1239x505)

109 KB PNG

#!/usr/bin/env -S sbcl --script

(load "~/.sbclrc")

(require :sb-sprof)


(declaim (optimize (speed 3) (safety 0) (debug 0)))

(defun main ()
  (declare (optimize (speed 3) (safety 0) (debug 0)))
  (let ((i 0)
        (line nil))
    (declare (fixnum i)
             (type (or string null) line))
    (loop
      do (setf line (read-line *standard-input* nil))
      while line
      do (incf i)
      finally (format t "size: ~a~%" i))))

(eval-when (:execute)
  (sb-sprof:with-profiling
      (:max-samples 100000
       :sample-interval 0.00001
       :report :graph)
    (main)))

It indeed wastes most of its time with UFT-8 conversion stuff. If I use (with-open-file (str file) :external-format :my-format ...) it goes faster, but then I can't use stdin.

>>106994238
no use case for lisp

File: evolution-of-lisp-aliens_white.jpg (392 KB, 1920x1080)

392 KB JPG

>CIEL Is an Extended Lisp
https://ciel-lang.org
https://github.com/ciel-lang/CIEL

>>106995452
Have you tried using read-byte instead and just treating the input as a raw binary?

>>106995650
but I need to run regex on it bruh

>>106995452
God the loop syntax is so fucking fugly

>>106995452
I think I found the fastest approach for this, but its incompatible with stdin, is that strictly neccesary?

Please give me GNOME 2 but written in Lisp (including REPL) thank you

>>106996861
Ideally for scripts yeah, but what have you found?

>>106997006
Ok so I tried everything under the sun
>streaming bytes
>buffers
>transducers
>alexandria
>fast-io

First of all my test was over a 1.5 GB log file from
https://github.com/logpai/loghub (HDFS_v1). I made a simple python benchmark to
check these regex over each line (not at the same time) using 'with open' and
'for line in f'.
<code>
HDFS_PATTERN = re.compile(r'^(\d{6} \d{6}) (\d+) (INFO|WARN|ERROR) ([^:]+): (.+)$')
BLOCK_PATTERN = re.compile(r'blk_-?\d+')
IP_PATTERN = re.compile(r'\d+\.\d+\.\d+\.\d+')
THREAD_PATTERN = re.compile(r'^\d{6} \d{6} (\d+)')
CLASS_PATTERN = re.compile(r'^\d{6} \d{6} \d+ (?:INFO|WARN|ERROR) ([^:]+):')
</code>

that got me these times
<code>
PYTHON
Log levels (11.2s): INFO 10.8M | WARN 363K
Blocks (10.2s): 575K unique
IPs (49.1s): 203 unique
Threads (9.3s): 27.8K unique, 11.2M total
Classes (9.9s): 9 unique
Top 5: FSNamesystem 3.7M | PacketResponder 3.4M | DataXceiver 2.5M
FSDataset 1.4M | DataBlockScanner 120K

</code>
in the end I had to use lparalel, splitting into chunks and mapping a regex over
each line in each chunk with a scanner defined by 'ppcre:create-scanner'
that got me these results:
<code>
LISP LPARALLEL
Log levels (5.4s): INFO 10.8M | WARN 363K
Blocks (10.2s): 575K unique
IPs (19.0s): 203 unique
Threads (3.3s): 27.8K unique, 11.2M total
Classes (4.2s): 9 unique
Top 5: FSNamesystem 3.7M | PacketResponder 3.4M | DataXceiver 2.5M
FSDataset 1.4M | DataBlockScanner 120K
</code>

Still that's pretty fucked, wasnt THAT complciated but still significantly more
involved than a sinelg python readline. This could probably be turned into a
library so you dont need to deal with it again though. And like I said, no
stdinput because we need mem access. I've simplified the script so it fits in here give me a sec.

>>106997848
lol fucked the code blocks my bad

>>106997006
Didn't fit here:

https://pastebin.com/MxqFqgmV

so for example with the log entry you showed before,

>[22/Oct/2025:03:39:03 -0300] GET /url/path/9f37f23a-61f6-4cff-a38f-d900d93c7ce6/ HTTP/1.1 status=200 rid="cb3c45f3-d98c-43ce-bc29-a61571902381" host="host" forwarded_host="host" forwarded_for="ip1, ip2, ip3" referer="referer_url" ua="ua" res_bytes="74693" req_duration_s=0.3885111

you could use the pmap function there in the script to do something like this

;;; Extract HTTP status codes
(defun extract-status (line)
  "Extract status code from: status=200"
  (multiple-value-bind (match groups)
      (ppcre:scan-to-strings "status=(\\d+)" line)
    (when match (aref groups 0))))

;; Usage:
(pmap "access.log" #'extract-status)
;; Returns: #<HASH-TABLE "200" -> 1523, "404" -> 42, "500" -> 3>


;;; Extract IP addresses from forwarded_for
(defun extract-client-ip (line)
  "Extract first IP from: forwarded_for=\"ip1, ip2, ip3\""
  (multiple-value-bind (match groups)
      (ppcre:scan-to-strings "forwarded_for=\"([^,\"]+)" line)
    (when match 
      (string-trim '(#\Space #\Tab) (aref groups 0)))))

;; Usage:
(pmap "access.log" #'extract-client-ip)
;; Returns: #<HASH-TABLE "192.168.1.1" -> 523, "10.0.0.5" -> 234, ...>

Could probably turn it into a library at some point.

>>106997917
>>106997848
that is so weird. I was never let down by CL's speed. This is such a silly use case and seems really weird to be so slow.

With-open-file with the appropriate external format gave me significant speedups, but it is still slower than python (probably there is a second or so to start sbcl, but still).

>>106998241
>>106998241
Cant let go now, I have to know why this shit is so slow.

>>106998355
I'll ask stackoverflow, there are some good lisp autism there

what are you coding?