> can be used as a database or a super fast, super easy data traansformation tool> dependency free> SQL based> Made by EvropeansBest tool for 95% of data related tasks. If you don't need spark (and you don't if you work in a normal company) it should be the default choice by now. FUCK pandas
>>109941471any good demos to share?I also don't like pandas much.
>Dutch projectAnd another one for the growing shitlist.>Python>NixOS>Vim>DuckDB>PowerDNS>Bluetooth
>>109941739mostly do boring data pipeline stuff with it so nothing worth sharing, but the flexibility to for example SELECT <your query statement> FROM 'path_to_file' as a way to ingest data for instant transformation is convenient as hell. I suggest just installing it and using "duckdb -ui" for exploring on your own. There's also a Python API you can use easily
>>109941797reminds me of this, written by my waifuhttps://github.com/mithrandie/csvq
>>109941471>Made by EvropeansThanks I'll add it to my blacklist.t. European
>>109941471there are people who rewrite databases, something I thought would be unfeasiblebut the new stuff is still just strings sent to a magic algorithm engineinstead of it being like LINQ, but not a string generator middle manI know I'm stupid, but man... I know I'm right so wtf is wrong with the smart people
>>109941471wtf? just use sqlite3
>>109942077they’re not interchangeableits fundamentally different because DuckDB is based on Arrow whilst sqlite uses the traditional storage format.https://arrow.apache.org/overview/I considered DuckDB but decided against it for my project because the traditional row-based format was more suitable.
>>109942137>>its fundamentally different because DuckDB is based on Arrow whilst sqlite uses the traditional storage format.sqlite literally works everywhere. if it ain't broke, don't fix.
>>109942146some people working with operations o big data need Arrow tho. like scientists or chinese thought police.but yeah most cases sqlite is just fine
>>109942180>like scientists or chinese thought police.neither of those people matter
>>109941471>single node
>>109942146you were huffing paint in the restroom during the lecture on OLTP and OLAP workloads
>>109944090OLAP deez nuts
>>109941471>>109942180>>109942187I'm a scientist and I've used it when the data wrangling became too annoying and slow on Pandas and Postgres. Just lovely, although the spatial stuff is pretty lacking atm
>>109944180I can't imagine doing spatial stuff outside of R + sf + dplyr
>>109941471pandas or polars is much easier to learn than sql.
>>109944641>learn tools in 2k26
>>109944619I've very rarely dipped my toes to R. For me, it's been geopandas, pysal, and sometimes dask thrown in or QGIS for cartography and other things that are easier with a GUI
>>109944652are you just going to prompt all the time? Shit takes ages until you get the right answer. No scientist is going to do that.
