What's up with tokens? All programmers know that a token is used in parsing. These AI companies are just arbitrarily assigning costs per token and we gladly pay for it and shape our economy for it and fire/hire people based on these tokens.A token is just a collection of characters, it could be the delimiter (space, comma, etc) or it could be some characters with a delimiter included. AI is so new that it seems like no one has taken a step back and really thought about "What am I paying for?".When your AI subscription (which is a bullshit payment model to begin with) runs out of tokens, it seems prudent to question, how and why? What does your preferred AI company/model decide what a token is and is not? Without digging too deep it would seem that utilizing the least amount of tokens in communicating with the LLM would be the best route to preserve tokens, however, the LLM spends tokens in the background and even in the foreground explaining thing to itself and you, whilst working.The AI HAS TO USE TOKENS TO EVEN WORK. Large Language Models cannot function without tokens, it is a foundational paradigm in their operation. The weights are assigned and calculated after the tokens are already delineated. It would seem there is no thought as to throughput or bandwidth (such as there is in streaming like Netflix for example), only tokens. You're charged the same for a token that is 1byte or >1byte.Charging per token without transparency as to how the system operates with tokens on the backend and frontend is a scam. The current crop of AI leaders understand what they are doing and barely anyone else outside really grasps what is happening. I'll put it this way...if Netflix charged you a monthly subscription fee and on top of that limited your streaming capabilities based on tokens (chunks of 1byte or >1byte) you wouldn't even make it through one episode of Stranger Things without exhausting your tokens. This is why AI is a bubble and a dishonest one at that.
I think there are legitimate concerns:Tokenization is opaque to most users.Different models tokenize the same text differently.Hidden prompts and hidden reasoning aren't always transparent.Subscription plans often market themselves more simply than the underlying usage limits.Those are reasonable things to question.Where I'd be more cautious is calling the entire system a scam. There is a genuine relationship between token count and inference cost, even if it's an imperfect proxy. Companies also compete on pricing, and over the past few years the cost per million tokens has generally fallen as hardware and model efficiency have improved.So I think the real debate isn't whether tokens are "real"—they're a genuine computational unit inside these models—but whether token count is the most understandable and transparent way to charge customers. That's a fair discussion, and I wouldn't be surprised if pricing evolves as AI services mature, just as cloud computing eventually developed billing models beyond raw CPU cycles.
bot thread
>>539856043>baitTokens can be small because typos or words the LLM has never seen before can be represented by tokens. They aren't abitrary. The training takes all its input and statistically maps tokens to the embedding space.
>>539856266This is (You), pic related
>>539856279>Says the LLM has never seen the words>Says it isn't arbitrary to tokenize unknown words
>>539856462it can tokenise individual letters if they don't fit into known words.
>>539856728Okay, but that doesn't change the point that there is no transparency in the delineation of tokenization. This is why there is such a heavy emphasis on prompting, how to utilize the AI/LLM in such a way that is efficient and burns the least amount of tokens (and we still get no transparency). If these companies are going to charge per token instead of a flat rate for computing power or some other metric, we shouldn't have to guess how many more prompts our context/conversation will last before being corrupted due to loss.
>>539856043What if AI, on surface is indeed just glorified Ask Jeeves, but in fact - it is just for mining bitcoins? All data centers, paid by taxpayers, to mine bitcoins for few nerds.>I would lol so hard.
The input of AI is tokens The output of AI is tokens > noooo you can’t charge by the token. because you just can’t!!!
>>539857123you can run a model yourself, on a computer that is not connected to the internet.https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF>>539857128you could just have free local ai.
>>539857123I mean, that is a thought.
>>539857071huh? one of the AI storytellers literally showed you token delineations as you typed. it's not really a secret. the prices behind it probably have to do with the cost per bit to process
>>539857256>you can run a model yourself, on a computer that is not connected to the internet.Yes, yes I could. I could not, however, mine bitcoins anywhere near that would be useful, with my basic 2060 Super and 16GB ram..we all know how energy intensive bitcoin mining is, and, on the other hand how light and simple basic LLM is on an average PC. Hmmm.. makes me think feverish thoughts.
>>539856043The Internal Formula (Finding the "Floor" Cost):Infrastructure teams figure out their absolute minimum cost-per-token using a multi-layered equation:Hardware Amortization: Calculate the cost of buying or building its AI chips, divides that by their expected lifecycle (usually 3–5 years), and calculates the baseline cost per minute of uptime.Energy & Cooling: They factor in the data centre's Power Usage Effectiveness (PUE) and local electricity rates to find the literal wattage cost of running an active inference workload.The Token Divisor: They measure the chip cluster's maximum realistic throughput—how many millions of tokens it can process per second at optimal capacity (called utilization rate).By dividing the continuous operational cost (hardware decay + power + network) by the average token throughput, they establish their true internal break-even cost.Why Retail Prices Diverge from Raw MathOnce the company knows the raw cost of a token, the marketing and enterprise teams adjust the final API price card based on three major factors:The Generation Premium (Input vs. Output): Reading text into a chip (input) happens all at once in parallel, using less energy per token. Generating new text (output) must be done sequentially, one token at a time. Even though a chip's overall hardware cost is static, output tokens require far more math and time.Massive Scaling Efficiencies: Companies process quadrillions of tokens a month. Techniques like "request batching" allow them to pack multiple users' queries into a single hardware cycle, dramatically dropping the energy cost per individual user.The "Loss Leader" Strategy: Tech companies frequently price their smallest or standard models (like Gemini Flash-Lite) extremely close to—or even below—the raw energy break-even point to build market share, hook developers into the ecosystem, attract and retain customers (Google search users looking at Ads), and beat out competitors.
>>539857071All the models have APIs where you can send text and find out what the tokens would be. This is often used for cost estimation and optimisation.You can also track exactly how many input and output tokens there were for each request. So there is complete transparency.>how to utilize the AI/LLM in such a way that is efficient and burns the least amount of tokensVarious tools help with this, e.g. caching, compaction, use of tool calls, use of sub-agents...
>>539857256you can also grow vegetables at home you fuckwit. stop buying from big grocery store faggot
>>539859534i dont eat vegetables
>>539856043Maybe it just sucks and doesn’t live up to a fraction of the sum of its parts.
>>539856043Tokenization reduces the required compute time. Going by character requires a compute pass for every single character. You would be paying significantly more if it was by the character. If you can't wrap your head around tokenization you have no business using anything beyond the free web endpoint anyway.You're just a retard looking for things to rage about. A tiny angry shitskin
>>539859893You don't have any reading comprehension.
This AI shit is hella gay.
Ignore the bots here. I was looking through the archives I've found some posts you might be interested in. This anon had the right ideas just like you original thinking humans. https://archive.4plebs.org/pol/thread/539447099/#539457056It’s only going to get worse. If you want to actually discuss things consistantly in a coherent manner - talk about geopolitics, history & conspiracy theories, finance and esoteric metaphysical ideas, places like image board where it's open to everyone is no longer an isloated layer where you can just laugh at normiefags. You are not observing it, you are being observed. If you're getting tired of talking to same-shit-spam-agitprop-algo-bots, anons will have to break the covenant for a while - a chat format not image boards again. /poI/ is only generals and they're bunch if circlejerk chat room on public web. But Discord & Telegram are namefags and privacy nightmare. SX doens't require your phone number to join and the first messaging app without user identifier. https://smp2.simplexonflux.com/g#bfkLxG6JUsC9QQ8dkkDZ0CDyc1z--q2ctuxlm-9Rn60
