@ShadowJonathan@tech.lgbtApple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
@ShadowJonathanNot too surprising, but a big weight like Apple standing behind this should shift sentiment more and more
@elexia@catcatnya.com@ShadowJonathan we knew this just based on how they arrive at answers. how can you reason if you don't understand the words you're using?
@elexia @ShadowJonathan Because OpenAI and co. marketed them as it. And many, many people swallowed it raw.
The technological realities of LLM have always been secondary in this AI rush, it's all been about marketing, and Oh boy has it worked.
It will take a long time and a lot of demonstrations, including the simplest ones, for the world to understand their mistakes and pull out of this impasse. Or the next revolutionary, distributive technology in the Silicon Valley cycle will hijack all the fund.
Now, it's a matter of proving that the ad was misleading, and that is harder than creating a pretty beautiful lie.
@ShadowJonathan the sentiment in some groups is turning to "what duct tape on top of these gives us the right stuff?" with some folks even discovering cognitive architecture as a solution; I'll take what I can get I guess.
Doesn't surprise me to see Marcus reporting this. I'll read it closely after work. Thanks.
This is not surprising at all and I don't understand why anyone had to waste time and resources on demonstrating a self-evident fact that was known before the research even started.
@lulu @ShadowJonathan Self-evident facts are not always true. That is why research to explore them and prove (or disprove) them is really important.
This is showing that the ability of LLMs to respond in a way that makes sense is very limited.
And the mathematical aspects are important - it is common that some numeric aspect is part of a question. And people trust LLMS far too much. Especially for things that they don't fully understand.
@ShadowJonathan Maybe now that *big tech company* not *independent, well-respected researchers* has told companies what they need to know, maybe they'll actually listen?
Maybe?
@ShadowJonathan it’s really weird that some people are pushing LLM as something that can reason, while its architecture is Key-Value storage with sophisticated probabilistic query and value encoding mechanisms.
LLMs just don’t have enough layers for anything besides queries, so it can’t have any relational capabilities that allow to make multi step decisions.
Also tokenization hides a lot of structure of the language from encoding process, which adds additional source of errors.
I’m sure we can build something that can reason at some point, but it requires very diffirent and more complex architecture.
@technobaboo@ordinary.cafe@alex @ShadowJonathan it's definitely not key-value storage, it's more latent space interpolation than anything... tokenization isn't that bad either, it's mostly just that there's no depth like you mentioned!
@technobaboo @ShadowJonathan but isn’t latent space interpolation just basically maps some input to some output? Input embeddings always return the same next most probable token? Sure, underlying structure is more complex with all this encodings, decoding and attention layers, but on a surface you just putting some query in and getting some value out from the constant storage. And we have no idea how to reliably write or read from that storage, but that’s entirely diffirent problem :D
@technobaboo@ordinary.cafe@alex @ShadowJonathan it's definitely a mapping function for sure but key value storage implies a direct mapping via table when it's way less linear than that
@ShadowJonathan Next up, water is wet. Surprising research by Apple. If only Apple did their research earlier, the Oceangate disaster could have been prevented.
@yassie_j@labyrinth.zone@ShadowJonathan if they can’t reason then they aren’t sapient
@ShadowJonathan seeing some folks in the replies on the article, they're not responding to the assertions of the article, but instead about it more broadly, and making incorrect ones there too. "Occasional error is not fatal" but when I ask something to summarize an article, if it grossly misrepresents the article, why should I have to rerun it to make sure it's accurate without me reading it first? "Um but it's good at other things!" The article didn't state otherwise, it was specifically about reasoning, there is no need for the defensiveness.
@ShadowJonathan Why would we judge LLMs on their ability to solve complex tasks? The interesting thing is if they can solve simple tasks well enough to be useful.
@anderspuck @ShadowJonathan it was given a simple task, and failed
@beeoproblem@mastodon.gamedev.place@anderspuck @ShadowJonathan because they're being sold as if they can solve complex tasks
LLMs can use a prompt to generate text based off of a huge pile of content produced by other people. Sometimes that text is an exact copy of the original text. They may "solve" a problem if the solution is contained in their training data and your prompt is able to retrieve it.
They're a (very) improved version of a Markov chain. Not a problem solver of any sort
@anderspuck @ShadowJonathan Which they also can't do.
@dalias @ShadowJonathan They can absolutely do certain things well enough to be useful. Create a fairly accurate transcript of a podcast, for example.
@anderspuck @dalias @ShadowJonathan Weeeeeeelllll.... "fairly" & "useful" are pretty load-bearing here - like, yes, they can, but they still make the sort of errors that completely change the meaning of the content & there's no way to check for it except human proofreading, which itself is unreliable at low-cost scale (i.e. a non-specialist low-paid worker checking many texts at a fast pace). Suffice to say that even for this, LLMs are wildly oversold.
@jwcph @dalias @ShadowJonathan I used an LLM to create a first draft of the transcript here, for example. Without that help there just wouldn’t be any transcript because it would take too much time. So that for me is definitely in the category of “useful”.
https://www.logicofwar.com/why-did-experts-fail-to-predict-russias-invasion-of-ukraine/
@anderspuck @dalias @ShadowJonathan Sure - now all you have to figure out is how much you'd pay for that usefulness, because this is only happening to become an extremely lucrative business for somebody.
(no, that's not a different topic; the problem complex here is functionality + usefulness + environmental impact + business model)
@jwcph Let’s see how it develops. Ollama is working great for me, but it does require a fairly good computer. So yes, either taht processing power has to be local with the user or somewhere centralized.
@anderspuck @dalias @ShadowJonathan
LLMs are NOT doing *speech to text* translation -- doing transcripts from audio (podcast). That's a different set of AI technologies.
The industry has been developing "AI" technologies since before I was born. Some are quite useful.
It's the "Generative AI" subset (which includes LLMs, chatbots) that is so misleading, mostly useless, and incredibly wasteful.
@JeffGrigg @anderspuck @ShadowJonathan This. 
The industry is all about muddling these differences so they can use the utility of one thing to justify a different piece of garbage they want to sell.
@JeffGrigg @dalias @ShadowJonathan True. I kind of bundled ChatGTP and Whisper in that statement.
I don’t find generative AI useless, though. There are many tasks for which it is very good, but probably not those flashy ones many people are thinking about. For example an LLM is much better at sentiment analysis than older methods.
@anderspuck @JeffGrigg @ShadowJonathan Are you sure about that? I'm pretty sure they do an extremely racist version of "sentiment analysis".
@dalias@hachyderm.io @anderspuck@krigskunst.social @JeffGrigg@mastodon.social @ShadowJonathan@tech.lgbt Let’s be clear, the LLM is not developing racism out of nowhere. It is just able to amplify racial bias in the dataset. The stuff used to train it was already racist. It’s extremely hard to filter that out. I still laugh at tip culture being ingrained into LLMs. Some would do "better work" than normal if you bribed it with a tip. Freaky stuff.
@anderspuck @ShadowJonathan 
Sure, let’s build a nuclear power plant to give enough power for this AI to do something simple a drunk human could do at 3am on just the power of a half-bag of Cheetos. LLM is crap, and wastes obscene amounts of water and fossil fuels to power it. Everybody should just back away from it, slowly until it dies.
@ShadowJonathan@anderspuck because they're expected to solve complex tasks, they're being sold as if they can solve complex tasks, and that they have a fail and error rate enough that they're not safe
They want these things to drive cars and make decisions that involve human lives.
@ShadowJonathan @anderspuck Not to mention they are insanely expensive to operate. The cost-to-benefit ratio is not sustainable, even for most of the tasks they *can* do.
@faoluin @ShadowJonathan Isn’t that more a question about green energy transition than about LLMs as such?
@anderspuck @ShadowJonathan No, it doesn't matter what kind of energy they're consuming, because energy always has a cost to produce, and again the cost-to-benefit ratio isn't there. LLMs are creating scarcity for relatively little actual positive benefit.
It's also not strictly about power; the same arguement applies to water consumption as well.
There is a huge difference between LLM and the automation used for cars. How one of the two behaves cannot be used to draw any conclusions about the other.
@kasperd @anderspuck @ShadowJonathan
Dittos! I was about to post the same thing.
The industry has been developing "AI" technologies since before I was born. Many work quite well, and are useful. Some save money. Some save lives.
You probably interact with "traditional" AI systems far more often than you realize.
Each has to be evaluated based on its costs and benefits and risks.
Generative AI / LLMs Chatbots are a dangerous wasteful SCAM.
Self-driving cars are still "iffy."
Self-driving cars are iffy, but human driven cars are dangerous. A self driving car might already be safer than one driven by a human.
The hard question is what will people choose if they are given the choice between two accidents that can be blamed on human drivers or one accident with a self-driving car where there isn't anyone to blame.
@avakining@hachyderm.io@kasperd The solution to cars being dangerous isn’t to replace the drivers. It’s to get rid of the cars in as many places as possible to replace them with free public transit, bikes, and other safer and greener options
By all means let's have better public transit. It doesn't need to be free in order to have an effect. How well it functions is a more important factor in getting people to use it than the price.
@avakining@hachyderm.io@kasperd Sure, but if it’s free at least you don’t get cops shooting people to death over a $3 fare (as happened in NYC less than a month ago)
@anderspuck As long as you can define "simple" as it applies to the LLM because some basic stuff it cannot do due to the lack of cognition. When I say that it cannot do these things I mean it needs to be able to do said tasks unaided (as in no further prompts or guiding prompts) with complete accuracy, if it can't do those things then it's not doing the task even if it's partially right.
@halva@labyrinth.zone
@halva @ShadowJonathan yeah, I appreciate the demonstrations, but this feels a little like, "New study confirms bicycles cannot fly."
@rubenerd@bsd.networkCompanies like OpenAI and their defenders claim generative AI can reason, learn, etc. We know it’s nonsense, but it’s still extremely important it gets called out.