I would think that, more than anything else, the issue would be more getting it through all the bureaucratic red tape. See the ESB debacle:
Weaver had been brought to Raytheon, the company the Air Force had hired to write the software for the next generation GPS satellites, because the Raytheon team was behind schedule and over budget. This issue of data transmission to the ground stations and back again was one of a few problems that was holding them back. There is an industry standard way of doing this, a simple, reliable protocol that is built into almost every operating system in the world.
But this team wasn’t using this simple protocol on its own. Instead, the team had written a piece of software to receive the message from that protocol, read the data, and then recode it into a different format, so they could feed it into a very complex piece of software called an Enterprise Service Bus, or ESB. The ESB eventually delivered the data to yet another piece of software, at which point the whole process ran in reverse order to deliver it back to the original, simple protocol. Because the data was taking such a roundabout route, it wasn’t arriving quickly enough for the ground stations to make the calculations needed. Using the simple protocol alone would have made the entire job a snap—as easy as nailing a couple of boards together. Instead, they had this massive Rube Goldberg contraption that was never going to work.
The people on this project knew quite well that using this ESB was a terrible idea. They’d have been relieved to just throw it out, plug in the simple protocol, and move on. But they couldn’t. It was a requirement in their contract. The contracting officers had required it because a policy document called the Air Force Enterprise Architecture had required it. The Air Force Enterprise Architecture required it because the Department of Defense Enterprise Architecture required it. And the DoD Enterprise Architecture required it because the Federal Enterprise Architecture, written by the Chief Information Officers Council, convened by the White House at the request of Congress, had required it.
I’m sure some of the fine folks at 18F would love to help various US agencies or state governments with migrating to Mastodon. I’m not so sure any of them would be able to convince geriatric politicians to do so.
I’ve been trying to help out with the app’s development and has been in a few conversations with the lead dev. Top notch dude! Has a really good vision for the app and is very good in handling all the community discussion.
My city had ranked choice voting implemented by Democrats in the 1970s. They elected the first black mayor, who is still one of our most beloved mayors in the city’s history, under RCV.
Then Republicans made it illegal at a state level when they had a trifecta. Democrats keep introducing bills at the state level to allow RCV, and Republicans take more and more drastic action against it. So yeah… I want more Democrats in my state government so we can have RCV.
Because I didn’t think it particularly important for countering this same old tired BS. But if you really want to know, it’s Ann Arbor, MI and the mayor was Albert Wheeler.
(Edit and fun fact: I’m writing this from Wheeler Park right now ᕕ( ᐛ )ᕗ)
This process is akin to how humans learn by reading widely and absorbing styles and techniques, rather than memorizing and reproducing exact passages.
Like fuck it is. An LLM “learns” by memorization and by breaking down training data into their component tokens, then calculating the weight between these tokens. This allows it to produce an output that resembles (but may or may not perfectly replicate) its training dataset, but produces no actual understanding or meaning–in other words, there’s no actual intelligence, just really, really fancy fuzzy math.
Meanwhile, a human learns by memorizing training data, but also by parsing the underlying meaning and breaking it down into the underlying concepts, and then by applying and testing those concepts, and mastering them through practice and repetition. Where an LLM would learn “2+2 = 4” by ingesting tens or hundreds of thousands of instances of the string “2+2 = 4” and calculating a strong relationship between the tokens “2+2,” “=,” and “4,” a human child would learn 2+2 = 4 by being given two apple slices, putting them down to another pair of apple slices, and counting the total number of apple slices to see that they now have 4 slices. (And then being given a treat of delicious apple slices.)
Similarly, a human learns to draw by starting with basic shapes, then moving on to anatomy, studying light and shadow, shading, and color theory, all the while applying each new concept to their work, and developing muscle memory to allow them to more easily draw the lines and shapes that they combine to form a whole picture. A human may learn off other peoples’ drawings during the process, but at most they may process a few thousand images. Meanwhile, an LLM learns to “draw” by ingesting millions of images–without obtaining the permission of the person or organization that created those images–and then breaking those images down to their component tokens, and calculating weights between those tokens. There’s about as much similarity between how an LLM “learns” compared to human learning as there is between my cat and my refrigerator.
And YET FUCKING AGAIN, here’s the fucking Google Books argument. To repeat: Google Books used a minimal portion of the copyrighted works, and was not building a service to compete with book publishers. Generative AI is using the ENTIRE COPYRIGHTED WORK for its training set, and is building a service TO DIRECTLY COMPETE WITH THE ORGANIZATIONS WHOSE WORKS THEY ARE USING. They have zero fucking relevance to one another as far as claims of fair use. I am sick and fucking tired of hearing about Google Books.
EDIT: I want to make another point: I’ve commissioned artists for work multiple times, featuring characters that I designed myself. And pretty much every time I have, the art they make for me comes with multiple restrictions: for example, they grant me a license to post it on my own art gallery, and they grant me permission to use portions of the art for non-commercial uses (e.g. cropping a portion out to use as a profile pic or avatar). But they all explicitly forbid me from using the work I commissioned for commercial purposes–in other words, I cannot slap the art I commissioned on a T-shirt and sell it at a convention, or make a mug out of it. If I did so, that artist would be well within their rights to sue the crap out of me, and artists charge several times as much to grant a license for commercial use.
In other words, there is already well-established precedent that even if something is publicly available on the Internet and free to download, there are acceptable and unacceptable use cases, and it’s broadly accepted that using other peoples’ work for commercial use without compensating them is not permitted, even if I directly paid someone to create that work myself.
If you put a gazillion monkeys on a typewriter they can write Shakespeare.
This is a mathematical curiosity borne out of pure randomness. An LLM trained on a dataset to generate similar content is quite the opposite of randomness.
But they all explicitly forbid me from using the work I commissioned for commercial purposes
I fear the courts will side with the tech companies on this as regardless of how illegal or immoral a certain act is, if you do it on a large enough scale it becomes “okay” again in the eyes of the system. Genocide, large scale fraud, negligent financial actions, pollution/poisoning, etc. You dump toxic chemicals into one person’s cup and you get the book thrown at you. You dump toxic chemicals into an entire city’s water supply and you pay a paltry fine that is never enough to seriously damage the company because that’s bad for the economy.
I recently visited a museum and i really loved it. Getting up close to an image and seeing none of the fuzziness, no AI “shimmer” on photos and every stroke made sense (as in you could see that an arm moved a brush and you could see the path it took etc.). Hands made sense. And while tryptichons were not exactly precise when it comes to the anatomy of humans, no humans had anything smeared etc.
Like fuck it is. An LLM “learns” by memorization and by breaking down training data into their component tokens, then calculating the weight between these tokens.
But this is, at a very basic fundamental level, how biological brains learn. It’s not the whole story, but it is a part of it.
there’s no actual intelligence, just really, really fancy fuzzy math.
You mean sapience or consciousness. Or you could say “human-level intelligence”. But LLM’s by definition have real “actual” intelligence, just not a lot of it.
Edit for the lowest common denominator: I’m suggesting a more accurate way of phrasing the sentence, such as “there’s no actual sapience” or “there’s no actual consciousness”. /end-edit
an LLM would learn “2+2 = 4” by ingesting tens or hundreds of thousands of instances of the string “2+2 = 4” and calculating a strong relationship between the tokens “2+2,” “=,” and “4,”
This isn’t true. At all. There are math specific benchmarks made by experts to specifically test the problem solving and domain specific capabilities of LLM’s. And you can be sure they aren’t “what’s 2 + 2?”
I’m not here to make any claims about the ethics or legality of the training. All I’m commenting on is the science behind LLM’s.
Fine, you win, I misunderstood. I still disagree with your actual point, however. To me, Intelligence implies the ability to learn in real-time, to adapt to changes in circumstance, and for self-improvement. Once an LLM is trained, it is static and unchanging until you re-train it with new data and update the model. Even if you strip out the sapience/consciousness-related stuff like the ability to think critically about a scenario, proactively make decisions, etc., an LLM is only capable of regurgitating facts and responding to its immediate input. By design, any “learning” it can do is forgotten the instant the session ends.
It’s not a competition, but I genuinely respect you for saying you misunderstood.
Once an LLM is trained, it is static and unchanging until you re-train it with new data and update the model.
Absolutely! I honestly think this is the main thing (or at least one of the main things) that prevent human-level intelligence or even sentience in LLM’s.
Think about how our minds work. From the moment we’re born (really, it’s way before that) our brains are bombarded with input and feedback from every sense. It takes a person many months of that to start recognizing things. That’s also why babies sleep so much, their brains are kinda “training” and growing fast. Organizing all the data into memories.
Side bar: this is actually what dreams are. Dreams are emotions, thoughts, ideas, or whatever concept a neuron or group of neurons are associated with getting triggered. When we dream it’s our brain taking the days inputs and building new connections. The neural connections in our brains are very much like weights and feed-forward process of neural activation is near identical to how artificial neural networks function. They aren’t called “artificial neural networks” for no reason.
Here’s a useful graphic that shows things that make up “intelligence”
A very basic definition of intelligence is “the ability to solve problems or make decisions”.
I think the term is just often misused in common parlance so often that people start applying in a scientific setting incorrectly. Kinda how people used to call an entire computer the CPU, which like the word intelligence everyone understands what’s being said, but it’s factually wrong.
Same thing today when people say “I bought a new GPU” when they should say “I bought a new video card” as the GPU is just a component.
I love that to make their point with a good amount of examples, they had to mirror image 1 to make image 3. Solid statistical analysis in practice there. The work of a professional.
I’d say we have a general problem in the US with certain people and legal entities constantly acting in bad faith but still having their words taken seriously.
I’m sure it’s a human issue across the globe, but we seem to be especially bad about it when it comes to rich people and corporations.
lemmy.world
Top