If it generates “I ate” and the next word can be “a” or “an”, then it will just generate one or the other based on how often they appear after “I ate”. It hasn’t decided by this point what it has eaten. After it has generated the next token, for example “I ate an”, then its next token is now limited to food items that fit the grammatical structure of this sentence so far. Now it can decide: did I eat an apple? An orange? An awesome steak? etc
To be overly simple about it, the LLM uses statistics and a bit of controlled RNG to pick its words. Words in the LLM have links to each other with statistical probabilities attached. If you take the sentence “I fed a peanut to an elephant” and “I fed a peanut to a elephant” and then asked 100 people which is more correct, there will be a percentage which favors one over the other. Now with LLMs its not choosing using weighted coin flips, but rather picking the most likely next word (most of the time). So if the 100 people choose “an elephant” over “a elephant” 65% of the time in its training data, then the LLM will be inclined to use “an elephant.” However, Its important to know that the words around “an elephant” will also bias its choice to use the word ‘an’ for the word ‘elephant’.
Really, its largely based on the training data and the contexts to which ‘a’ and ‘an’ are used. Or in other words, the LLM knows because people figured it out for the LLM. People did all the thinking, LLM’s just use statistics on our bottled phrases to know when to use which. Of course, because it got its data from people - it will sometimes get it wrong which is based on how often people got it wrong.
Well, my example of the word ‘elephant’ has the same property as ‘herb’ where the use of ‘a’ or ‘an’ can depend on who you ask. I chose my example trying to anticipate this exact question, and I believe I gave you an answer.
Let me put it this way: it depends… It depends on the data the LLM (Chat GPT for example) has been given to train its output. If we have an LLM dataset which uses only text by people in the United Kingdom, then the data will favor “a herb” as the ‘h’ is pronounced, where data from the United States will favor the other way as the ‘h’ is usually silent when spoken out loud.
As a fairly general rule, people use the article “an” before a vowel sound (like a silent “h”) and “a” before a consonant sound (like a pronounced, or aspirated, “h”). Usually the data gathered is from multiple English speaking countries, so both “an herb” and “a herb” will exist in the training data, and from there the LLM will favor picking the one that is shown more often (as the data will biased.)
Just for fun, I asked the LLM running on my local machine. Prompt: "Fill in the blank: “It is _ herb” Response: “It is an herb.”
GPT creates plausible looking sentences, it has no concept of truth or anything like that. Since if you have an “an” it’s overwhelmingly likely that the next word will begin with a vowel it will choose one which plausibly fits with the corpus of text that came before. Likewise for an “a”.
There is no compromise in ability. It doesn’t have anything to “say” or whatever. What it produces is more like nonsense poetry than speech.
They’re very good at predicting the next word, so their choice of “a” or “an” is likely to make sense in context. But you can absolutely ask a GPT to continue a sentence that appears to use the wrong word.
For instance, I just tried giving a GPT this to start with:
My favorite fruit grows on trees, is red, and can be made into pies. It is a
And the GPT finished it with:
delicious and versatile fruit called apples!
So as you can see, language is malleable enough to make sense of most inputs. Though occasionally, a GPT will get caught up in a nonsensical phrase due to this behavior.
I think when people say it’s only predicting the next word, it’s a bit of an oversimplification to explain that the AI is not actually intelligent. It’s more or less stringing words together in a way that seems plausible.
It depends on whether the light is within a medium or just in vacuum. Afaik light in vacuum behaves entirely linear (so waves of different frequencies don’t interact). But there are materials where light does indeed interact with light of different frequencies. One effect like this is so-called four-wave mixing. https://en.wikipedia.org/wiki/Four-wave_mixing?wprov=sfla1
In general you can take a look at non linear optics
I will! Thank you! Also, it’s super fun that there’s exceptions based on the medium; I had no idea. I was picturing air or vacuum when I conceived of the original question, so now I have other things to look into!
I second the other poster’s suggestion to look into nonlinear optics. A really common application is frequency doubling, also known as second harmonic generation, which doubles the energy of the photons. So an 800 nm laser (red) can be converted to 400 nm (green) with this method.
Another optics-based phenomenon that I think maybe strays too far from the intent of your initial question, but is too cool not to share, is laser Wakefield acceleration. A very high power laser pulse will push electrons out of its path in plasma or materials via the ponderomotive force. This charge separation creates electric field gradients on the order of billions of volts per centimeter, which can accelerate electrons or other charged particles to relativistic energies. So you can start with a green laser pulse and wind up producing gamma-ray beams, either by slamming the electrons into a stopping material or by Compton scattering other low energy photons off the relativistic electrons.
I really appreciate the extra info! It’s fascinating.
I’m recently an ex-fundie, so learning about all the cool stuff happening in science is like finding out your childhood house has a million secret rooms you never knew about.
It all depends on what you mean by affect. Two em waves in the same space will have a different overall amplitude at any frequency.
If you mean as in the overall color of light, that will change based on how much/what frequency waves are combined. Think about adding a bit of black sand to a jar of white sand – from a distance it will appear grey but the actual colors of individual grains of sand (frequency of “individual” em waves) won’t change.
For wifi, data transmission is via phase modulation of the em wave, so the signal is resilient against adding different frequencies/amplitudes but may suffer if the same frequency is transmitted at a different phase.
No, because frequency is a fundamental quality of a wave. Things that are affected by one frequency could also be affected by another in certain cases (such as when two frequencies are integer multiples), but the waves themselves will never affect each other.
As an analogy, imagine playing a specific note on a flute. It doesn’t matter how many other notes you play, that original note will always be there alongside the others.
Thanks for the analogy! It’s pretty easy to understand how that works. I think I was imagining that EM waves shared some qualities with mechanical waves like sound, but I suppose that’s not the case!
Particles are just a way of looking at excited quantum fields. The Higgs field is always everywhere, giving things mass.
Honestly, depending on interpretation of quantum mechanics, you don’t need to acknowledge particles exist at all. It could all be fields becoming ever more entangled and wrinkled.
Photons are also bosons, right? Why do we need all the huge energy particle smashing experiment at LHC, while we can get any energy photons everywhere? What’s the difference?
Its better to not think of it as something we created in a lab. Higgs plays a part in making nature do what it does.
If you want to learn more about the Higgs Mechanism, check out this video from PBS Space Time. You might also find some good info in the comments as well.
PBS space time, as user friendly as it is, constantly misrepresents the nature of reality for the convenience of explanation, just as a grain of salt. Not saying it’s a bad channel, just saying it’s supposed to be entry level.
Definitely. There’s always whole swathes of nuance and you have to do that. Even so I still find some of it hard to follow.
Similar to viascience. Great introductory material that gets harder and harder the deeper you go.
Which, to me, just speaks of the incredible depth of knowledge we have and astounds me that we figured out as much as we have as it gets less and less intuitive.
I’m bad at regurgitating information that I’ve only quasi understood from afar, but I will tell you the “Chris the Brain” on YouTube has some fantastic videos about things like this. I know he’s covered it in a few videos. They are long, but he’s SUCH a good teacher. That man is an absolute fucking genius, mark my words.
You’re in for SUCH a wild ride!! I’m so excited for you. He’s opened my mind SO MUCH, and the theory he’s working on is going to be revolutionary I suspect.
The Higgs boson isn’t an atom like plutonium, it’s “further down”. I think of it in levels:
atoms, which are made up of
electrons “orbiting” the nucleus, the nucleus being made up of protons and neutrons. In turn, protons and neutrons are made up of
quarks
Quarks are a kind of elementary particle called fermions, which are at the same level as bosons (and electrons). Down here it’s all weird and quantum but in an oversimplified nutshell, it’s not so much that they physically exist as that in the maths* we can treat them as existing which makes it easier to think about.
of the physics models we use
I’m a computer scientist, not a real scientist, so I stand ready to be corrected by those more knowledgable.
edit: @SzethFriendOfNimi is more knowledgable and helped me fix this up a bit.
The fermions are particles with mass, an electron is already a fundamental fermion and not made up of quarks like protons and neutrons. The fundamental bosons (as far as I know) are particles that “handle” the interactions between other particles for instance gluons enable the strong force, while W and Z Bosons enable the weak force.
I believe the fundamental Higgs boson does occur in nature but likely immediately decays. (if I’m wrong I’d love to know how it actually enables certain interactions in nature)
Also I’m not studying quantum physics so I wouldn’t be surprised if someone needs to correct me. :)
Edit: clarified when fundamental fermions/bosons were meant.
Yup, should’ve clarified that I meant fundamental bosons, as any particle with integer spin is considered bosonic, while particles with half integer spin are fermionic, fundamental bosons alone still can’t make up matter though and protons/neutrons are fermionic.
Sorry, can’t answer your question. Quick correction though, uranium is the highest atomic number that occurs nationally.
Edit: so I’m wrong about this. In school we learned that it was uranium and that’s also what it said when I checked sources, but not enough. Apologies.
askscience
Hot
This magazine is from a federated server and may be incomplete. Browse more on the original instance.