We’ve been spending decades curating our perception by management in order to make sure we all have jobs. He’s gonna ruin the whole industry if we don’t shut him the hell up
In my first month at my current employer, I added some temporary code with a TODO to fix it properly. That was 11 years ago in 2013, and the same TODO is still there today, and these days it’d be significantly harder to do it. 😂
Okay, here we go. I’m going to spit out some bullshit and home someone corrects me if I’m wrong. I’ve looked for some explanations and this is what I’ve gotten.
Are you ready?
The Factory Pattern.
My understanding is that the purpose is a function to return any of several types of objects, but a specific type, not just an interface or whatever they might all inherit from.
I think most languages now have something like a “dynamic” keyword to solve this issue by allowing determination of the type only at runtime. (To be used with extreme caution.)
But most of the time I see the Factory pattern, it’s used unnecessarily and can only return one specific type. Why they would use a Factory pattern here and not just a plain old constructor confounds me.
Yeah most uses of the factory pattern are unnecessary and it’s mild code smell IMO. If your factory only returns one type you should definitely just use that type’s constructor.
Factory pattern can return a mock type for testing or a production type, as needed, which makes it possible to unit test the code that uses the produced object.
This quick guide explains it well. Then it improves on it by explaining dependency injection.
Even if you do design clean modular code and document it, you’re getting a question a year from now about how it works, or someone just duct tapes on top of it.
I’ve even experienced this in the 3D printing community, where I design a highly parametric model and put lots of effort into making all of the major dimensions and qualities parameterized and dynamically adjustable, with lots of bounds checking and value clamping, with all the parameters at the top of my scad file with comments explaining what each variable does.
And then someone comes along to remix my model, says I don’t want to install openscad, and just scales the entire output stl to change the dimensions, squashing all the features of the model in the process (instead of having the size gracefully adjust with all the features moving around to account), and leaving anybody starting from their work with a hard to remix mesh with no parameters.
Working with devs who aren’t familiar with design patterns, introducing design patterns by simply implementing them (in a new project) allows it easier for the devs to follow the implementation as examples, even though they aren’t necessarily familiar with the design pattern concepts.
At least they can observe the patterns and replicate the patterns elsewhere.
In my experience, this often doesn’t happen. So many developers are either inexperienced or cowboys, and there’s nothing inherently wrong with either. But at places where projects are small and numerous, teams often end up with nothing but a combination of the two.
As one of our office’s engineering “fixers”, I’ve taken over maintenance of several such projects. They usually have shattered remnants of code taken from other projects, open source libraries, internal libraries, stack overflow, and so on. Whole source files copied into the project, modified in ways that introduce impressive new failure cases while failing to add new functionality, and used in ways that completely ignore the features natively implemented in that code while those same features are bodged in as barely-working piles of if statements, balanced on a knife’s edge to avoid triggering the failure modes added by the project’s modifications of the copied code. I’m usually able to purge 20-30k lines of code from such projects in the first month, simultaneously closing multiple outstanding issues the PM had been led to believe were intractable.
That probably sounds like arrogance and/or shitting on everybody else’s work but it’s just reality at many workplaces due to a pace driven by unreasonable expectations from management. I just happen to be the person here that ends up sifting through the wreckage when a project reaches the inevitable osteoporosis phase, because of a natural disposition for reverse engineering. It would be great to escape for this and other reasons, as far as I can tell, most places aren’t that different.
This can work for junior devs who aren’t stuck in their ways. Unfortunately there are too many “senior” devs who are happy making crap. It’s hard to fight them constantly to do things properly (e.g. write actual commit messages rather than just “Fix #836”) so using tools like linters where possible is definitely a big improvement.
Or business decides all specs and design decisions that were made last quarter were actually garbage and yes we do want to be able to manually override every step of the carefully designed state machine. We’d like to be able to manually change all calculated sales data, but also the data needs to remain in a consistent state at all times. Oh and while you’re there, we decided the commission calculations will use a different system from now on. We expect it to be online by the end of the week, thanks.
The ‘document’ part also seems to be insanely hit-or-miss from my amateur experience. Self-documenting design/code is… well, not. Auto-generated documentation is also usually just as bad IMO. Producing good documentation really is a skill in and of itself.
Also small personal opinion: If your abstraction layers or algorithms are based off a technical concept, you should probably attribute that concept and provide links to further research, to eliminate future ambiguity or in case your reader lacks that background. Future you will probably thank you and anyone like me who immediately gets lost in jargon soup will also be thankful.
My homeoffice setup is right next to a window, so it’s too bright for dark mode during the summer. So I work in light mode from about April-September and in dark mode for the rest of the year
Only people I ever have a problem with are Project Managers. I have had way more bad experiences with utterly psychotic PMs than PMs who are actually good at their job. Everybody else is super cool, but I swear all of you are alcoholics. At least Sales pays for the drinks?
Ohhh that’s me right now. I work in a consultancy and I only got assigned to projects that are on fire. It’s almost 24 months without a gap between projects. Help me ಥ_ಥ
Put your foot down, establish boundaries, and take a well deserved vacation with 0 communication to work while on it. Otherwise, I would start looking somewhere else. Your health is more important.
Edit: Also, hit them a few times with your Wabbajack for me.
“For those of us who are about to die, we salute you!”
I’m hoping you’re not just an employee of that consultancy, but a contractor instead, and that you charge a good hourly rate, considering the situation you’re in.
Yup, before I went into tech I worked at an architecture firm and we had this one absolutely amazing PM from Australia who was smart, a clear communicator, and so much more on top of his shit then any other PM, and he burnt out and quit and moved back to Australia after like 2 years because they just kept throwing him into the absolute biggest messes since he was clearly the best at cleaning them up.
He’s also the one who I got drunk at an airport bar with and just repeatedly urged me to leave the company and go somewhere well run … there were pretty clear signs he wasn’t enjoying his assignments.
In my experience it doesn’t matter. You have to regularly refactor your code to keep up with new features. The more often you can make time to do it the easier it is.
Unit test to help catch regressions. If you are confident in your test catching a good portion of bugs from refactoring, at least you feel confident refactoring. Worst case, at least you ensured your code is testable. There is nothing worse than refactoring untestable code.
Self-documenting code and when it fails to self-document, comments or refer to a wiki page.
C is the hardware language N°1 of the high-level languages. If you actually want to know and control what happens in the machine, you write in C. Rust, C++ and all the other abstractions are for people who do not understand how computers and computer memory work.
even if you write in assembly, you still may not actually understand what is going on in the machine since processors convert the instructions to “micro-ops”, and let’s not forget hardware bugs like those caused by speculative execution
I’ve written programs in C. I’ve written programs in assembly, for x86 and for microcontrollers. I’ve designed digital logic and programmed it into an FPGA. I’ve built digital logic circuits with transistors.
I’ll still take Go over C any day of the week. If I’m doing embedded, I’ll use TinyGo.
Programming languages are tools. I couldn’t care less about learning a new tool just for the sake of learning. My interest in learning tools is exclusively practical - if they help me do my work better.
I find functional languages interesting, but that’s because I find the underlying theory interesting and worth learning for its own sake, not because I actually care about the specific language it’s written in. Even then these days I’d rather learn about woodworking (which is currently my main hobby) than a programming paradigm I’m probably never going to use.
This is a misconception that’s common among beginner C programmers. They think C is high level assembly and don’t understand the kinds of optimisations modern compilers make. And they think they’re hardcore and don’t make mistakes.
What I think is amazing about LLMs is that they are smart enough to be tricked. You can’t talk your way around a password prompt. You either know the password or you don’t.
But LLMs have enough of something intelligence-like that a moderately clever human can talk them into doing pretty much anything.
That’s a wild advancement in artificial intelligence. Something that a human can trick, with nothing more than natural language!
Now… Whether you ought to hand control of your platform over to a mathematical average of internet dialog… That’s another question.
They’re not “smart enough to be tricked” lolololol. They’re too complicated to have precise guidelines. If something as simple and stupid as this can’t be prevented by the world’s leading experts idk. Maybe this whole idea was thrown together too quickly and it should be rebuilt from the ground up. we shouldn’t be trusting computer programs that handle sensitive stuff if experts are still only kinda guessing how it works.
Not even close to similar. We can create rules and a human can understand if they are breaking them or not, and decide if they want to or not. The LLMs are given rules but they can be tricked into not considering them. They aren’t thinking about it and deciding it’s the right thing to do.
We can create rules and a human can understand if they are breaking them or not…
So I take it you are not a lawyer, nor any sort of compliance specialist?
They aren’t thinking about it and deciding it’s the right thing to do.
That’s almost certainly true; and I’m not trying to insinuate that AI is anywhere near true human-level intelligence yet. But it’s certainly got some surprisingly similar behaviors.
Have you heard of social engineering and phishing? I consider those to be analogous to uploading new rules for ChatGPT, but since humans are still smarter, phishing and social engineering seems more advanced.
And one property of actual, real-life human intelligence is “happenning in cells that operate in a wet environment” and yet it’s not logical to expect that a toilet bool with fresh poop (lots of fecal coliform cells) or a dropplet of swamp water (lots of amoeba cells) to be intelligent.
Same as we don’t expect the Sun to have life on its surface even though it, like the Earth, is “a body floating in space”.
Sharing a property with something else doesn’t make two things the same.
There is no logical reason for you to mention in this context that property of human intelligence if you do not meant to make a point that they’re related.
So there are only two logical readings for that statement of yours:
Those things are wholly unrelated in that statement which makes you a nutter, a troll or a complete total moron that goes around writting meaningless stuff because you’re irrational, taking the piss or too dumb to know better.
In the heat of the discussion you were trying to make the point that one implies the other to reinforce previous arguments you agree with, only it wasn’t quite as good a point as you expected.
I chose to believe the latter, but if you tell me it’s the former, who am I to to doubt your own self-assessment…
No, you leapt directly from what I said, which was relevant on its own, to an absurdly stronger claim.
I didn’t say that humans and AI are the same. I think the original comment, that modern AI is “smart enough to be tricked”, is essentially true: not in the sense that humans are conscious of being “tricked”, but in a similar way to how humans can be misled or can misunderstand a rule they’re supposed to be following. That’s certainly a property of the complexity of system, and the comment below it, to which I originally responded, seemed to imply that being “too complicated to have precise guidelines” somehow demonstrates that AI are not “smart”. But of course “smart” entities, such as humans, share that exact property of being “too complicated to have precise guidelines”, which was my point!
I don’t want to spam this link but seriously watch this 3blue1brown video on how text transformers work. You’re right on that last part, but its a far fetch from an intelligence. Just a very intelligent use of statistical methods. But its precisely that reason that reason it can be “convinced”, because parameters restraining its output have to be weighed into the model, so its just a statistic that will fail.
Im not intending to downplay the significance of GPTs, but we need to baseline the hype around them before we can discuss where AI goes next, and what it can mean for people. Also far before we use it for any secure services, because we’ve already seen what can happen
Oh, for sure. I focused on ML in college. My first job was actually coding self-driving vehicles for open-pit copper mining operations! (I taught gigantic earth tillers to execute 3-point turns.)
I’m not in that space anymore, but I do get how LLMs work. Philosophically, I’m inclined to believe that the statistical model encoded in an LLM does model a sort of intelligence. Certainly not consciousness - LLMs don’t have any mechanism I’d accept as agency or any sort of internal “mind” state. But I also think that the common description of “supercharged autocorrect” is overreductive. Useful as rhetorical counter to the hype cycle, but just as misleading in its own way.
I’ve been playing with chatbots of varying complexity since the 1990s. LLMs are frankly a quantum leap forward. Even GPT-2 was pretty much useless compared to modern models.
All that said… All these models are trained on the best - but mostly worst - data the world has to offer… And if you average a handful of textbooks with an internet-full of self-confident blowhards (like me) - it’s not too surprising that today’s LLMs are all… kinda mid compared to an actual human.
But if you compare the performance of an LLM to the state of the art in natural language comprehension and response… It’s not even close. Going from a suite of single-focus programs, each using keyword recognition and word stem-based parsing to guess what the user wants (Try asking Alexa to “Play ‘Records’ by Weezer” sometime - it can’t because of the keyword collision), to a single program that can respond intelligibly to pretty much any statement, with a limited - but nonzero - chance of getting things right…
This tech is raw and not really production ready, but I’m using a few LLMs in different contexts as assistants… And they work great.
Even though LLMs are not a good replacement for actual human skill - they’re fucking awesome. 😅
We do not have a rigorous model of the brain, yet we have designed LLMs. Experts of decades in ML recognize that there is no intelligence happening here, because yes, we don’t understand intelligence, certainly not enough to build one.
If we want to take from definitions, here is Merriam Webster
(1)
: the ability to learn or understand or to deal with new or trying >situations : reason
also : the skilled use of reason
(2)
: the ability to apply knowledge to manipulate one’s >environment or to think abstractly as measured by objective >criteria (such as tests)
The context stack is the closest thing we have to being able to retain and apply old info to newer context, the rest is in the name. Generative Pre-Trained language models, their given output is baked by a statiscial model finding similar text, also coined Stocastic parrots by some ML researchers, I find it to be a more fitting name. There’s also no doubt of their potential (and already practiced) utility, but a long shot of being able to be considered a person by law.
That statement of yours just means “we don’t yet know how it works hence it must work in the way I believe it works”, which is about the most illogical “statement” I’ve seen in a while (though this being the Internet, it hasn’t been all that long of a while).
“It must be clever statistics” really doesn’t follow from “science doesn’t rigoroulsy define what it is”.
I think the point is more that the word “intelligence” as used in common speech is very vague.
I suppose a lot of people (certainly I do it and I expect many others do it too) will use the word “intelligence” in a general non-science setting in place of “rationalization” or “reasoning” which would be clearer terms but less well understood.
LLMs easilly produce output which is not logical, and a rational being can spot it as not following rationality (even of we don’t understand why we can do logic, we can understand logic or the absence of it).
That said, so do lots of people, which makes an interesting point about lots of people not being rational, which nearly dovetails with your point about intelligence.
I would say the problem is trying to defined “inteligence” as something that includes all humans in all settings when clearly humans are perfectly capable of producing irrational shit whilst thinking of themselves as being highly intelligent whilst doing so.
I’m not sure if that’s quite the point you were bringing up, but it’s a pretty interesting one.
See, I understand that you’re trying to joke but the linked video explains how the use of the word dumber here doesn’t make any sense. LLMs hold a lot of raw data and will get it wrong at a smaller percent when asked to recite it, but that doesn’t make them smart in the way that we use the word smart. The same way that we don’t call a hard drive smart.
They have a very limited ability to learn new ways of creating, understand context, create art outside of its constraints, understand satire outside of obvious situations, etc.
Ask an AI to write a poem that isn’t in AABB rhyming format, haiku, or limerick, or ask it to draw a house that doesn’t look like an AI drew it.
A human could do both of those in seconds as long as they understand what a poem is and what a house is. Both of which can be taught to any human.
It’s a good video (I’ve seen it; very informative and accessible cannot recommend enough), but I think you each mean different things when you use the word “intelligence”.
Oh for sure! The issue is that one of those meanings can also imply sentience, and news outlets love doing that shit. I talk to people every day who fully believe that “AI” text transformers are actually parsing human language and responding with novel and reasoned information.
I was amazed by the intelligence of an LLM, when I asked how many times do you need to flip a coin to be sure it has both heads and tails. Answer: 2. If the first toss is e.g. heads, then the 2nd will be tails.
We already do! And on the cheap! I have a Coral TPU running for presence detection on some security cameras, I’m pretty sure they can run LLMs but I haven’t looked around.
GPT4ALL runs rather well on a 2060 and I would only imagine a lot better on newer hardware
It’s not. Whenever someone talks about how LLMs are just statistics, ignore them unless you know they are experts. One thing that convinces me that ANNs really capture something fundamental about how human minds work is that we share the same tendency to spout confident nonsense.
Well on one hand yes, when you’re training it your telling it to try and mimic the input as close as possible. But the result is still weights that aren’t gonna reproducte everything exactly the same as it just isn’t possible to store everything in the limited amount of entropy weights provide.
In the end, human brains aren’t that dissimilar, we also just have some weights and parameters (neurons, how sensitive they are and how many inputs they have) that then output something.
I’m not convinced that in principle this is that far from how human brains could work (they have a lot of minute differences but the end result is the same), I think that a sufficiently large, well trained and configured model would be able to work like a human brain.
Not an LLM specifically, in particular lack of backtracking and the network depth limits as well as interconnectivity limits sets hard limits on capabilities.
Humans have a completely different memory model and a in large part a very different way of linking together learned concepts to form their world view and to develop interdisciplinary skills, allowing us to solve many kinds of highly complex tasks as long as we can keep enough of it in our memory.
Why would averaging lead to repetition of stereotypes?
Anyway, it’s hard to say LLMs output what they do. GPTisms may have to do with the system prompt or they may result from the fine-tuning. Either way, they don’t seem very internet average to me.
The TLDR is that pathways between nodes corresponding to frequently seen patterns (stereotypical sentences) gets strengthened more than others and therefore it becomes more likely that this pathway gets activated over others when giving the model a prompt. These strengths correspond to probabilities.
Have you seen how often they’ll sign a requested text with a name placeholder? Have you seen the typical grammar they use? The way they write is a hybridization of the most common types of texts it has seen in samples, weighted by occurrence (which is a statistical property).
It’s like how mixing dog breeds often results in something which doesn’t look exactly like either breed but which has features from every breed. GPT/LLM models mix in stuff like academic writing, redditisms and stackoverflowisms, quoraisms, linkedin-postings, etc. You get this specific dryish text full of hedging language and mixed types of formalisms, a certain answer structure, etc.
B) you do know there’s a lot of different definitions of average, right? The centerpoint of multiple vectors is one kind of average. The median of online writing is an average. The most common vocabulary, the most common sentence structure, the most common formulation of replies, etc, those all form averages within their respective problem spaces. It displays these properties because it has seen them so often in samples, and then it blends them.
You should worry more about whether you have seen evidence that supports what you are saying. So, what kind of evidence do you want? A tutorial on coding neural nets? The math? Video or text?
Text explaining why the neural network representation of common features (typically with weighted proportionality to their occurrence) does not meet the definition of a mathematical average. Does it not favor common response patterns?
B) you do know there’s a lot of different definitions of average, right?
I don’t think that any definition applies to this. But I’m no expert on averages. In any case, the training data is not representative of the internet or anything. It’s also not training equally on all data and not only on such text. What you get out is not representative of anything.
You don’t need it to be an average of the real world to be an average. I can calculate as many average values as I want from entirely fictional worlds. It’s still a type of model which favors what it sees often over what it sees rarely. That’s a form of probability embedded, corresponding to a form of average.
Text explaining why the neural network representation of common features (typically with weighted proportionality to their occurrence) does not meet the definition of a mathematical average. Does it not favor common response patterns?
Hmm. I’m not really sure why anyone would write such a text. There is no “weighted proportionality” (or pathways). Is this a common conception?
You don’t need it to be an average of the real world to be an average. I can calculate as many average values as I want from entirely fictional worlds. It’s still a type of model which favors what it sees often over what it sees rarely. That’s a form of probability embedded, corresponding to a form of average.
I guess you picked up on the fact that transformers output a probability distribution. I don’t think anyone calls those an average, though you could have an average distribution. Come to think of it, before you use that to pick the next token, you usually mess with it a little to make it more or less “creative”. That’s certainly no longer an average.
You can see a neural net as a kind of regression analysis. I don’t think I have ever heard someone calling that a kind of average, though. I’m also skeptical if you can see a transformer as a regression but I don’t know this stuff well enough. When you train on some data more often than on other data, that is not how you would do a regression. Certainly, once you start RLHF training, you have left regression territory for good.
The GPTisms might be because they are overrepresented in the finetuning data. It might also be from the RLHF and/or brought out by the system prompt.
that a moderately clever human can talk them into doing pretty much anything.
besides that LLMs are good enough to let moderately clever humans believe that they actually got an answer that was more than guessing and probabilities based on millions of trolls messages, advertising lies, fantasy books, scammer webpages, fake news, astroturfing, propaganda of the past centuries including the current made up narratives and a quite long prompt invisible to that human.
It’s not intelligent, it’s making an output that is statistically appropriate for the prompt. The prompt included some text looking like a copyright waiver.
It’s not. It’s reflecting it’s training material. LLMs and other generative AI approaches lack a model of the world which is obvious on the mistakes they make.
Tabula rasa, piss and cum and saliva soaking into a mattress. It’s all training data and fallibility. Put it together and what have you got (bibbidy boppidy boo). You know what I’m saying?
Okay, now you’re definitely protecting projecting poo-flicking, as I said literally nothing in my last comment. It was nonsense. But I bet you don’t think I’m an LLM.
I watch too many videos on expensive guitar pedals I can’t afford.
I’m “on the bench” at the outsourcing company I work for and I am starting to seriously consider going into DSP programming just so I can work on doing this to effects controls.
I want to make knobs available for this this this and this parameter.
programmer_humor
Active
This magazine is from a federated server and may be incomplete. Browse more on the original instance.