I am researching doing the same, but know nothing about running my own yet. Did you train your llm for programming in any way, or just download and run an open source one? If so which model etc do you use?
Lot’s of technical details, but essentially the llamafile is a engine + model + web ui, in a single executable file. You just download it and run it and stuff happens.
Run an open source one. Training requires lots of knowledge and even more hardware resources/time. Fine tuned models are available for free online, there is not much use in training it yourself.
I recommend llavafiles, as this is the easiest option to run. The GitHub has all the stuff you need in the “quick start” section.
Though the default is a bit restricted on windows. Since the llavafiles are bundling the LLM weights with the executable and Windows has a 4GB limit on executables you’re restricted to very small models. Workarounds are available though!
Im gonna give llamafile a go! I want to try to run it at least once with a different set of weights just to see it work and also see different weights handle the same inputs.
The reason I am asking about training is because of my work where fine tuning our own is going to come knocking soon, so I want to stay a bit ahead of the curve. Even though it already feels like I am late to the party.
Isn’t that the idea. Like you know that you had a viable solution to a complex problem previously so why go through the trouble of solving it again if you already did. Even if you have to modify it, it saves time for new novel problems. I’m
That’s the way. I’ve been programming for nigh on four decades, and it’s almost a daily occurrence with junior devs going to stack overflow or chatGPT to solve an issue instead of just searching the code where nine times out of ten the problem (or a very similar one) is already solved.
In my experience, Copilot does a fairly good job when you already know what you’re doing, but can’t be bothered to write the code yourself.
For example, basic stuff like read data from that file, use dplyr, remove these columns, do these calculations, plot the result using ggplot2, label the axes this way, use those colors etc. Copilot gives you the code that does roughly what you want, but you usually need to tweak it a bit it to suit your preferences. Copilot also makes absurd mistakes, but fixing them is fairly easy. If this is the sort of stuff you’re doing, copilot can indeed boost your productivity.
However, if you don’t know how to do something a bit more exotic like principal component analysis, and you ask copilot to do the job for you, expect plenty of trouble. You may end up on a wild goose chase, using the wrong tools, doing unnecessary calculations and all sorts of crazy nonsense. When you know what you’re doing, you can ask a very specific thing. When you don’t, you may end up being too ambiguous in your prompt, which will result copilot leading you down the wrong path.
You can do it this way too, but before implementing a single line of that garbage code, you absolutely have to ask copilot a bunch of questions just to make sure you really understand what you’re doing, what the new functions do, where do you really want to go etc. You’re probably going to have to tweak the code before running it, and that’s why you need to know what you’re doing. That’s the one big area you can’t outsource to copilot just yet.
But is it still faster than reading the documentation and building your own experimental tests? If you spend an hour and get a pile of broken garbage, then certainly not. If you spend a bit more, ask plenty of questions, make sure you know what you’re doing, then maybe it is worth it.
I 100% agree. I especially love when copilot literally just starts making up shit that doesn’t work or doesn’t exist. Like it can’t be wrong. It just freaking guesses… God forbid it can’t admit it doesn’t have enough data to answer the question.
Best part is when you say “that command doesn’t exist” it’s like “I apologize. Here is a real command to accomplish your task”
SMH
Again, to your point, I agree that copilot is amazing if you already know how to write the code you want. We’re smart enough to know that the suggestions will work for our task. It’s definitely not smart enough to replace you
Yeah, I was lucky that I snuck into my company’s pilot program for it.
I’m impressed at how often it predicts what I’m about to do. The code almost always needs a slight bit of editing, but it almost always at least shaves a bit of time off of whatever task I was doing.
I no longer go straight to stackoverflow, I always ask the copilot first. Sometimes even just phrasing the question in natural language, something I wouldn’t do it trying to find it via search or stackoverflow, is kind of like rubber duck debugging, and I’ll come up with the answer while writing it out.
It reuses MY OWN CODE STYLE. So if I ignore a suggestion and setup a try catch in my own quriky way it’ll actually reuse it later on when I’m scripting. This works best when you add comments for the sections you write FIRST. So you comment # create array for x data it’ll do that or try catch for query it’ll give you a suggestion for the next block right away.
DEBUGGING. github copilot can see your terminal and script so it’ll give you a detailed breakdown and suggestions. Blew my mind the first time.
I was originally a chip designer. Then I shifted into embedded development. Now I’m mainly a C# guy.
But when I shifted into embedded development, I also shifted into doing power engineering. I grabbed a couple of books on the topic at hand, taught myself a lot, and designed the electronics to meet the need. We sold the product to city utilities.
I remember one time I was in a room with probably 10 engineers from one of the utilities. After having described the product to them, and went through a lot of our settings and stuff, I was explaining the difference between two of algorithms we put in (because different utilities use different algorithms, and I just wanted one device that could do both). At some point I was like “which of the two algorithms do you use?” and one responded “well, which do you recommend?” So I talked about why I thought one was better than the other.
They all started looking at each other and nodding and saying “Yeah, that’s the one were going to use.” I realize I could have said anything at that point and they would have agreed. They thought I was expert. And that was my “last two frames” of this comic moment.
Now as a senior dev, I’ve seen enough shit to realize that most people have no idea what is going on, and are flying by the seat of their pants. So I figure my ignorance is a little less than theirs, and that gives me a lot of confidence, but I also realize that I can learn a lot from most people.
Now as a senior dev, I’ve seen enough shit to realize that most people have no idea what is going on, and are flying by the seat of their pants.
What’s helpful in my industry is that new development happens so frequently that the absolute best answer today is probably the wrong one in the next few years. Since I’m never on the absolute cutting edge, I have to trust my team to pitch the plan and we roll with it.
What I think makes me a senior is me knowing that we don’t know anything, but being able to create a plan if/when we have to make changes.
I’ve always felt like I don’t deserve my role, I climbed the proverbial ladder quick and I am very young for my position (Principal Engineer). But I sleep fine at night because at the end of the day I was always honest with my skills, my intentions and my motivations, and I’m always sure to get full agreement from everyone before doing stuff. If after all that nobody figured out I’m a fucking idiot just making an informed guess, that’s on them.
I always fear the next company I join will have “real technical leaders” who will inevitably show me my place, but it hasn’t happened so far (3-4 massive companies in the last decade).
Maybe one day I will meet this person, but it is not this day… And then I try to teach the same to younger engineers to work through problems as a team and just do it until somebody stops you, because in a lot of cases nobody has a clue either, and that’s what it means to lead.
When I was working stand-in positions such as after a move, for example retail, my favorite go-to when asked “Whyyyy?” was “I have no idea. No one told me anything.” I sometimes miss those days.
You’re right though. Most people have enough knowledge to do the steps of the job or task. For many of them skipping a step shuts down that memory, if only temporarily. I’ve met only a handful of true experts. People who can do things forwards, backwards, upside-down, and mix things up on the fly. They are BY FAR the most uncommon.
Yeah I’m shocked by some of the things I read in these types of thread. I’m not perfect, we all make mistakes, but I am definitely confident in that I know what I am doing.
I however slipped much faster into the senior dev position than I liked. Basically after 2 years full time (and 5 years as a student on part time) I am now expert in a few things and the go to person for questions. For people who work longer in there than me.
programmer_humor
Oldest
This magazine is from a federated server and may be incomplete. Browse more on the original instance.