ChatGPT Isn't as Good at Coding as We Thought

wpuckering , 9 months ago

Whoever thought it was good at coding? That’s not what it’s designed for. It might get lucky and spit out somewhat functional code sometimes based on the prompt, but it never constructed any of that itself. Not truly. It’s conceptually Googling what it thinks it needs, copying and pasting together an answer that seems like it might be right, and going “Here, I made this”. It might be functional, it might be pure garbage. It’s a gamble.

You’re better off just writing your own code from the beginning. It’s likely going to be more efficient anyways, and you’ll properly understand what it does.

1984 , 10 months ago

It’s fine. I’ve been using it for all sorts of languages. Some issues with rust though, but that’s because I’m a newbie and I can’t fix the errors chatgpt is producing.

Python, go, terraform… All great.

_e____b , 10 months ago

I don’t know how others are using chatGPT for coding, but I found I get the best results when starting small and iterate over the results few times. Like:

1. write a function to make a GET request;
2. write a function to handle this example JSON;
3. write a function that combines the first two;
4. etc etc

I use it mostly for Typescript, Bash and Clojure and results vary from good to OK (Clojure). The whole process is way faster if you use a tool like sGPT.

shotgun_crab , 10 months ago

I always thought of chat gpt as a “companion tool” that isn’t meant to write good code by itself, but to help experienced programmers write good code (just like search engines and documentation)

ratz , 10 months ago

Have you used it for this lately?

I want to believe it used to be okay for this, but just yesterday I uses it to generate some pretty basic bash and I’m honestly not convinced it saved me any time after I cleaned it all up and actually made it functional

Alexstarfire , 10 months ago

I thought it was terrible. Is it worse than that?

mintiefresh , 10 months ago

So so good for regex though.

MKBandit , 10 months ago

And SQL queries

Immersive_Matthew , 10 months ago

I am using ChatGPT 4+ with the code interpreter to code c# scripts inside my Unity project and it works and to be unreliable, b it about a month ago when the code interpreter came out it became very useful. Like it rarely makes a compile error and when it does it often fixes it with no further issues. The code it writes is solid and it has even been able to write multiple, interacting scripts with singletons, etc to do all sorts of more complex things in Unity. It has saved my so much time I am blown away. Some of these scripts are 200-300 lines. Beyond 300 it seems to have many issues though al really only good for the smaller stuff, which is mostly what Unity tends to be.

It is also amazing as feeding it error logs and having it tell you the bits that matter and why. Everyone should be using it for this at a minimum.

I love it, but look forward to the day where it is in my Unity project editor and is able to see all and address all the ridiculous and mundane issues that consume far too much of my time and other developers time. Just finished implementing AssetBundles with it, which triggered many of my scripts needing to be updated, which it did in just a few seconds each. Amazing.

alphacyberranger , 10 months ago

It’s like a rookie programmer or an intern at best. There has been times it has been really helpful though.

ricecooker , 10 months ago

I used it extensively to help me code my PHP for an art portfolio site. Briefly thought about using 11ty but needed to put something up quick after being laid off and I knew PHP.

For the most part it was good. It was really good at creating simple functions for me. My issue came when I asked it to build me a JS lightbox in Bootstrap. i wanted it to look a certain way so I had to edit my prompt multiple times because it would edit the code and “forget” my previous modification. Ended up using someone else’s JS code.

It was incredibly frustrating. It’s powerful, but still limited.

turkalino , 10 months ago

I asked it to write a bash script which simply read a couple inputs using readline and then ran a couple commands inside an if/else

It declared a variable that it never used. I pointed out the mistake and asked to remove the line. It simply renamed the variable.

I’d trust an unpaid intern more

fluxion , 10 months ago

Hah hah… What a dummy!

discreetly pushes his unused variables under desk with his foot

Ib_dI , 10 months ago

An unused variable!?

/clutches_pearls

tonytins , 10 months ago

Shouldn’t ChatGPT’s code be used as a template, anyway?

mom , 10 months ago

I would say is really capable at HELPING a human with coding tasks, but I found it to be kind of limited and sort of dumb. For example I was able to creare in Flask a “Ticket Management System” Web-App just for fun, but I had to do 90% of the work, and I had to be very specific along every step to make sure the output would do what I wanted and I had to provide very technical details that a beginner won’t really know. I think in the future we will have more capable tools that will create better apps without too much human interaction.

JoBo , 10 months ago

As who thought? There have been examples of wonky code being posted since the day it went live.

Jagger2097 , 10 months ago

That wonky code went into thousands of wonky scripts that barely work. People got promoted and the next fool gets to debug in ChatGPT5 or whatever

Cyo , 10 months ago

It can be useful for basic coding or to answer questions like ‘Is there any way to do X thing in Javascript?’ I were talking about it with some classmates , they said the same. There was one program I was doing on my own with Js & Html (I’m still learning) and for relying to much on GPT without much knowledge I ended up “walking on circles” for 6 Hours without any progress. It is good for giving some information and sometimes finding a bug, but never, never use it as if it were capable of doing everything. It’s a tool, not a programmer.

drkt , 10 months ago

Who are ‘we’? Surely not anyone who’s ever used it?

TropicalDingdong , 10 months ago

Who are ‘we’? Surely not anyone who’s ever used it?

I’m using it right now. It works great.

tenextrathrills , 10 months ago

I would hate to see your code if you think ChatGPT’s code works great. I’m guessing you’ve never asked it do anything complex.

TropicalDingdong , 10 months ago

I would hate to see your code if you think ChatGPT’s code works great. I’m guessing you’ve never asked it do anything complex.

Its like anything else, in that if you aren’t getting good results working with ChatGPT, you simply might not be informed enough to ask the right questions. If you don’t know what the right question is to ask, or how to form it, you’ll get poor results. I’ve been working in my field long enough to know where almost all of the bodies are buried, so I don’t fall into the kinds of traps that novices do when they really don’t understand what they are doing (although its not always obvious to them that they don’t).

abhibeckert , 10 months ago

If you don’t know what the right question is to ask, or how to form it, you’ll get poor results

ChatGPT could probably help you learn what questions to ask.

abhibeckert , 10 months ago (edited 10 months ago)

So this isn’t a real example, it’s just something I slapped together now as an example of how I generally use ChatGPT 4.0. In a more realistic scenario, I’d be asking more detailed questions, pasting in my existing code, and asking the AI to write smaller sections (maybe 10 lines of code at a time).

A single chat might run for five or six hours, continuously discussing an individual task I’m working on, and I won’t just be asking it to write a bunch code. For example in this case I might spend 10 minutes hashing out the exact contents of the ‘Activity’ record (or, maybe I already did that a month ago, and I might just paste in an example). I’d also be testing the code as I go and writing some of it myself, occasionally asking questions like ‘how do I format a date as 4 Feb 2016’ in JavaScript?’ or ‘this line of code fails with X. Why?’

In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before using this tool… note I’ve been doing work example like this example for about two decades so I was fairly productive before).

I’m also only pasting the code - ChatGPT also explains the code it outputs, and often those explanations are more useful than the actual code.

In JavaScript write a “reserveSeat(‘A42’, ‘Adult’)” function that sends an ActivityStreams 2.0 activity to (this.apiUrl)/reserve and inform the user if it failed to be reserved

<span style="color:#323232;">class SeatReservationService {
</span><span style="color:#323232;">  constructor(apiUrl) {
</span><span style="color:#323232;">    this.apiUrl = apiUrl;
</span><span style="color:#323232;">  }
</span><span style="color:#323232;">
</span><span style="color:#323232;">  async reserveSeat(seatNumber, seatType) {
</span><span style="color:#323232;">    // Construct an ActivityStreams 2.0 compliant JSON object
</span><span style="color:#323232;">    const activity = {
</span><span style="color:#323232;">      '@context': 'https://www.w3.org/ns/activitystreams',
</span><span style="color:#323232;">      'type': 'Reserve',
</span><span style="color:#323232;">      'actor': {
</span><span style="color:#323232;">        'type': 'Person',
</span><span style="color:#323232;">        'name': 'User'
</span><span style="color:#323232;">      },
</span><span style="color:#323232;">      'object': {
</span><span style="color:#323232;">        'type': 'Seat',
</span><span style="color:#323232;">        'id': seatNumber,
</span><span style="color:#323232;">        'category': seatType
</span><span style="color:#323232;">      }
</span><span style="color:#323232;">    };
</span><span style="color:#323232;">
</span><span style="color:#323232;">    try {
</span><span style="color:#323232;">      // Send the activity object to the specified API endpoint using an HTTP POST request
</span><span style="color:#323232;">      const response = await fetch(`${this.apiUrl}/reserve`, {
</span><span style="color:#323232;">        method: 'POST',
</span><span style="color:#323232;">        headers: {
</span><span style="color:#323232;">          'Content-Type': 'application/json'
</span><span style="color:#323232;">        },
</span><span style="color:#323232;">        body: JSON.stringify(activity)
</span><span style="color:#323232;">      });
</span><span style="color:#323232;">
</span><span style="color:#323232;">      // Handle the server’s response
</span><span style="color:#323232;">      if (response.ok) {
</span><span style="color:#323232;">        // If the request was successful, log a success message
</span><span style="color:#323232;">        console.log('Seat reserved successfully.');
</span><span style="color:#323232;">      } else {
</span><span style="color:#323232;">        // If the server responds with a status code other than 2xx, throw an error
</span><span style="color:#323232;">        throw new Error('Failed to reserve seat. Server responded with ' + response.status);
</span><span style="color:#323232;">      }
</span><span style="color:#323232;">    } catch (error) {
</span><span style="color:#323232;">      // Inform the user if the reservation failed
</span><span style="color:#323232;">      console.error('Failed to reserve seat:', error);
</span><span style="color:#323232;">      alert('Failed to reserve the seat. Please try again.');
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">  }
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">// Usage
</span><span style="color:#323232;">const service = new SeatReservationService('https://example.com/api');
</span><span style="color:#323232;">service.reserveSeat('A42', 'Adult');
</span>

Write a PHP request handler that does $this->reserveSeat(‘A42’), and checks if it failed with a SeatNotAvailable exception

<span style="color:#323232;">class SeatNotAvailable extends Exception {
</span><span style="color:#323232;">    public function __construct($message = "Seat not available", $code = 0, Exception $previous = null) {
</span><span style="color:#323232;">        parent::__construct($message, $code, $previous);
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">class SeatReservationService {
</span><span style="color:#323232;">
</span><span style="color:#323232;">    public function reserveSeat($seatNumber) {
</span><span style="color:#323232;">        // Mockup condition that the seat 'A42' is not available
</span><span style="color:#323232;">        if ($seatNumber === 'A42') {
</span><span style="color:#323232;">            throw new SeatNotAvailable("Seat $seatNumber is not available.");
</span><span style="color:#323232;">        }
</span><span style="color:#323232;">
</span><span style="color:#323232;">        // Normally here you'd interact with a database or another kind of storage
</span><span style="color:#323232;">        // to actually reserve the seat.
</span><span style="color:#323232;">        // ... (reserve seat logic) ...
</span><span style="color:#323232;">        
</span><span style="color:#323232;">        return true;
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">class SeatReservationController {
</span><span style="color:#323232;">
</span><span style="color:#323232;">    private $service;
</span><span style="color:#323232;">
</span><span style="color:#323232;">    public function __construct() {
</span><span style="color:#323232;">        $this->service = new SeatReservationService();
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">
</span><span style="color:#323232;">    public function handleRequest() {
</span><span style="color:#323232;">        try {
</span><span style="color:#323232;">            $this->service->reserveSeat('A42');
</span><span style="color:#323232;">            echo "Seat reserved successfully.";
</span><span style="color:#323232;">        } catch (SeatNotAvailable $e) {
</span><span style="color:#323232;">            echo $e->getMessage();
</span><span style="color:#323232;">        }
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">}
</span>

Write a PHPUnit test for SeatReservationController, and a Jest one for SeatReservationService

<span style="color:#323232;">createMock(SeatReservationService::class);
</span><span style="color:#323232;">        $serviceMock->method('reserveSeat')
</span><span style="color:#323232;">                    ->willReturn(true);
</span><span style="color:#323232;">
</span><span style="color:#323232;">        $controller = new SeatReservationController($serviceMock);
</span><span style="color:#323232;">        $this->expectOutputString("Seat reserved successfully.");
</span><span style="color:#323232;">        $controller->handleRequest();
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">
</span><span style="color:#323232;">    public function testSeatNotAvailableException() {
</span><span style="color:#323232;">        $serviceMock = $this->createMock(SeatReservationService::class);
</span><span style="color:#323232;">        $serviceMock->method('reserveSeat')
</span><span style="color:#323232;">                    ->will($this->throwException(new SeatNotAvailable()));
</span><span style="color:#323232;">
</span><span style="color:#323232;">        $controller = new SeatReservationController($serviceMock);
</span><span style="color:#323232;">        $this->expectOutputString("Seat not available");
</span><span style="color:#323232;">        $controller->handleRequest();
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">}
</span>

<span style="color:#323232;">const SeatReservationService = require('./SeatReservationService'); // Adjust the path as necessary
</span><span style="color:#323232;">
</span><span style="color:#323232;">test('reserves a seat successfully', async () => {
</span><span style="color:#323232;">    const service = new SeatReservationService('https://example.com/api');
</span><span style="color:#323232;">    global.fetch = jest.fn(() => 
</span><span style="color:#323232;">        Promise.resolve({
</span><span style="color:#323232;">            ok: true
</span><span style="color:#323232;">        })
</span><span style="color:#323232;">    );
</span><span style="color:#323232;">
</span><span style="color:#323232;">    await service.reserveSeat('A41', 'Adult');
</span><span style="color:#323232;">    expect(fetch).toHaveBeenCalledWith(
</span><span style="color:#323232;">        'https://example.com/api/reserve', 
</span><span style="color:#323232;">        expect.objectContaining({
</span><span style="color:#323232;">            method: 'POST',
</span><span style="color:#323232;">            headers: {
</span><span style="color:#323232;">                'Content-Type': 'application/json'
</span><span style="color:#323232;">            },
</span><span style="color:#323232;">        })
</span><span style="color:#323232;">    );
</span><span style="color:#323232;">});
</span><span style="color:#323232;">
</span><span style="color:#323232;">test('fails to reserve a seat', async () => {
</span><span style="color:#323232;">    const service = new SeatReservationService('https://example.com/api');
</span><span style="color:#323232;">    global.fetch = jest.fn(() => 
</span><span style="color:#323232;">        Promise.resolve({
</span><span style="color:#323232;">            ok: false,
</span><span style="color:#323232;">            status: 400
</span><span style="color:#323232;">        })
</span><span style="color:#323232;">    );
</span><span style="color:#323232;">
</span><span style="color:#323232;">    global.console = { error: jest.fn(), log: jest.fn() };
</span><span style="color:#323232;">
</span><span style="color:#323232;">    await service.reserveSeat('A42', 'Adult');
</span><span style="color:#323232;">    expect(console.error).toHaveBeenCalledWith(
</span><span style="color:#323232;">        'Failed to reserve seat:', 
</span><span style="color:#323232;">        expect.any(Error)
</span><span style="color:#323232;">    );
</span><span style="color:#323232;">});
</span>

TropicalDingdong , 10 months ago

In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before I started paying $20/month for ChatGPT+).

This is very comparable to the ROI I would say that I’ve been seeing for my programming work. I feel like a super hero, or a 22 year old on adderall. I know everything I need to do for any project, but between client meetings, executive meetings, business development, meetings with the product team, mentoring, the actual amount of focused time I get is so little. I can offload a huge amount of the “I know how to do this and I’ll know if you do it wrong, but please do this for me” to the machine. This past May I took on a task that would have taken a comparable person, probably 6 months, and I knocked it out in 2.5 weeks. If you already know what you are doing, ChatGPT is steroids.

SeymoureButtocks , 10 months ago (edited 10 months ago)

This. It’s incredibly useful considering it’s age and is more useful than the ego trip that StackOverflow became. Niche topics are a struggle sure, but if you know what to ask it and to check what it says, it is an amazing coding companion.