BecomeMe

801 readers

1 users here now

Social Experiment. Become Me. What I see, you see.

founded 1 year ago

MODERATORS

Renneder@sh.itjust.works

ChatGPT gets code questions wrong 52% of the time (www.theregister.com)

submitted 1 year ago by Renneder@sh.itjust.works to c/becomeme@sh.itjust.works

10 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Qfuiyh@lemm.ee 2 points 1 year ago (3 children)

Title feels misleading, it gets stack overflow questions wrong 52% of the time

However it got 77% of easy Leetcode questions correct. Also I believe that's first try, which is not generally how chatgpt should be used.

Also also, you should probably be using a coding specific model if you want good coding results

[–] nanoUFO@sh.itjust.works 5 points 1 year ago

Every leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.

[–] OskarAxolotl@lemmy.world 3 points 1 year ago

Probably because the model has seen thousands of possible solutions to those exact Leetcode problems. Actual questions people ask on StackOverflow tend to be much more specialized.

[–] HackerJoe@sh.itjust.works 2 points 1 year ago

But it confidently explains the wrong answers.

I just hope politicians don't find out how to use it. It'll be our doom.