“Don’t Fight Me, Al”

I have plenty of thoughts on ChatGPT and its clones that I haven’t fully organized. Long story short, the availability of generative chat bots is going to change a lot about the kind of work that I assign in my classes, and I think that any academic who’s pretending that there won’t be a significant adjustments is whistling past the graveyard. One professor I spoke with estimates that 30% of the work he received this last semester was generated by ChatGPT in some form. In any case, I’ve been playing around with ChatGPT quite a bit and I found this exchange awfully interesting:

What’s interesting here isn’t that ChatGPT got a pretty basic question (how many films have Hopkins and Dalton appeared in together) wrong; we know that it gets things wrong all the time. Rather, the interesting bit is that when challenged ChatGPT decided to double down and stick to its guns rather than update. Only on the second challenge (when I offered evidence) did it come around. I’d be inclined to think that it had the wrong Lion in Winter in mind (the 2003 version) but it correctly identified Hopkins as Richard in the first answer, so that’s not quite right either.

Again, I’ll have more thoughts later. I do think that ChatGPT and its clones can be remarkably effective tools when used in the correct circumstances. I know, for example, that ChatGPT will change the art of Dungeon Mastering forever; you now have an instantly available backstory for every goblin that the PCs capture and try to interrogate or every tavern owner that they mistake for an important NPC. But as the above shows, for real world questions there are some pretty stark limitations.

Also, I still don’t know how many movies Dalton and Hopkins have appeared together in.