CSSE 413: Assessment of ChatGPT
In order to assess the current strenghts and weaknesses of LLMs,
select, assess and present on in-depth experiments with ChatGPT. If
you have access to the most recent version 4.0 that is great, if not,
that will be fine too.
Organizational Aspects
- Please work on the project in pairs.
- Please use the following Google
sheet to sign up. Please fill in your names and topic.
- You may wish to select a topic about which you know a lot. This
could be a very narrow, unusual topic, such as underwater basket
weaveing, about which there is not a lot of training
data. Alternatively, you may also choose an area for which there is a
lot of training data, such as LinkedLists. Again, ensure that you have
a fair amount of expert knowledge.
- I will review your proposal, make suggestions as appropriate and
let you know whether it has been approved as is or with modifications.
- Conduct a fair number of experiments to assess the power and
limitations of ChatGPT. Please ensure you state which version you are
using. You could also study both of them, to see how much version 4 is
an improvement of version 3.5.
- Write-up your experiments and your subjective evaluation. Please
include some representative sample dialogs you had with ChatGPT.
- You will be asked to give an 8 minute class presentation of your
key findings and assessments.