Presentation

What generative AI systems know about cybersecurity

October 12, 2023

1903225 bytes

PDF Document - Need a reader? Get one here

The public release of OpenA's ChatGPT system in November 2022 signaled an inflection point for AI technology and its applications. While these AI systems have well-known shortcomings, they have the potential to help in many ways. After describing the technology, I report on a recent evaluation of OpenAI's ChatGPT and Google's Bard ability to solve cybersecurity problems using two datasets designed to test students' knowledge: the Cybersecurity Concept Inventory (CCI) and the Cybersecurity Curriculum Assessment (CCA). The CCA results are compared with those from a recent evaluation of 193 students from seven colleges and universities. Spoiler: one of the AI systems performed surprisingly well.

Presentation for an online meeting of Maryland Computer Science teachers

Counter: 45 downloads

OWL Tweet