Artificial intelligence chatbots – a help or hindrance to computer science education

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Downloads (Pure)


Recent developments in artificial intelligence (AI) chatbot systems have gained significant coverage over the past few months across all subject disciplines and educational levels. Emergent technologies such as Chat GPT from Microsoft and BARD from Google have demonstrated extraordinary use of machine learning to enable detailed responses with high-utility for simple text-based queries. Students at all levels are broadly aware of the significance of such systems in providing an apparently easy route to homework solutions, with associated concerns from educators. While many consider such systems to be a threat to educational practices, others have embraced the technology, exploring these capabilities to support and enhance learning and development.

The work detailed in this paper considers the use of Chat GPT in suggesting solutions to simple programming problems, typically used when leaning to develop software. Problem specifications from current programming assignments in year 1 of an undergraduate computing degree program are considered in the context of this system, over a range of input fragments and compared against utility of output (functionality and correctness of generated code). Responses generated from Chat GPT for a problem specification are considered alongside current student work and submitted to a blind assessment process.

Results indicate that for simple problems, a significant proportion of code generated through Chat GP produces a fairly high utility, although amendments are required in all cases to enable testing. In many cases, text directly lifted from a problem specification provided enough material for Chat GPT to generate a reasonable response, although increased complexity resulted in reduced utility. The paper provides an overview analysis of initial experimentation and results, focusing specifically on how such systems could potentially benefit the novice programmer.

Original languageEnglish
Title of host publicationProceedings of the International Conference on Education and New Developments, END 2023
EditorsMafalda Carmo
PublisherinScience Press
ISBN (Electronic)9789893510643
ISBN (Print)9789893510636
Publication statusPublished - 24 Jun 2023
EventEducation and New Developments Conference 2023 - Lisbon, Portugal
Duration: 24 Jun 202326 Jun 2023

Publication series

NameEducation and New Developments
ISSN (Print)2184-044X
ISSN (Electronic)2184-1489


ConferenceEducation and New Developments Conference 2023
Abbreviated titleEND 2023


Dive into the research topics of 'Artificial intelligence chatbots – a help or hindrance to computer science education'. Together they form a unique fingerprint.

Cite this