Abstract
Recent developments in artificial intelligence (AI) chatbot systems have gained significant coverage over the past few months across all subject disciplines and educational levels. Emergent technologies such as Chat GPT from Microsoft and BARD from Google have demonstrated extraordinary use of machine learning to enable detailed responses with high-utility for simple text-based queries. Students at all levels are broadly aware of the significance of such systems in providing an apparently easy route to homework solutions, with associated concerns from educators. While many consider such systems to be a threat to educational practices, others have embraced the technology, exploring these capabilities to support and enhance learning and development.
The work detailed in this paper considers the use of Chat GPT in suggesting solutions to simple programming problems, typically used when leaning to develop software. Problem specifications from current programming assignments in year 1 of an undergraduate computing degree program are considered in the context of this system, over a range of input fragments and compared against utility of output (functionality and correctness of generated code). Responses generated from Chat GPT for a problem specification are considered alongside current student work and submitted to a blind assessment process.
Results indicate that for simple problems, a significant proportion of code generated through Chat GP produces a fairly high utility, although amendments are required in all cases to enable testing. In many cases, text directly lifted from a problem specification provided enough material for Chat GPT to generate a reasonable response, although increased complexity resulted in reduced utility. The paper provides an overview analysis of initial experimentation and results, focusing specifically on how such systems could potentially benefit the novice programmer.
The work detailed in this paper considers the use of Chat GPT in suggesting solutions to simple programming problems, typically used when leaning to develop software. Problem specifications from current programming assignments in year 1 of an undergraduate computing degree program are considered in the context of this system, over a range of input fragments and compared against utility of output (functionality and correctness of generated code). Responses generated from Chat GPT for a problem specification are considered alongside current student work and submitted to a blind assessment process.
Results indicate that for simple problems, a significant proportion of code generated through Chat GP produces a fairly high utility, although amendments are required in all cases to enable testing. In many cases, text directly lifted from a problem specification provided enough material for Chat GPT to generate a reasonable response, although increased complexity resulted in reduced utility. The paper provides an overview analysis of initial experimentation and results, focusing specifically on how such systems could potentially benefit the novice programmer.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the International Conference on Education and New Developments, END 2023 |
| Editors | Mafalda Carmo |
| Publisher | inScience Press |
| Pages | 571-574 |
| Volume | 2 |
| ISBN (Electronic) | 9789893510643 |
| ISBN (Print) | 9789893510636 |
| Publication status | Published - 24 Jun 2023 |
| Event | Education and New Developments Conference 2023 - Lisbon, Portugal Duration: 24 Jun 2023 → 26 Jun 2023 |
Publication series
| Name | Education and New Developments |
|---|---|
| ISSN (Print) | 2184-044X |
| ISSN (Electronic) | 2184-1489 |
Conference
| Conference | Education and New Developments Conference 2023 |
|---|---|
| Abbreviated title | END 2023 |
| Country/Territory | Portugal |
| City | Lisbon |
| Period | 24/06/2023 → 26/06/2023 |