Generative AI in Higher Education: The ChatGPT Effect

Cover....1

Endorsement Page....2

Half Title....4

Title Page....6

Copyright Page....7

Dedication....8

Table of Contents....10

List of Figures....15

List of Tables....17

Author Biographies....19

Prologue....20

Acknowledgments....22

Short Summary of Each Chapter....24

Chapter 1: Introduction to Artificial Intelligence in Higher Education....24

Chapter 2: AI Literacy....24

Chapter 3: Strengths and Weaknesses in Embracing ChatGPT in Curriculum Design....24

Chapter 4: Redesigning Assessment in the AI Era....25

Chapter 5: Developing an AI in Education Policy....25

Chapter 6: Technology Behind GenAI....25

Chapter 7: The Future of AI in Education....26

Chapter 1: Introduction to Artificial Intelligence in Higher Education....28

1.1 Introduction....28

1.2 Artificial Intelligence (AI)....28

1.2.1 Three Types of AI....29

1.3 Big Data and the Internet of Things (IoT)....30

1.3.1 What Is Big Data?....31

1.3.1.1 Characteristics of Big Data....31

1.3.1.2 Example of Big Data and Its Characteristics in Higher Education....31

1.3.2 What is the Internet of Things (IoT)?....34

1.3.2.1 Characteristics of IoT....34

1.3.2.2 Example of IoT and Its Characteristics in Higher Education....35

1.4 Generative Artificial Intelligence (GenAI)....36

1.4.1 Applications of GenAI....37

1.5 ChatGPT – A Game Changer in Education?....38

1.6 Generative AI Concerns and Challenges....41

1.7 How Does ChatGPT Work?....42

1.7.1 So What Happens When You Type in a Question (Also Known as Prompt)?....42

1.8 How does GenAI Generate Images from Text?....43

1.8.1 So What Happens When You Type in a Question (Also Known as Prompt) to Generate an Image?....45

1.9 Conclusions....46

References....47

Chapter 2: AI Literacy....51

2.1 Introduction....51

2.2 The Importance of AI Literacy....52

2.3 Literacy in the Digital World....52

2.4 AI Literacy Frameworks....55

2.5 The Definition of AI Literacy....59

2.5.1 Expanding the Fundamental AI Literacy Framework: Incorporating Role-Specific and Multilevel Considerations....61

2.5.1.1 A Case Study: AI Literacy Amongst Different Professionals....62

2.5.2 AI Literacy for Teachers....62

2.6 Advantages and Disadvantages of Developing AI Literacy....65

2.7 Conclusions....67

References....68

Chapter 3: Strengths and Weaknesses in Embracing ChatGPT in Curriculum Design....71

3.1 Introduction....71

3.2 ChatGPT and GenAI Text Tools....72

3.2.1 What Is ChatGPT?....72

3.2.2 Common Functions of ChatGPT....72

3.3 GenAI Tools and Its Strengths and Potential Opportunities in Higher Education....73

3.3.1 User-Centric Design....74

3.3.2 Humanistic Conversational Style....75

3.3.3 Variability....76

3.3.4 Multimodal Capability....77

3.3.5 Scalability....77

3.3.6 Customisability....78

3.3.7 Comprehensive Coverage in Breadth and Depth....79

3.3.8 Contextual Relevance....79

3.3.9 Multilingual Support....80

3.3.10 Appropriateness of Topics....80

3.3.11 Code Generation....81

3.4 GenAI Tools and its Weaknesses and Potential Threats in Higher Education....82

3.5 AI-Partnered Pedagogy through the Lens of Bloom’s Taxonomy....91

3.5.1 Literary Study of ChatGPT’s GPT-4 via Bloom’s Taxonomy....91

3.5.1.1 Methodology....92

3.5.1.2 Findings of ChatGPT’s Literary Analysis of “Oliver Twist”....92

3.5.1.3 Evaluation....92

3.6 Redesigning Pedagogical Activities with GenAI....95

3.6.1 List of Pedagogies Facilitated by AI–Human Partnership....96

3.7 Case Scenarios Embracing ChatGPT into the Higher Education Classroom....101

3.8 Introduction to Prompt Engineering....107

3.8.1 The Essence of Prompts....107

3.8.2 The Prompt Engineering Components....108

3.8.2.1 Other Prompting Techniques....109

3.9 Conclusions....110

References....111

Chapter 4: Redesigning Assessment in the AI Era....114

4.1 Introduction....114

4.2 The Evolution of Assessment and Outcomes-Based Learning in Higher Education....114

4.3 Challenges in Assessment in the Traditional Era....118

4.4 Challenges in Assessment in the GenAI Era....119

4.5 Redesigning Assessment with GenAI....121

4.5.1 Integrate Multiple Assessment Methods....122

4.5.2 Promote Authentic Assessments in the AI Era....123

4.5.3 Promoting Academic Integrity and Genuineness in the AI Era....123

4.5.4 Embracing AI as a Learning Partner....124

4.5.5 Prioritising Soft Skills in Assessments in the AI Era....125

4.5.6 Prioritising Feedback Over Grades in the AI Era....126

4.6 AI Assessment Integration Framework....127

4.6.1 Performance-Based Assessment....129

4.6.2 Personalised or Contextualised Assessment....131

4.6.3 Human-Centric Competency Assessment....133

4.6.4 Human– Machine Partnership Assessment....134

4.6.5 Project- or Scenario-Based Assessment....136

4.6.6 Time-Sensitive AI-Generated Adaptive Assessment....139

4.6.7 Meta-cognitive Assessment....140

4.6.8 Ethical and Societal Impact Assessment....142

4.6.9 Lifelong Learning Portfolio Assessment....145

4.7 GenAI Text Detection....147

4.7.1 GenAI Text Detection Approaches....147

4.7.2 GenAI Text Detection Tools....149

4.8 Conclusions....150

References....151

Chapter 5: Developing an AI in Education Policy....154

5.1 Introduction....154

5.2 Are There Similar Laws for ChatGPT and GenAI?....154

5.2.1 Transparency, Explainability, and Interpretability....155

5.2.2 Fairness and Bias....155

5.2.3 Accountability....156

5.2.4 Safety and Robustness....157

5.2.5 Privacy and Data Protection....157

5.2.6 Autonomy and Human Oversight....159

5.2.7 AI Alignment for Humanity....159

5.2.7.1 The AI Alignment Problem....160

5.2.7.2 Tackling the AI Alignment Problem....161

5.3 AI Policy Around the World....161

5.3.1 China....162

5.3.2 The United States (US)....162

5.3.3 The European Union (EU)....162

5.3.4 United Kingdom (UK)....163

5.3.5 Australia....164

5.3.6 India....164

5.3.7 Japan....165

5.3.8 UNESCO....165

5.3.9 Differentiation and Progressiveness in Global AI Regulation....165

5.3.10 Business Reactions Towards the Regulations....166

5.4 AI Policy in Education....166

5.4.1 UNESCO’s Guidance for GenAI in Education and Research....167

5.5 Research Findings from the Perception of Students, Teachers and Staff on GenAI in Education Policy in Hong Kong....173

5.5.1 Methods....173

5.5.2 Quantitative Findings....174

5.5.3 Qualitative Findings....174

5.5.3.1 Governance Dimension (Senior Management)....174

5.5.3.2 Operational Dimension (Teaching and Learning and IT Staff)....174

5.5.3.3 Pedagogical Dimension (Teachers)....174

5.5.4 The AI Ecological Education Policy Framework....175

5.6 Devising an AI Policy in Higher Education....178

5.6.1 An Example of an AI Policy in Higher Education....184

5.6.2 University of ABC....185

5.6.2.1 Artificial Intelligence in Education Policy....185

5.7 Conclusions....187

References....188

Chapter 6: Technology Behind GenAI....193

6.1 Introduction....193

6.2 The History....194

6.2.1 The Genesis Phase (1940s–1980s)....197

6.2.1.1 New Fields of Study....197

6.2.1.2 The Symbolists....197

6.2.1.3 The Connectionists....200

6.2.1.4 Philosophical and Ethical Considerations....201

6.2.1.4.1 The Turing Test – Alan Turing....201

6.2.1.4.2 Three Laws of Robotics – Isaac Asimov....201

6.2.1.4.3 Computer Power and Human Reason – Joseph Weizenbaum....201

6.2.1.4.4 Father of Cybernetics – Norbert Wiener....202

6.2.2 The Maturing Phase (1980s to 2010s)....202

6.2.3 The Acceleration Phase (2010s to 2030s)....204

6.3 Creating a Model....206

6.3.1 The Training Data....208

6.3.1.1 Text Data – Common Crawl....208

6.3.1.2 Text Data – Colossal Clean Crawled Corpus (C4)....209

6.3.1.3 Image Data – LAION-5B....211

6.3.1.4 Image Data – LabelMe....211

6.3.1.5 Other Sources....211

6.3.1.6 Other Data Customisations....212

6.3.2 Model Design and Structuring....212

6.3.2.1 Artificial Neurons – Weights, Bias, and Activations Functions....213

6.3.3 Model Layers and Connections....215

6.3.4 Key Model Capabilities....216

6.3.4.1 Text – Tokenisation....216

6.3.4.2 Text – Encoding....217

6.3.4.3 Text – Embedding....218

6.3.4.4 Text – Attention....218

6.3.4.5 Text – Next Word Prediction Learning Goal....219

6.3.4.6 Image – Pre-processing....219

6.3.4.7 Image – Encoding....220

6.3.4.8 Image – Diffusion Learning Goal....220

6.3.4.9 Hyperparameters....220

6.3.5 Foundational Model Training....220

6.3.5.1 Supervised Learning....221

6.3.5.2 Self-Supervised Learning....221

6.3.5.3 Unsupervised Learning....221

6.3.5.4 Reinforcement Learning with Human Feedback (RLHF)....221

6.3.5.5 Learning....222

6.3.6 Foundational Model Testing....223

6.3.6.1 Testing Benchmarks....224

6.3.6.2 Common Sense Reasoning Benchmarks....224

6.3.6.2.1 BoolQ (Clark et al., 2018)....224

6.3.6.2.2 PIQA (Bisk et al., 2020)....225

6.3.6.2.3 SIQA (Sap et al., 2019)....226

6.3.6.2.4 SWAG (Zellers et al., 2018)....227

6.3.6.2.5 HellaSwag (Zellers et al., 2019)....228

6.3.6.2.6 WinoGrande (Sakaguchi et al., 2019)....228

6.3.6.2.7 ARC Easy and Challenge (Clark et al., 2018)....229

6.3.6.3 Question Answering Benchmarks....231

6.3.6.3.1 OpenBookQA (Mihaylov et al., 2018)....231

6.3.6.3.2 Natural Questions (Kwiatkowski et al., 2019)....232

6.3.6.3.3 TriviaQA (Joshi et al., 2017)....233

6.3.6.3.4 SQuAD v1.1 (Rajpurkar et al., 2016) and 2.0 (Rajpurkar et al., 2018)....233

6.3.6.4 Reading Comprehension Benchmarks....234

6.3.6.4.1 RACE reading comprehension benchmark (Lai et al., 2017)....234

6.3.6.5 Mathematical Reasoning Benchmarks....235

6.3.6.5.1 MATH (Hendrycks et al., 2021)....235

6.3.6.5.2 GSM8k (Cobbe et al., 2021)....235

6.3.6.6 Code Generation Benchmarks....236

6.3.6.6.1 HumanEval (Chen et al., 2021)....236

6.3.6.6.2 MBPP (Austin et al., 2021)....237

6.3.6.7 Multi-task Language Understanding Benchmarks....238

6.3.6.7.1 GLUE (Wang et al., 2018)....238

6.3.6.7.2 SuperGLUE (Wang et al., 2019)....239

6.3.6.7.3 MMLU (Hendrycks et al., 2020)....240

6.3.6.8 Toxicity Benchmarks....241

6.3.6.8.1 RealToxicityPrompts (Gehman et al., 2020)....241

6.3.6.9 Biases Benchmarks....241

6.3.6.9.1 CrowS-Pairs (Nangia et al., 2020)....241

6.3.6.9.2 WinoGender benchmark (Rudinger et al., 2018)....242

6.3.6.10 Truthfulness Benchmarks....243

6.3.6.10.1 TruthfulQA (Lin et al., 2022)....243

6.3.7 The Fine-Tuning....243

6.3.7.1 Fine-Tuning in Language Models (LLM)....244

6.3.7.2 Fine-Tuning in Image Generation Models....244

6.3.7.3 Steps for Fine-tuning....244

6.3.8 The Deployment....244

6.3.9 Model Use (aka Inference)....246

6.3.9.1 Prompt Engineering....247

6.3.9.2 Prompt Injection....247

6.3.9.3 Jailbreaking....247

6.3.9.4 Prompt Leaking....248

6.4 Models and Ecosystems....248

6.4.1 Custom Interfaces and Chat....248

6.4.2 APIs and Functions....250

6.4.3 Plugins and Agents....250

6.4.4 Custom Fine-Tuning....251

6.4.5 Custom Models....251

6.4.6 Vector Databases and Retrieval Augmented Generation (RAG)....251

6.5 State-of-the-Art Models Overview....252

6.5.1 LLM – OpenAI ChatGPT-4.0....252

6.5.2 LLM Meta LLaMA-2....253

6.5.3 LLM – Google Bard and PaLM-2....254

6.5.4 LLM – Anthropic Claude....255

6.5.5 LLM – Mistral AI Mistral....256

6.5.6 Diffusion Model – stability.ai Stable Diffusion....257

6.5.7 Diffusion Model – OpenAI DALL-E 3....258

6.5.8 Diffusion Model – Midjourney Inc. Midjourney....258

6.5.9 Speech Recognition – OpenAI Whisper....259

6.6 Conclusions....260

References....261

Chapter 7: The Future of Artificial Intelligence in Education....267

7.1 Introduction....267

7.2 Previous Technology Adoption....267

7.3 Predictions on General GenAI Implications....268

7.3.1 GenAI Adoption by General Population....268

7.3.2 Job Impact....268

7.3.3 AI Safety Impact – Adoption....270

7.3.4 AI Safety Impact – Post-Adoption....271

7.3.5 Impact from Governments....272

7.3.6 AI Development Companies....273

7.4 Predictions on GenAI’s Implications in Higher Education....274

7.4.1 Pedagogy and Assessment in Higher Education....274

7.4.2 Research in Higher Education....276

7.4.3 Drafting Funding Proposals....276

7.4.4 Submitting and Reviewing Journal Articles....277

7.5 Can AI or GenAI Predict the Future?....277

7.5.1 What did OpenAI ChatGPT Say?....278

7.5.2 What did Google Bard Say?....279

7.5.3 What did Microsoft Bing Chat Say?....280

7.6 Conclusion....280

References....281

Index....283

Chan and Colloton’s book is one of the first to provide a comprehensive examination of the use and impact of ChatGPT and Generative AI (GenAI) in higher education.

Since November 2022, every conversation in higher education has involved ChatGPT and its impact on all aspects of teaching and learning. The book explores the necessity of AI literacy tailored to professional contexts, assess the strengths and weaknesses of incorporating ChatGPT in curriculum design, and delve into the transformation of assessment methods in the GenAI era. The authors introduce the Six Assessment Redesign Pivotal Strategies (SARPS) and an AI Assessment Integration Framework, encouraging a learner-centric assessment model. The necessity for well-crafted AI educational policies is explored, as well as a blueprint for policy formulation in academic institutions. Technical enthusiasts are catered to with a deep dive into the mechanics behind GenAI, from the history of neural networks to the latest advances and applications of GenAI technologies.

With an eye on the future of AI in education, this book will appeal to educators, students and scholars interested in the wider societal implications and the transformative role of GenAI in pedagogy and research.

The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CC-BY-NC-ND) 4.0 license.