Introduction to Poe<\/h3>\n
\u201cBelieve nothing you hear, and only one half that you see<\/em>\u201d. \u2013 Edgar Allen Poe<\/p>\n I happened to see a demonstration of Poe<\/a>, an AI chatbot aggregator platform in one of the Digi Fest<\/a> sessions. The demonstrator explained how they have been using the tool in a higher secondary school environment where students were provided this tool to do some self-paced learning about a particular topic from specific knowledge sources. Some of the knowledge sources provided were books downloaded as PDF. It seemed too fantastical but the fact that it was demonstrated live was enough to pique my interest and get me thinking. I just made note of the tool name and left it at that. This was a time when I had just joined Cardiff University and had taken over someone else\u2019s module to teach. I was trying to personalize the content, changing the materials to align with my style of teaching. The assessments were already moderated and released to the students. But the students were yet to start working on them. I was warned of the barrage of questions that would be coming my way once the students start them. But before that I needed time to go through the coursework assessment and the solution guidance to prepare myself. This was when I decided to try out Poe.<\/p>\n \u201cIt is a happiness to wonder; — it is a happiness to dream.\u201d<\/em> \u2013 Edgar Allen Poe<\/p>\n I started with the free version using a Gmail account I had created for this purpose and created a \u201cprompt bot\u201d.<\/p>\n This effectively creates a chatbot much like the \u201cHi there!\u201d style assistant bot you see in various websites. This prompt bot used the default base bot of \u201cGPT 3.5 Turbo\u201d. I provided the coursework and solution guidance as its knowledge source (after removing the university related data like email addresses etc.) and a very minimal prompt to act as my assistant to help the students understand the coursework (thereby helping me understand too).<\/p>\n Once I created the chatbot, I tested it by asking questions related to the coursework just as a student would. I tried combinations of straightforward questions about what a task was about, what are the word limits, how I would achieve highest mark, asking for direct solutions, and indirect way of getting answers etc. I had to learn a bit more about prompt engineering to fine tune my initial prompt to the bot to ensure that it doesn\u2019t reveal the solutions directly or too much in depth but enough to help the students. In some cases, it gave some basic answers, especially for questions which were of low points, but I was alright with that. Poe also provides options to control the creativity of the bot\u2019s responses to prevent hallucination. I had set it to the default of 0.5, but if we did lower it, we would get more consistent responses.<\/p>\n Poe is part of Quora, and it needs an account to start with. The privacy concerns are like any other Gen AI tool and technology available. The data will be saved to enhance its performance and since Poe is an aggregator and uses a base bot, it will in turn refer to the privacy concerns of the base bot.<\/p>\n \u201cLet my heart be still a moment and this mystery explore..\u201d.<\/em> \u2013 Edgar Allen Poe<\/p>\n Once I was convinced of its performance, I shared the bot with the students after week 9 by making it publicly accessible and asked them to use it as an additional tool for any queries regarding the coursework especially during the time when I won\u2019t be able to answer their queries especially during out of office hours. I did provide a disclaimer that since I won\u2019t be able to see their prompts and the responses from the bot, which I will come to later, the students were asked to send me the same question and get an official response from me. Whenever I received a question from the student, I tested it against the bot and verified its response. If it was dissatisfactory then I updated the knowledge source with the related information. It is possible to add additional knowledge source using plain text as and when required, which I found very helpful. Once the semester was done and dusted, I sent out a feedback form to the students to provide their opinions on the Coursework Assistant bot. I received only three responses and all of them were in favor of trying out such bots in the future. One of the comments on the performance of the bot was \u201cI was sometimes questioning whether the bot was giving correct interpretations of the coursework, and I would normally prefer emailing the module leader anyway to get a definite answer<\/em>\u201d. This showed the student\u2019s distrust in Gen AI tools especially if it has a direct impact on their grades. The students do use Gen AI tools to write their assessments like the programming exercises and reflective questions, but they do not yet trust the Gen AI tool which acts as a digital twin to their module instructor.<\/p>\n \u201cThere is no exquisite beauty\u2026 without some strangeness in the proportion.\u201d<\/em> – Edgar Allan Poe<\/p>\n \u201cIt is a happiness to wonder; — it is a happiness to dream.\u201d<\/em> – Edgar Allan Poe<\/p>\n Even though Poe was introduced to me as a learning aid, I decided to make it my digital twin, sort of. The first attempt was a rushed one since I was very new to the module. I had very less time to test the tool and create a bot that was fit for purpose. I barely scratched the surface of this tool and its potential. I am planning to use it again this year and I have already created a chatbot for my coursework with much tighter measures in place. I have currently provided the chatbot to other instructors and teaching associates of my module for testing. When I asked my students about the use of such tool as their aid, I received similar responses to last academic year\u2019s.<\/p>\n I am planning to release the tool to the students after week 7 and test them live during some of the tutorial sessions to evaluate the performance of the chatbot. I dream of a time when this chatbot would respond to all the repetitive coursework queries leaving me to provide specific individual help where and when necessary. Fingers crossed!<\/p>\n","protected":false},"excerpt":{"rendered":"This blog is written by Ramalakshmi Vaidhiyanathan and explores using AI to aid student\u2019s assessment queries. Ramalakshmi Vaidhiyanathan is a lecturer in the School of Computer Science and Informatics, with […]","protected":false},"author":7054,"featured_media":5158,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[371],"tags":[],"custom_author":[1015],"class_list":["post-5149","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-digital-education","custom_author-ramalakshmi-vaidhiyanathan"],"lang":"en","translations":{"en":5149,"cy":5160},"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4KqgK-1l3","meta_box":[],"jetpack_featured_media_url":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-content\/uploads\/sites\/286\/2024\/12\/Sized-34.png","pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/posts\/5149","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/users\/7054"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/comments?post=5149"}],"version-history":[{"count":5,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/posts\/5149\/revisions"}],"predecessor-version":[{"id":5159,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/posts\/5149\/revisions\/5159"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/media\/5158"}],"wp:attachment":[{"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/media?parent=5149"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/categories?post=5149"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/tags?post=5149"},{"taxonomy":"custom_author","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-json\/wp\/v2\/custom_author?post=5149"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Practicing Poe<\/h3>\n
![\"\"](\"http:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-content\/uploads\/sites\/286\/2024\/12\/pic-1.png\")
<\/figure>\n![\"\"](\"http:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-content\/uploads\/sites\/286\/2024\/12\/Picture2.png\")
<\/figure>\n![\"\"](\"http:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-content\/uploads\/sites\/286\/2024\/12\/Picture3.png\")
<\/figure>\n![\"\"](\"http:\/\/blogs.cardiff.ac.uk\/LTAcademy\/wp-content\/uploads\/sites\/286\/2024\/12\/Picture4-1.png\")
<\/figure>\nCoursework Assistant Bot at your service<\/h3>\n
Pros and Cons of Poe<\/h3>\n
\n
Marching on<\/h3>\n
\n\n
\n \n Question<\/strong><\/h5>\n<\/td>\n
\n Use of Coursework AI Assistant<\/strong><\/h5>\n<\/td>\n<\/tr>\n
\n \n Respondents<\/strong><\/h5>\n<\/td>\n
\n 119<\/h5>\n<\/td>\n<\/tr>\n
\n <\/td>\n <\/td>\n<\/tr>\n \n \n Choices<\/strong><\/h5>\n<\/td>\n
\n Votes<\/strong><\/h5>\n<\/td>\n<\/tr>\n
\n \n I would definitely use it<\/h5>\n<\/td>\n
\n 20<\/h5>\n<\/td>\n<\/tr>\n
\n \n I don’t trust AI<\/h5>\n<\/td>\n
\n 34<\/h5>\n<\/td>\n<\/tr>\n
\n \n I will use it but still ask you questions<\/h5>\n<\/td>\n
\n 65<\/h5>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n