Assignment Details

Phase One

1. Use a general purpose GenAI tool of your choice to create an annotated bibliography containing 8 references (including 5 peer-reviewed articles) with annotations and which are based on your research question (exploring a specific issue around AI, writing and image production). (You may use the questions we brainstormed in class on Thursday as a starting point.)

2. Review each entry for accuracy and fabrications and create a list that notes any inaccuracies, fabrications or if the information is correct.

3. Select two fabrications to peer review and note the fabrications (author doesn't exist, the article doesn't exist or the annotation is wrong). 

Phase Two

1. Using the AI generated bibliography as a starting point, create a revised eight item annotated bibliography. You may correct inaccuracies such as publication dates, volume or page numbers and, so long as the annotation is correct, these may remain in this revised bibliography. (IF the annotation is inaccurate (completely fabricated) remove it from the revised bibliography and find a new source. See next step.) 

2. Using the standard library search tools, replace the remaining articles with actual articles with the proper bibliographic information. 

3. Take two of the new articles you have located, read them and create an annotation for each.

Your final package will contain

1. The AI generated bibliography.

2. A listing of each of the articles in the AI generated bibliography identifying the accuracy of the information.

3. A revised bibliography of corrected entries including annotations for those that were correct in the first draft and the two you have done yourself.

Reflection/Analysis

This portion of the assignment requires the student to reflect on the use of generative AI as a research tool. To begin the reflection, the student will conduct a “peer review” of the content of the annotated bibliography created by the AI tool. This review includes reading and analyzing two (2) of the full journal articles provided (not just the abstracts) in the AI-created annotated bibliography.

Then based on these reviews of the full journal articles, students will write a short analysis (500-600 words) on what is correct information related to the topic, what is incorrect information related to their topic, and what is missing and should be discussed as it relates to the topic.

You may draw on your personal experiences, the course content, and your additional research on the topic to help your reflection.

The reflection should be 2-3 pages (500-600 words typed, 11 point font, 1.5 spacing), not including title page and references, and include (at a minimum):

1. A brief introduction to the issue that was identified as a focus for the annotated bibliography and the question you posed.

2. Identify the AI tool used and why

3. An analysis of what research was provided by the AI tool and what the student feels was

a. Correct, and why

b. Incorrect, and why

c. Not included, and why it should be

4. New research should be included to support the analysis of what was missed or incorrect. This research must be found by the student (not the AI tool) and must be cited and referenced.

*** Outside resources used in this analysis paper must be in an established citation and reference format, such as APA citation/reference format style.

***The new Reference/Bibliography created for the analysis paper must have a DOI number or website link for each source used. If no DOI or link is possible, the actual referenced document must be attached as an appendix. If this is not included, the instructor will begin a plagiarism investigation.

*** As the analysis assignment requires students to use their personal experiences, outside research, and writing skills, using ChatGPT, or any other AI writing software program is prohibited. This is considered Academic Misconduct, and MRU has plagiarism software to detect AI-written papers. Any students found to be using AI writing software for this portion of the assignment will be reported to the Student Conduct Office for Academic Misconduct.

Your Research Question

Clear: It is easily understood by your audience/reader.

Focused: It is narrow enough to allow you to address it in your assignment.

Concise: It is expressed in the fewest possible words (not too wordy).

Complex: It cannot be answered with a yes or no, and it is not a leading question (biased).

Open: It actually can be researched - naturally, it should generate more questions.

Examples:

Your topic:

• Artificial intelligence

Rephrase your topic as a question:

• What are some of the benefits and risks associated with the mass-scale use of generative AI to create images?

Possible search concepts/terms:

• opportunities / benefits

• challenges / risks / issues

• artificial intelligence / generative artificial intelligence / ChatGPT (or other specific GenAI tool)

• images / graphics / illustrations

What is an Annotated Bibliography?

An annotated bibliography includes a referenced source followed by a summary that describes the source and explains its relevance to your topic. In addition to the reference, each annotated bibliography entry must include:

• A brief explanation that establishes the author’s expertise. For example, where the author works and their professional title. This information is usually listed on the article’s title page or the book’s dust jacket.

• A sentence (or two) on the general topic or research question that the work addresses.

• A sentence (or two) on the thesis or argument of the work.

• A sentence on the author’s methodology. For example: What kinds of sources are used? Is it a case study or an overview of scholarship on the subject? How is the book/article organized?

• A sentence on how this source is relevant to your topic, and how it compares to other scholarship on the topic.

• A DOI that directly links to the journal paper/source.

AI tools do not always know how to create annotated bibliographies, so part of your role is to teach the AI tool how to create one.

***Students must ensure the AI tool uses 8 resources from established and credible sources (i.e., peer-reviewed journals, published books and credible print media such as newspapers of record including the Globe and Mail, Calgary Herald (not the Calgary Sun) and the Toronto Star). Strictly web-based resources are not permitted.*** Students must ensure the AI tool puts the resources in an established citation and reference format, such as APA, MLA, or Chicago citation/reference format style. APA is often the preferred format and I suggest you use that format.

***Students must ensure that the AI tool has provided a DOI number for each Reference/Bibliography source used and the DOI link aligns with the annotated bibliography information. If there is no DOI it is probably not a real source.

Different AI Tools

Image Generation Tool Examples:

• DALL·E 3

• Craiyon

Generative AI Product Tracker (Ithaka S+R).

The categories are:

• General Purpose Tools (pp. 1-9)

• Discovery Tools (pp. 10-19)

• Teaching & Learning Tools (pp. 19-31)

• Workflow Tools (pp. 31-42)

• Writing Tools (pp. 42-46)

• Coding Tools (pp. 46-48)

• Image Generation Tools (pp. 49-50)

• Other (pp. 50-53)

For this assignment, you will be using General Purpose Tools (pp. 1-9).

For the purposes of text generation, here are a few GenAI LLM chatbot tools you could use. (This list is not exhaustive.):

• OpenAI's ChatGPT (requires a free account to use ChatGPT 4o a limited number of times and ChatGPT 4o mini, their free chatbot)

• Google AI's Gemini (formerly known as Bard, but was renamed Gemini) (requires a Google account to use Gemini chatbot)

• Perplexity AI's Perplexity AI (doesn't require an account, but a free account is required to try Perplexity AI Pro and to save chats/threads)

• Anthropic's Claude (requires a free account)

• Microsoft's Copilot/Bing search (doesn't require an account, but supposedly works best with Microsoft account and in the Microsoft Edge browser)

• HuggingFace's HuggingChat (doesn't require an account to use the chatbot)

Keep in mind:

• These models work by performing a calculation to predict what the next most likely word in a sequence is.

• These models are not search engines, or, at least, they weren't designed as search engines originally. Some of them have search engine functionality now (like ChatGPT) and some will even provide footnotes (like Copilot/Bing), but it is still worth examining the linked source to see how the chatbot has represented the source. This is different than Google Search snippets where an excerpt from the actual source is provided to the user.

AI as a Research Tool

Generative artificial intelligence (AI) is a hot topic these days that is having an impact on many areas of cultural life, education, and the economy.

Many people in education (including myself) are still getting their heads around generative AI as a topic, and this is made difficult by how quickly the technology changes and how little non-experts understand about it. It is incredibly complex, blackboxed technology.

For this assignment, be sure to scrutinize any source(s) that generative AI provides you with on a topic. This is because, at this point, it is prone to error: what some have called "hallucination," but that I prefer to call "fabrication."

If generative AI provides you with a source:

(1) make sure that the source actually exists, and, if it does exist;

(2) make sure that the source actually contains the information that generative AI has attributed to it in an annotation.

What is fabrication?

An Investigation of ChatGPT's Sources

1. Book: Influencer Marketing for Dummies by Kristy Sammis, Cat Lincoln, and Stefania Pomponi

   • This source does exist and it was written by these authors, but it is a For Dummies book that wouldn't be considered scholarly.

2. Book: Influencer Marketing: Building Brand in a Digital Age by Duncan Brown and Nick Haye

   • This source does exist and it was written by those authors, but ChatGPT has fabricated a subtitle for it that it doesn't have.

3. Academic Article: "The Rise of Influencer Marketing and Its Impact on Consumer Behavior" by Liu, Hu, and Zhang (2019)

   • To the best of my searching abilities, this source does not exist.

4. Academic Article: "Ethical and Legal Issues in Influencer Marketing" by Brenner, A. and Capron, L. (2019)

   • To the best of my searching abilities, this source does not exist.

5. Academic Article: "The Dark Side of Social Media: A Consumer Psychology Perspective" by Phua, J., Jin, S.V., and Kim, J.J. (2017)

   • This source is a Frankenstein composite of 2 sources. The authors have been taken from this article and the title has been taken from this edited book with which those authors had no involvement.

Prompting

The CLEAR framework, created by Librarian Leo S. Lo at the University of New Mexico, is a framework to optimize prompts given to generative AI tools. To follow the CLEAR framework, prompts must be: 

Concise: "brevity and clarity in prompts"

• This means to remain specific in your prompt. 

Logical: "structured and coherent prompts" 

• Maintain a logical flow and order of ideas within your prompt.

Explicit: "clear output specifications"

•  Provide the AI tool with precise instructions on your desired output format, content, or scope to receive a stronger answer. 

Adaptive: "flexibility and customization in prompts"

• Experiment with various prompt formulations and phrasing to attempt different ways of framing an issue to see new answers from the generative AI 

Reflective: "continuous evaluation and improvement of prompts" 

• Adjust and improve your approach and prompt to the AI tool by evaluating the performance of the AI based on your own assessments of the answers it gives. 

Adapted from University of Calgary Libraries & Cultural Resources' AI guide and based on the following article:

Lo, L. S. (2023). The CLEAR path: A framework for enhancing information literacy through prompt engineering. The Journal of Academic Librarianship, 49(4), 102720–. https://doi.org/10.1016/j.acalib.2023.102720

Here is a graphic (created by the United States Military Academy Library) outlining the CLEAR method with examples for each criterion.

2 Examples of Generated Bibliographies

• ChatGPT

• Claude

Issues Identified with ChatGPT Generated Bibliography Above

• #1 is a preprint, so technically not peer-reviewed, but otherwise correct (at least for the reference) - NO PR (PEER REVIEW)

• #2 has inaccurate date and is not peer-reviewed (from NY Mag) - NO PR

• #3 has fabricated author and inaccurate date and is not peer-reviewed (from The Times) - NO PR

• #4 has fabricated authors, and inaccurate volume, issue, and page range info, but it is peer-reviewed! - YES PR

• #5 has fabricated authors and inaccurate date and is paywalled - we don't even have access to it through MRU Library! - also not sure if Nature Reviews Psychology's "Viewpoint" section would be peer-reviewed technically - MAYBE PR

• #6 has fabricated authors and inaccurate volume, issue and page range info - also a conference pub so not peer-reviewed, though Dr. Easton said he would accept conference publications in place of peer-reviewed resources for this assignment - TECHNICALLY NOT PR BUT CONSIDERED PR FOR THE PURPOSES OF THIS ASSIGNMENT

• #7 has inaccurate date, volume, issue, and page range info and an error in the title, but it is peer-reviewed - YES PR

• #8 has inaccurate date and is not peer-reviewed - NO PR

So, on this first try maybe 2-4 are peer-reviewed (not the required 5), and there are fabrications and inaccuracies.

Conducting Academic Research With LibrarySearch 

LibrarySearch is MRU Library's one-stop search interface/catalogue that brings together resources across format, time, and subject. 

We have millions of e-resources and 221,000 physical resources in our collection, and LibrarySearch searches across those.

Things to remember when using LibrarySearch:

1. Sign in to save searches, items, and to request materials.

2. Use the pin icon to save books and articles to your Favorites for future reference.

3. Use the filters on the right. You will use Availability, Resource Type, and Date filters most often. Filter settings can be "locked in" so that you don't have to reapply them to every search that you make.

4. Some items may not be available, however, you can request unavailable items using what is called interlibrary loan.

5. When viewing an item record, scroll down to the Get it (for hardcopy/physical items) or Access options (for electronic items) section to get access to the item.

Helpful Search Operators to Use in LibrarySearch

You can use what are called search operators to search in a way to combine or omit different terms by telling the search engine exactly what you want and this can help you save some time (and frustration!)

• Use quotation marks to keep specific phrases together:

  • "artificial intelligence"

• Use AND to combine search terms (LibrarySearch automatically creates an AND when you write terms one after another, but it can be good practice to use an AND to help you understand the searches that you build) (AND narrows your search):

  • "generative artificial intelligence" AND copyright

• Use OR to connect two or more similar terms (OR broadens your search):

  • GenAI OR "artificial intelligence"

• Use wild cards to substitute a letter or suffix with a symbol:

  • ethic* (in this example, the search ethic* will search for records that contain strings such as ethics, ethical, and ethically)

Conducting Academic Research With Google Scholar

Google Scholar

Google Scholar is another great way to find high quality resources.

Besides providing links to resources in MRU databases, Google Scholar links to online repositories that contain articles the author has been allowed to upload.  Academia.edu and ResearchGate are among the repositories searched by Google Scholar.

By clicking on the Settings icon, you can select library links to show library access for up to 5 libraries (type in Mount Royal and click on save).  If you are logged into MRU library, links should automatically populate if you are running a Google search in another window. 

Google Scholar has a nifty citation chaining function. The Cited by function will forward you to indexed scholarly material that has cited a resource that you may be interested in. The Related articles link will direct you to similar articles that may have the same metadata or keywords. 

Helpful Search Operators to Use in Google Scholar

Google Scholar's Advanced Search is found by clicking the menu icon in the top left.

You can also add search operators to Google Scholar searches to build your own custom advanced searches in similar ways to LibrarySearch:

• Use quotation marks to keep specific phrases together:

  • "artificial intelligence"

• Avoid using AND to combine search terms with Google Scholar, as the search engine automatically creates ANDs between concepts and sometimes adding an additional AND can confuse the search syntax.

• Use OR to connect two or more similar terms:

  • GenAI OR "generative artificial intelligence"

• Use wild cards to substitute a letter or suffix with a symbol:

  • ethic* (in this example, the search ethic* will search for records that contain strings such as ethics, ethical, and ethically)

Figure 1

Peter Steiner's Famous 1993 New Yorker Cartoon Illustrating an Issue Central to Information Evaluation

Note. From "On the Internet, nobody knows you're a dog" [Cartoon], by P. Steiner, 1993, Wikimedia (https://upload.wikimedia.org/wikipedia/en/f/f8/Internet_dog.jpg).

Evaluating Information

It is good to find lots of search results, but, in order to use information skilfully, you need to know how to evaluate that information to determine whether a specific resource is appropriate to use in a specific use case (i.e. for a specific assignment).

The phrase "evaluating information" actually stands in for a wide range of judgments that we make about information in many different contexts, whether those judgments are about relevance, timeliness, quality, etc.

Librarians have developed several different acronyms to help people remember useful criteria to use in information evaluation. One of my personal favourites is RADAR!

RADAR stands for

Relevance

Authority

Date

Accuracy

Reason for Creation

We can ask the following questions to help us assess each criterion:

Relevance:

• Does this source fit my topic?

• What is this source's intended audience?

  • Is that intended audience appropriate for my use case in this assignment?

Authority:

• Is/are the creator(s) of this source clearly identified or known to us?

• How important is it in this use case to trust the source's creator(s)?

  • If it is important, why should we trust the source's creator(s)?

  • Is the source's creator credentialed or an expert in their field?

Date:

• Is the creation or publication date of this source identified or known to us?

• Is this source too old?

Accuracy:

• Do this source's facts "check out"?

• Does the source have references of its own?

Reason for Creation (take your best guess at this question using judgments from earlier criteria):

• Why was this source made?

• Was this source made to sell a product or service, to inform/educate, to entertain, etc?

(Adapted from Mandalios, J. (2013). RADAR: An approach for helping students evaluate Internet sources. Journal of Information Science, 39(4), 470-478. https://doi.org/10.1177/0165551513478889)

Scholarly vs. Popular Sources

A scholarly publication contains articles written by experts in a particular field. The primary audience of these articles is other experts.

Many of these publications are also referred to as "peer-reviewed," academic, or "refereed." They all mean essentially the same thing and refer to the editorial and publication process in which scholars in the same field review the research and findings before the article is published.

Some Helpful Questions for Identifying a Scholarly/Academic Article

1. What are the author’s credentials? Was it written by an expert?

2. Was it published in a journal (is there a DOI?)? (If you are not sure if a source is a journal article, you can enter the title of the publication into Ulrichs Web to check.)

3. Does it use academic or more technical language?

4. Does it includes a reference list of sources that it is citing?

5. How long is it? (Scholarly articles are typically longer than popular or news articles.)

6. Does it have a "Received" and "Accepted" date on it?

7. Is it an actual article? (Sometimes other types of content are included in scholarly publications, such as editorials/opinion pieces and book reviews. Make sure you are looking at an article.)

 Add / Reorder 

Citation Help

• Use the "cite" feature in most search tools to get you started with most resources (you will need to review and correct the citation).

• Cite Sources: Learn the correct way to cite sources by using these guides, tutorials, and videos.

  • MRU Guide to APA Style for Referencing (2024-2025)

  • MRU Guide to APA Style for Formatting a Paper (2024-2025)

• Academic Success Workshops: Academic Success Workshops are 75 minutes long and are offered both in-person and online. Registration is required.

• Appointments: Personalized online or in-person 30-minute appointments with a Learning Strategist at Student Learning Services located on the 2nd floor of the Riddell Library & Learning Centre.

• Use the Service Desk on the 1st floor of the RLLC for assistance as well as the library chat feature on the library website for quick citation questions.