From Language to Information

Ethics Content Description

There are three ethics units in the class: one on sentiment classification, one on information retrieval, and one on large language models. The ethics content is embedded within a three labs that students work on in groups during the scheduled class period. Therefore, the ethics material is a mix of technical problems students work on to practice the material, as well as broader questions they discuss in small groups and with the rest of the class. The topics covered by these three units include: biases introduced by sentiment classifiers, issues of data privacy and data sovereignty related to personalized search engines, and the societal implications of large langauge models.

Course Description

Extracting meaning, information, and structure from human language text, speech, web pages, social networks. Introducing methods (string algorithms, edit distance, language modeling, machine learning, logistic regression, neural networks, neural embeddings, inverted indices, collaborative filtering, PageRank), applications (chatbots, sentiment analysis, information retrieval, text classification, social networks, recommender systems), and ethical issues.

Contributors

Ethics materials created by Veronica Rivera (with Uma Phatak, Deveshi Buch, and Dan Jurafsky).

Assignments

Naive Bayes, Sentiment, and Harms of Classification

Naive Bayes Sentiment classification Automoderation of toxic content Bias

Assignment Page

Learning goal: Understanding what bias is in NLP and how to analyze NLP systems for bias.

Description: This class operates under a flipped classroom model, where students work on problem sets, called labs, in class. In this lab, students are prompted to consider basic approaches to building a sentiment classifier using Naive Bayes and to think about the potential harms introduced by how their classifer would operate on specific words in context.

Contains: Lab handout

Information Retrieval

Information retrieval Precision Recall Search engines Data sovereignty Privacy

Assignment Page

Learning goal: Understanding the implications of personalized information retrieval systems, and the tradeoff between precision and recall.

Description: This class operates under a flipped classroom model, where students work on problem sets, called labs, in class. In this lab, students are prompted to discuss the benefits and tradeoffs of personalized search engines and the kinds of data these search engines should be allowed to query. Additionally, student practice making precisions vs. recall decisions for a variety of hypothetical scenarios for which they might be tasked with building an information retrieval system, considering the societal implications of prioritizing one over the other.

Contains: Lab handout

Large Language Models

Information retrieval Precision Recall Search engines Data sovereignty Privacy

Assignment Page

Learning goal: Understanding LLM vulnerabilities like hallicination and disinfomation.

Description: This class operates under a flipped classroom model, where students work on problem sets, called labs, in class. In this lab, students are prompted to discuss the limitations of large-language models. In particular, students are prompted to play with different inputs to an LLM to observe if/how it hallucinates, and to observe the ease with with an LLM can produce mis/disinformation.

Contains: Lab handout

Download all

From Language to Information

Ethics Content Description

Course Description

Contributors

Stanford course number

Course Level:

Topics

Assignments

Naive Bayes, Sentiment, and Harms of Classification

Information Retrieval

Large Language Models