Skip to main content

Ctrl+K

Welcome to SSIE 541/441 Applied Soft Computing

Intro

Why applied soft computing?
Set up
Deliverables

Word and Document Embeddings

Module 1: Word Embedding
Pen and Paper Exercise
Teaching computers how to understand words
word2vec: a small model with a big idea
GloVe and FastText: Building on Word2Vec’s Foundation
Understanding SemAxis: Semantic Axes in Word Vector Spaces
Bias in Word Embeddings
Doc2Vec: From Words to Documents
Summary

Recurrent Neural Networks

Module 2: Language Models
🧠 Learn RNNs Through Physics!
Recurrent Neural Network (RNN)
Long Short-Term Memory (LSTM)
Embedding from Language Models (ELMo)
Sequence-to-Sequence Models
Summary

Transformers

Module 3: Transformers
Transformers
Bidirectional Encoder Representations from Transformers (BERT)
Sentence-BERT
Generative Pre-trained Transformer (GPT)
From Language Model to Instruction-Following
Prompt Tuning
Prompt Tuning Exercise
Summary
Appendix: Text-to-Text Transfer Transformer (T5)

Teaching Machine How to Understand Images

Module 4: Image Processing
Pen and paper exercises
Preliminaries: Image Processing
From Traditional Image Processing to Learning
LeNet and LeNet-5: Pioneering CNN Architectures
AlexNet: A Breakthrough in Deep Learning
VGGNet - A Deep Convolutional Neural Network for Image Recognition
GoogleNet and the Inception Module
Batch Normalization Explained
ResNet (Residual Neural Networks)

Teaching Machine How to Understand Graphs

Module 4: Graph
Pen and paper exercises
Spectral Embedding
Graph embedding with word2vec
Spectral vs Neural Embedding
From Image to Graph
Graph Convolutional Networks
Popular Graph Neural Networks
Appendix
Software for Network Embedding

Repository
Open issue

.md

Prompt Tuning

Prompt Tuning#

The notebook on prompt tuning is available at Prompt Tuning.

previous

From Language Model to Instruction-Following

next

Prompt Tuning Exercise

By Sadamori Kojaku