Welcome to SSIE 541/441 Applied Soft Computing

``Don’t think! Feeeeeel’’ is a famous quote by Bruce Lee in the movie Enter the Dragon, and this is my guiding philosophy of learning.

This course is designed to help you feel the concepts and tools of applied soft computing through pen-and-paper exercises and hands-on coding.

Module	Exercise	Jupyter Notebook
M01	Word Embedding	- Word Embedding - SemAxis - Doc2Vec
M02	- Simple RNN - LSTM	- PyTorch
M03	- Transformers Inside Out - BERT	- Practical Guide to Sentence Transformers

Intro

• Why applied soft computing?
• Set up
• Deliverables

Word and Document Embeddings

• Module 1: Word Embedding
• Pen and Paper Exercise
• Teaching computers how to understand words
• word2vec: a small model with a big idea
• GloVe and FastText: Building on Word2Vec’s Foundation
• Understanding SemAxis: Semantic Axes in Word Vector Spaces
• Bias in Word Embeddings
• Doc2Vec: From Words to Documents
• Summary

Recurrent Neural Networks

• Module 2: Language Models
• 🧠 Learn RNNs Through Physics!
• Recurrent Neural Network (RNN)
• Long Short-Term Memory (LSTM)
• Embedding from Language Models (ELMo)
• Sequence-to-Sequence Models
• Summary

Transformers

• Module 3: Transformers
• Transformers
• Bidirectional Encoder Representations from Transformers (BERT)
• Sentence-BERT
• Generative Pre-trained Transformer (GPT)
• From Language Model to Instruction-Following
• Prompt Tuning
• Summary
• Appendix: Text-to-Text Transfer Transformer (T5)

Teaching Machine How to Understand Images

• Module 3: Image Processing
• Pen and paper exercises
• Preliminaries: Image Processing
• From Traditional Image Processing to Learning
• LeNet and LeNet-5: Pioneering CNN Architectures
• AlexNet: A Breakthrough in Deep Learning
• VGGNet - A Deep Convolutional Neural Network for Image Recognition
• GoogleNet and the Inception Module
• ResNet (Residual Neural Networks)

Teaching Machine How to Understand Graphs

• Module 4: Graph
• Pen and paper exercises
• Spectral Embedding
• Graph embedding with word2vec
• Spectral vs Neural Embedding
• From Image to Graph
• Graph Convolutional Networks
• Popular Graph Neural Networks
• Appendix
• Software for Network Embedding