Purdue CS440: Large-scale Data Analytics
(Fall 2024)


Course Description

"Big data" has been a buzzword for a long time. Many disruptive techniques have been developed to address various aspects of big data. This course will cover the key concepts, design principles, and systems to analyze large-scale data in order to extract novel and transformative insights. Tentative topics include database fundamentals, big data storage (e.g., HDFS), big data computing frameworks (e.g., Hadoop and Spark), data warehouses, data lakes, graph analytics (e.g., Spark Graph), data streaming (e.g., Spark Streaming), large-scale machine learning (e.g., Spark MLlib), vector databases, and cloud-native data analytics.



Instructor

  • Jianguo Wang
  • Email: csjgwang@purdue.edu (note: must include "[CS440]" in the subject)


Teaching Assistants

  • Shige Liu (liu3529@purdue.edu)
  • Yunan Zhang (zhan4404@purdue.edu)
  • Yunxin Sun (sun1114@purdue.edu)


Logistics

  • When: MW 2:30p-3:20p
  • Where: LWSN 1142
  • Office hour: after class or make appointment
  • Pre-requisites: CS242, CS251, and CS373


Labs and PSOs

Labs and PSOs will start from the 2nd week (in LWSN B148).
  • L01: Friday 1:30pm-3:20pm
  • L02: Wednesday 3:30pm-5:20pm
  • L03: Friday 11:30am-1:20pm
  • L04: Tuesday 3:30pm-5:20pm


Online communications

  • We'll use Piazza, e.g., announcements, discussions, and Q&A.
  • We'll NOT use Brightspace except for sending emails occasionally.
  • We'll use Gradescope for submitting and grading homeworks.


Textbooks (Optional)

Note that textbooks are optional and the lectures slides are self-contained.


Grading

  • Homeworks: 20% (2 * 10%)
  • Midterm exam: 25%
  • Final exam: 35%
  • Project: 20%
    • Projects are related to the labs and will be explained in the labs.
  • Extra credits: 5%


Academic Integrity and More


Schedule

Lecture

Topic
Lec 1 (08/19) Course Introduction
Lec 2 (08/21) Relational DB
Lec 3 (08/26) No class due to a business conference
Lec 4 (08/28) SQL (Zoom Lecture)
Lec 5 (09/02) No class due to Labor day
Lec 6 (09/04) Database Storage
Lec 7 (09/09) Index
Lec 8 (09/11) Query Processing 1
Lec 9 (09/16) Query Processing 2
Lec 10 (09/18) Transaction
Lec 11 (09/23) Concurrency Control
Lec 12 (09/25) Crash Recovery
Lec 13 (09/30) Crash Recovery 2
Lec 14 (10/02) Distributed Databases
Lec 15 (10/07) No class due to October break
Lec 16 (10/09) Midterm Exam (In-class)
Lec 17 (10/14) Hadoop
Lec 18 (10/16) SQL-on-Hadoop
Lec 19 (10/21) Big Data Storage
Lec 20 (10/23) Big Data Storage 2
Lec 21 (10/28) Spark Core
Lec 22 (10/30) Spark SQL
Lec 23 (11/04) Spark ML
Lec 24 (11/06) Spark Streaming
Lec 25 (11/11) Spark Graph
Lec 26 (11/13) Vector Data Analytics
Lec 27 (11/18) Vector Data Analytics 2
Lec 28 (11/20) Cloud-Native Data Analytics
Lec 29 (11/25) Cloud-Native Data Analytics 2
Lec 30 (11/27) No class due to Thanksgiving
Lec 31 (12/02) Guest Lecture
Lec 32 (12/04) Review