InfoCoBuild

Distributed Systems

Distributed Systems. Instructor: Dr. Rajiv Misra, Department of Computer Science and Engineering, IIT Patna. A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Distributed applications (distributed apps) are applications or software that runs on multiple computers within a network at the same time and can be stored on servers or with cloud computing. This course provides an in-depth understanding of fundamental principles and models underlying the theory, algorithms, and systems aspects of distributed computing. Few Emerging topics such as Peer-to-Peer computing, Distributed Hash Table, Google File System, HDFS, Spark, Sensor Networks and Security in Distributed Systems will also be covered for significant impact. Upon completing this course, students will have intimate knowledge about how things work in a distributed environment. (from nptel.ac.in)

Lecture 23 - Spark

This lecture covers the following topics: Motivation; Difference between Hadoop MapReduce vs. Apache Spark; Programming Model; Resilient Distributed Datasets (RDDs); RDD Operations; Built-in Libraries; Examples.


Go to the Course Home or watch other lectures:

Basic Instruction to Distributed Systems
Lecture 01 - Instruction to Distributed Systems
Lecture 02 - Basic Algorithms in Message Passing System
Lecture 03 - Leader Election in Rings
Lecture 04 - Models of Distributed Computation, Causality and Logical Time
Logical Time, Global State and Snapshot and Distributed Mutual Exclusion
Lecture 05 - Size of Vector Clock, Matrix Clocks, Virtual Time and Physical Clock Synchronization
Lecture 06 - Global State and Snapshot Recording Algorithms
Lecture 07 - Distributed Mutual Exclusion Algorithms and Non-Token Based Approaches
Lecture 08 - Quorum Based Distributed Mutual Exclusion Algorithms
Distributed Mutual Exclusion, Consensus and Agreement, Checkpointing and Rollback Recovery
Lecture 09 - Token Based Distributed Mutual Exclusion Algorithms
Lecture 10 - Consensus and Agreement Algorithms
Lecture 11 - Checkpointing and Rollback Recovery
Deadlock Detection, DSM and Distributed MST
Lecture 12 - Deadlock Detection in Distributed Systems
Lecture 13 - Distributed Shared Memory
Lecture 14 - Distributed Minimum Spanning Tree
Termination Detection, Message Ordering and Group Communication, Fault Tolerance
Lecture 15 - Termination Detection in Distributed System
Lecture 16 - Message Ordering and Group Communication
Lecture 17 - Self-Stabilization
Case Studies: Distributed Randomized Algorithms, DHT and P2P Computing
Lecture 18 - Distributed Randomized Algorithms
Lecture 19 - Peer-to-Peer Computing and Structured Overlay Network
Lecture 20 - Google File System (GFS)
Case Studies: MapReduce, HDFS and Spark, Distributed Algorithms for Sensor Networks
Lecture 21 - MapReduce
Lecture 22 - The Hadoop Distributed File System (HDFS)
Lecture 23 - Spark
Lecture 24 - Distributed Algorithms for Sensor Networks
Case Studies: Authentication in Distributed Systems, Bitcoin and BlockChain
Lecture 25 - Authentication in Distributed Systems
Lecture 26 - Bitcoin: A Peer-to-Peer Electronic Cash System
Lecture 27 - BlockChain Technology