Distributed Data Management (WT 2019/20) - tele-TASK

The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization.

Big Data Analytics is a multi-million dollar market that grows constantly! Data and the ability to control and use it is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements.

In this lecture, we take a look a various technologies involved in building distributed, data-intensive systems. We discuss theoretical concepts (data models, encoding, replication, ...) as well as some of their practical implementations (Akka, MapReduce, Spark, ...). Since workload distribution is a concept which is useful for many applications, we focus in particular on data analytics.
Distributed Data Management (WT 2019/20) - tele-TASK

馃帶 Podcast episodes

Listen to 27 episodes
Exam Preparation
91 minutes

Exam Preparation

Distributed DBMSs
87 minutes

Distributed DBMSs

Stream Processing
86 minutes

Stream Processing

Spark Batch Processing 2
90 minutes

Spark Batch Processing 2

Spark Batch Processing
88 minutes

Spark Batch Processing

Beyond MapReduce
89 minutes

Beyond MapReduce

Transactions & Batch Processing
71 minutes

Transactions & Batch Processing

Distributed Systems
92 minutes

Distributed Systems

Replication & Partitioning
89 minutes

Replication & Partitioning

Replication 2
84 minutes

Replication 2

The Graph Data Model
89 minutes

The Graph Data Model

Data Models and Query Languages
88 minutes

Data Models and Query Languages

Akka Actor Programming 2
91 minutes

Akka Actor Programming 2

Akka Actor Programming
91 minutes

Akka Actor Programming

Encoding and Communication 2
92 minutes

Encoding and Communication 2

Encoding and Communication
90 minutes

Encoding and Communication

Foundations
82 minutes

Foundations

Introduction
76 minutes

Introduction

Similar podcasts

CLINICAL DATA MANAGEMENT

All Call with Task Force 20

20/20

Reshaping Education - Online Education, Cohort-based Courses, Bootcamps, and More

RegTech 20/20

Maryland Risk Management Education Podcast

EMS 20/20

20/20 MONEY