Distributed Data Management (ST 2021) - tele-TASK
The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization.Big data analytics and management are a multi-million dollar markets that grow constantly! The ability to control and utilize large amounts of data is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements.
In this lecture, we take a look at various technologies involved in building distributed, data-intensive systems. We start by discussing fundamental concepts in distributed computing, such das data models, encoding formats, messaging, data replication and partitioning, fault tollerance, and batch- and stream processing. In between, we consider different practical systems from the Big Data Landscape, such as Akka and Spark. In the end, we concentrate on data management aspects, such as distributed database management system architectures and distributed query optimization. www.tele-task.de
馃帶 Podcast episodes
Listen to 26 episodes
Similar podcasts
Distributed Data Management (WT 2019/20) - tele-TASK
The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at aro...
Distributed Data Management (WT 2018/19) - tele-TASK
The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at aro...
Distributed Data Analytics (WT 2017/18) - tele-TASK
The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at aro...
Hardware-Conscious Data Processing (ST 2022) - tele-TASK
Hardware development continuously advances, with different technologies improving at different pace. While the amount of transistors in a CPU package are growing, the single core performance is stagnating due to physical limitations. These trends require changes in data processin...
Big Data Systems (WT 2019/20) - tele-TASK
The amount of data that can be generated and stored in academic and industrial projects and applications is increasing rapidly. Big data analytics technologies have established themselves as a solution for big data challenges to the scalability problems of traditional database sy...
Internet Security (WT 2018/19) - tele-TASK
"Internet Security - Weaknesses and Targets" is based on "Internet- & WWW-Technologies" and gives a detailed introduction on problems concerning Internet and Intranet security. After starting with some remarks on risk analysis and computer crimes, security weaknesses and targets...
CLINICAL DATA MANAGEMENT
It is Clinical Data Management Best Practices podcast by IBA. Scientists need professional Data managers and Tools to be able to make the right decisions and establish their hypotheses correctly. We bring you interviews with leading specialists in Data Management, Biotechnology,...
St. Andrew Going Backstage 2021
Learn more about Methodist theology, history, traditions, and unique ways we study the Bible.聽聽