Theme: Presents and classifies use cases of online analytics applications. The
introduction distinguishes offline analytics from online analytics from the point of view
of use case and used technology stack.
Discussion: A business user/analyst utilizes an interactive, online application that offers an analytics dashboard, in order to obtain insights into the data. The running example can be an application that analyzes the pharmaceutical literature for insights on correlations between diseases, substances etc. The user computes statistics and mines various slices of the data. She drills-down and rolls-up to change the level of detail at which she investigates. She generates and tests what-if hypotheses. The application uses rich visualization to convey the results and allows the slicing, mining, drilling-down, roll-up and what-if testing to be accomplished at a click of a button.
The result is the derivation of knowledge via the live interaction of the user with the data.
Such derived knowledge is written in the database and consequently also participates in
the analytics.
Presentation of course goals: The topic of the course is the development of systems
for online analytics in today’s era of Big and Fast Data. The implementation of online
analytics systems is supported by architectures (and corresponding programming
patterns) that combine databases (often novel combinations of databases with the
MapReduce/Hadoop stack) and data visualization. Notice the requirements distinction
from offline analytics applications.
Theme: As in title.
Discussion:
A baseline architecture, against which improvements will be made in later
chapters. The architectural principles focus on logical abstractions that (i)
simplify implementation and (ii) minimize complexity.
o Model (data) layer: Relational databases, REST/WSDL web APIs,
wrappers over semistructured data (messages, documents, clickstream
logs, etc).
o View (visualization) layer: Ajax page is a rendered view over the model’s
state (i.e., over data). When state changes, page is updated incrementally.
Use a templating engine and component frameworks as abstraction
layer for page updates, with goal of achieving the incremental update
automatically.
o Controller layer: Application logic that maps user actions to (i) how to
update model state (ii) which page to display.
Code snippets are first introduced.
Theme: Data warehouse and materialized views architectures for supporting analytics
applications with precomputation.
Overview of database and data warehouse architectures that support analytics
applications through precomputation:
o Data warehouses: Description of the data warehouse architecture and
comparison to transactional relational DBMSs. Description of the model:
Star/snowflake schema, cubes. Description of the operators: Slice and
dice, roll up, drill down, pivoting.
o Materialized views: Overview of materialized views. Description of
materialized view maintenance.
Theme: Reduce cost of development/ownership by utilizing high performance dbs.
Overview of novel database architectures for big data management
o Column-based database management systems (DBMSs): Databases
that store data by columns. Explanation of the optimizations enabled by
column-stores: late tuple materialization, compression, invisible joins.
Description of how a column-store can be simulated in a traditional row-
based DBMS (gaining some of the advantages for the price conscious).
o Parallel DBMSs: Overview of parallel architectures: shared memory,
shared disk and shared nothing architecture. Focus on shared nothing
architectures. Description of different types of parallelization in DBMSs:
intra-operator, inter-operator (pipelining), inter-query parallelization.
Partitioning and examples of intra-operator parallelization for common
relational operators.
Theme: Analytics that provide the latest insights over live data streams
Theme: Utilization of approximations and statistical estimation in order to achieve online
performance.
Discussion: Performing analytics over large amounts of data is a very time-intensive
task. As such, computing analytics on the fly is not a viable solution for interactive
applications that have strict requirements on the system’s response time. In this section
we describe how a developer can provide interactive analytics performance suitable
for such applications, by trading the computation’s precision for latency. We present
techniques for approximating analytics based on sampling. We distinguish between
samples that are precomputed and samples that are computed incrementally on the fly (as
is the case in online aggregation). We also discuss how one can use statistical estimation
to provide formal approximation guarantees (ensuring a low margin of error with high
confidence).