The Analytical Views of Arcadia Enterprise are a semantic caching mechanism that allows users to compute and cache results of expensive SQL operations, such as grouping and aggregation.
Let us start with a brief introduction to analytical views:
The use of analytical views is transparent, because queries always target the base table.
Explore and build reports first; Arcadia Engine enhances query performance by using existing analytical views, or recommending new ones.
You can update the analytical views incrementally.
Storage format for analytical views is non-proprietary; it is as simple as Parquet or Kudu tables.
Analytical tables can read data from base tables in many different formats. See Supported Data Storage.
Analytical views can reside on HDFS/S3 and Kudu, and can be exposed through the Hive metastore.
Arcadia Engine shares analytical views automatically with all authorized users.
We integrated analytical views with Hadoop-native security: Sentry and Ranger.
Analytical views confer a number of significant performance benefits.
Queries from apps are automatically routed to analytical views with matching SQL expressions.
Predictable workloads (querying) can be optimized and completed within a few seconds.
As the workload becomes more predictable, the implicit use of analytical views increases.
Arcadia Engine incrementally refreshes analytical views that are well-partitioned (and partitioned identically to the base tables). This leads to measurable performance advantages. The same is true for analytical views defined on top of logical views that reference a single base table.
To understand the basic principals of analytical views, see Architecture of Analytical Views.
To learn more about the use of aggregate function in analytical views, see Aggregate Functions in Analytical Views.
The information in Authorization for Analytical Views explains the impact of access authorization for base tables on the use of analytical views.
Arcadia Data has two approaches for analytical views:
The refresh mechanism for these analytical views depends on the analytical views and the base table being partitioned on the same columns. Arcadia Engine detects changes to partitions of the base table, and determines which partitions of the analytical view it must refresh. Release 4.5.0.0, partition-based analytical views work for tables stored in HDFS, S3, and ADLS.
Learn more about partition-based analytical views:
The refresh mechanism for these analytical views relies on the use of a sequence column. In Release 4.5.0.0, sequence-based analytical views work for tables stored in Kudu format. Kudu is a storage manager that provides a combination of fast inserts and updates, and efficient columnar scans to enable multiple real-time analytic workloads across a single storage layer.
Learn more about sequence-based analytical views:
The Arcadia Visualization Server, ArcViz, supports the management and creation of partition-based analytical views. See Managing Analytical Views in the User Interface.