Release Notes, 4.5.0.2

Arcadia Enterprise Release 4.5.0.2

(ArcViz 4.5.0.2, ArcEngine 2.11.1.1)

This document contains release information notes for Arcadia Enterprise Release 4.5.0.2, and is accurate at the time of publication.

To see the notes for other releases, see Release Notes.

In this release, we made the following enhancements:

To see our certification matrix, read Certifications.

To see the catalog of known issues, read Known Issues.

Upgrade to this Release

Arcadia Enterprise upgrades to Release 4.5.0.2 from the baseline of Release 4.2.X and Release 4.3.X. If you have earlier versions of the product, you must perform a two-step upgrade: to Release 4.2.1.1, and then to Release 4.5.0.2.

Upgrade from HDP 2.X to HDP 3.X

In Hortonworks installations, all managed tables in HDP 3.X must be transactional tables. See Using Apache HiveQL.

ArcEngine cannot read transactional tables. For ArcEngine to read tables in HDP 3.X, which only supports transactional managed tables, you must create external tables.

Developer Notes:
  • The external tables cannot use transactional table properties.
  • For information on troubleshooting an upgrade process, see Apache Ambari Major Upgrade .

Starting with Release 4.5.0.2, follow these guidelines to prepare to upgrade from HDP 2.X to HDP 3.X.

If you perform DDL operations through Hive
  • If you use HDP 3.X, you must create all new tables as external tables.
  • If you use HDP 2.X, you must convert all managed tables to external tables before upgrading to HDP 3.X.

    Use the following command:

    ALTER TABLE table_name SET TBLPROPERTIES ('EXTERNAL'='TRUE', 'external.table.purge'='true');
  • If you already use transactional tables in HDP 3.X, you must create external tables that correspond to all these managed tables.

    Use CREATE TABLE AS SELECT (CTAS) or INSERT FROM SELECT commands, and keep the external tables synchronized with the transactional tables.

    You can access the data in the external version of the table through ArcEngine. You can continue to access the transactional version of the table through Hive.

If you perform DDL operations through ArcEngine:
  • Starting HDP 3.X, ArcEngine creates all tables as external by default.

    This is true for catalog tables, tables created in ArcViz, when saving or importing tables as CSV. It also applies to analytical views: ArcEngine creates them as external tables even if the user does not specify the EXTERNAL keyword.

  • If you use HDP 2.X, you must convert all managed tables to external tables before upgrading to HDP 3.X.

    Use the following command:

    ALTER TABLE table_name SET TBLPROPERTIES ('EXTERNAL'='TRUE', 'external.table.purge'='true');
  • If you already use transactional tables in HDP 3.X, you must create external tables that correspond to all these managed tables.

    Use CREATE TABLE AS SELECT (CTAS) or INSERT FROM SELECT commands, and keep the external tables synchronized with the transactional tables.

    You can continue to access the data in the external version of the table through ArcEngine. You can access the transactional version of the table through Hive.

  • For performance reasons, we recommend that you create PARQUET tables on top of catalog tables. Remember to create these tables as EXTERNAL tables.

  • On HDP 3.X, if you issue the DROP TABLE and DROP ANALYTICAL VIEW commands, it has the following behavior:

    • If you create the tables using the CREATE TABLE or CREATE ANALYTICAL VIEW commands without specifying the EXTERNAL keyword, when you drop the tables and analytical views, the system sets the external.table.purge property to TRUE to delete the corresponding files.
    • If you create the tables using the EXTERNAL keyword, the system does not delete the corresponding files when dropping the tables and analytical views.