Starting with Release 4.5.0.2, if you upgrade from HDP 2.X to HDP 3.X, you must create external tables.
In Hortonworks installations, all managed tables in HDP 3.X must be transactional tables. See Using Apache HiveQL.
ArcEngine cannot read transactional tables. For ArcEngine to read tables in HDP 3.X, which only supports transactional managed tables, you must create external tables.
If you use HDP 2.X, you must convert all managed tables to external tables before upgrading to HDP 3.X.
Use the following command:
ALTER TABLE table_name SET TBLPROPERTIES ('EXTERNAL'='TRUE', 'external.table.purge'='true');
If you already use transactional tables in HDP 3.X, you must create external tables that correspond to all these managed tables.
Use CREATE TABLE AS SELECT
(CTAS) or INSERT FROM
SELECT
commands, and keep the external tables synchronized with the
transactional tables.
You can access the data in the external version of the table through ArcEngine. You can continue to access the transactional version of the table through Hive.
Starting HDP 3.X, ArcEngine creates all tables as external by default.
This is true for catalog tables, tables created in ArcViz, when saving or importing
tables as CSV. It also applies to analytical views: ArcEngine creates them as
external tables even if the user does not specify the
EXTERNAL
keyword.
If you use HDP 2.X, you must convert all managed tables to external tables before upgrading to HDP 3.X.
Use the following command:
ALTER TABLE table_name SET TBLPROPERTIES ('EXTERNAL'='TRUE', 'external.table.purge'='true');
If you already use transactional tables in HDP 3.X, you must create external tables that correspond to all these managed tables.
Use CREATE TABLE AS SELECT
(CTAS) or INSERT FROM
SELECT
commands, and keep the external tables synchronized with the
transactional tables.
You can continue to access the data in the external version of the table through ArcEngine. You can access the transactional version of the table through Hive.
For performance reasons, we recommend that you create PARQUET
tables on top of catalog tables. Remember to create these tables as
EXTERNAL
tables.
On HDP 3.X, if you issue the DROP TABLE
and DROP
ANALYTICAL VIEW
commands, it has the following behavior:
external.table.purge
property to TRUE
to delete the corresponding files.EXTERNAL
keyword, the system
does not delete the corresponding files when you drop the tables and analytical
views.