diff --git a/content/docs/latest/user/Hive-Transactions-ACID.md b/content/docs/latest/user/Hive-Transactions-ACID.md index 438b1b02..2633a83a 100644 --- a/content/docs/latest/user/Hive-Transactions-ACID.md +++ b/content/docs/latest/user/Hive-Transactions-ACID.md @@ -94,7 +94,7 @@ This module is responsible for discovering which tables or partitions are due fo #### Worker -Each Worker handles a single compaction task.  A compaction is a MapReduce job with name in the following form: -compactor-...  Each worker submits the job to the cluster (via [hive.compactor.job.queue]({{< ref "#hive-compactor-job-queue" >}}) if defined) and waits for the job to finish.  [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) determines the number of Workers in each Metastore.  The total number of Workers in the Hive Warehouse determines the maximum number of concurrent compactions. +Each Worker handles a single compaction task.  A compaction is a MapReduce job with name in the following form: \-compactor-\.\.\.  Each worker submits the job to the cluster (via [hive.compactor.job.queue]({{< ref "#hive-compactor-job-queue" >}}) if defined) and waits for the job to finish.  [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) determines the number of Workers in each Metastore.  The total number of Workers in the Hive Warehouse determines the maximum number of concurrent compactions. #### Cleaner @@ -170,7 +170,7 @@ A number of new configuration parameters have been added to the system to suppor | metastore.compactor.long.running.initiator.threshold.error | *Default:* 12h | Metastore | Initiator cycle duration after which an error will be logged. Default time unit is: hours | | hive.compactor.worker.sleep.time | *Default:*10800ms | HiveServer2 | Time in milliseconds for which a worker threads goes into sleep before starting another iteration in case of no launched job or error | | hive.compactor.worker.max.sleep.time | *Default:* 320000ms | HiveServer2 | Max time in milliseconds for which a worker threads goes into sleep before starting another iteration used for backoff in case of no launched job or error | -| [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) deprecated. Use metastore.compactor.worker.threads instead. | *Default:* 0*Value required for transactions:* > 0 on at least one instance of the Thrift metastore service | Metastore | How many compactor worker threads to run on this metastore instance.2 | +| [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) deprecated. Use metastore.compactor.worker.threads instead. | *Default:* 0*Value required for transactions:* \> 0 on at least one instance of the Thrift metastore service | Metastore | How many compactor worker threads to run on this metastore instance.2 | | [hive.compactor.worker.timeout]({{< ref "#hive-compactor-worker-timeout" >}}) | *Default:* 86400s | Metastore | Time in seconds after which a compaction job will be declared failed and the compaction re-queued. | | [hive.compactor.cleaner.run.interval]({{< ref "#hive-compactor-cleaner-run-interval" >}}) | *Default*: 5000ms | Metastore | Time in milliseconds between runs of the cleaner thread. ([Hive 0.14.0](https://issues.apache.org/jira/browse/HIVE-8258) and later.) | | [hive.compactor.check.interval]({{< ref "#hive-compactor-check-interval" >}}) | *Default:* 300s | Metastore | Time in seconds between checks to see if any tables or partitions need to be compacted.3 | @@ -236,7 +236,7 @@ If a table owner does not wish the system to automatically determine when to com Table properties are set with the TBLPROPERTIES clause when a table is created or altered, as described in the [Create Table]({{< ref "#create-table" >}}) and [Alter Table Properties]({{< ref "#alter-table-properties" >}}) sections of Hive Data Definition Language. The "`transactional`" and "`NO_AUTO_COMPACTION`" table properties are case-insensitive. -More compaction related options can be set via TBLPROPERTIES. They can be set at both table-level via [CREATE TABLE](/docs/latest/language/languagemanual-ddl#createdroptruncate-table), and on request-level via [ALTER TABLE/PARTITION COMPACT](/docs/latest/language/languagemanual-ddl#alter-tablepartition-compact).  These are used to override the Warehouse/table wide settings.  For example, to override an MR property to affect a compaction job, one can add "compactor.=" in either CREATE TABLE statement or when launching a compaction explicitly via ALTER TABLE.  The "=" will be set on JobConf of the compaction MR job. Similarly, "tblprops.=" can be used to set/override any table property which is interpreted by the code running on the cluster.  Finally, "compactorthreshold.=" can be used to override properties from the "New Configuration Parameters for Transactions" table above that end with ".threshold" and control when compactions are triggered by the system.  Examples: +More compaction related options can be set via TBLPROPERTIES. They can be set at both table-level via [CREATE TABLE](/docs/latest/language/languagemanual-ddl#createdroptruncate-table), and on request-level via [ALTER TABLE/PARTITION COMPACT](/docs/latest/language/languagemanual-ddl#alter-tablepartition-compact).  These are used to override the Warehouse/table wide settings.  For example, to override an MR property to affect a compaction job, one can add "compactor.\=\" in either CREATE TABLE statement or when launching a compaction explicitly via ALTER TABLE.  The "\=\" will be set on JobConf of the compaction MR job. Similarly, "tblprops.\=\" can be used to set/override any table property which is interpreted by the code running on the cluster.  Finally, "compactorthreshold.\=\" can be used to override properties from the "New Configuration Parameters for Transactions" table above that end with ".threshold" and control when compactions are triggered by the system.  Examples: **Example: Set compaction options in TBLPROPERTIES at table level** diff --git a/content/docs/latest/user/configuration-properties.md b/content/docs/latest/user/configuration-properties.md index 50105ef9..b31f8272 100644 --- a/content/docs/latest/user/configuration-properties.md +++ b/content/docs/latest/user/configuration-properties.md @@ -86,7 +86,7 @@ The locations of the plugin jars, which can be comma-separated folders or jars. Scratch space for Hive jobs. This directory is used by Hive to store the plans for different map/reduce stages for the query as well as to stored the intermediate outputs of these stages. -*Hive 0.14.0 and later:*  HDFS root scratch directory for Hive jobs, which gets created with write all (733) permission.  For each connecting user, an HDFS scratch directory ${**hive.exec.scratchdir**}/ is created  with ${ **[hive.scratch.dir.permission]({{< ref "#hivescratchdirpermission" >}})** }. +*Hive 0.14.0 and later:*  HDFS root scratch directory for Hive jobs, which gets created with write all (733) permission.  For each connecting user, an HDFS scratch directory ${**hive.exec.scratchdir**}/\ is created  with ${ **[hive.scratch.dir.permission]({{< ref "#hivescratchdirpermission" >}})** }. Also see  [**hive.start.cleanup.scratchdir**]({{< ref "#**hive-start-cleanup-scratchdir**" >}}) and **[hive.scratchdir.lock]({{< ref "#hivescratchdirlock" >}})** .  When running Hive in local mode, see  [**hive.exec.local.scratchdir**]({{< ref "#**hive-exec-local-scratchdir**" >}}). @@ -1019,7 +1019,7 @@ String used as a file extension for output files. If not set, defaults to the co Whether to insert into multilevel nested directories like "insert directory '/HIVEFT25686/chinna/' from table". The following error may be shown when inserting into a nested directory that does not exist: -*ERROR org.apache.hadoop.hive.ql.exec.Task: Failed with exception Unable to rename: * +*ERROR org.apache.hadoop.hive.ql.exec.Task: Failed with exception Unable to rename: \* To enable automatic subdirectory generation set 'hive.insert.into.multilevel.dirs=true' @@ -1114,7 +1114,7 @@ The maximum memory to be used for hash in RS operator for top K selection. The * Added In: Hive 0.12.0 with [HIVE-4209](https://issues.apache.org/jira/browse/HIVE-4209) * Bug Fix: Hive 0.14.0 with [HIVE-7314](https://issues.apache.org/jira/browse/HIVE-7314) (expression caching doesn't work when using UDF inside another UDF or a Hive function) -If true, the evaluation result of a deterministic expression referenced twice or more will be cached. For example, in a filter condition like "... where key + 10 > 10 or key + 10 = 0" the expression "key + 10" will be evaluated/cached once and reused for the following expression ("key + 10 = 0"). Currently, this is applied only to expressions in select or filter operators. +If true, the evaluation result of a deterministic expression referenced twice or more will be cached. For example, in a filter condition like "... where key + 10 \> 10 or key + 10 = 0" the expression "key + 10" will be evaluated/cached once and reused for the following expression ("key + 10 = 0"). Currently, this is applied only to expressions in select or filter operators. ##### hive.resultset.use.unique.column.names @@ -2324,7 +2324,7 @@ For more information, see [Metastore Schema Consistency and Upgrades]({{< ref "# + Hive 2.x and later: `true` ([HIVE-12320](https://issues.apache.org/jira/browse/HIVE-12320)) * Added In: Hive 0.12.0 with [HIVE-4409](https://issues.apache.org/jira/browse/HIVE-4409) -If true, ALTER TABLE operations which change the type of a column (say STRING) to an incompatible type (say MAP) are disallowed. RCFile default SerDe (ColumnarSerDe) serializes the values in such a way that the datatypes can be converted from string to any type. The map is also serialized as a string, which can be read as a string as well. However, with any binary serialization, this is not true. Blocking the ALTER TABLE prevents ClassCastExceptions when subsequently trying to access old partitions. +If true, ALTER TABLE operations which change the type of a column (say STRING) to an incompatible type (say MAP\) are disallowed. RCFile default SerDe (ColumnarSerDe) serializes the values in such a way that the datatypes can be converted from string to any type. The map is also serialized as a string, which can be read as a string as well. However, with any binary serialization, this is not true. Blocking the ALTER TABLE prevents ClassCastExceptions when subsequently trying to access old partitions. Primitive types like INT, STRING, BIGINT, etc. are compatible with each other and are not blocked. @@ -2344,7 +2344,7 @@ Allow JDO query pushdown for integral partition columns in metastore. Off by def Whether the Hive metastore should try to use direct SQL queries instead of the DataNucleus for certain read paths. This can improve metastore performance when fetching many partitions or column statistics by orders of magnitude; however, it is not guaranteed to work on all RDBMS-es and all versions. In case of SQL failures, the metastore will fall back to the DataNucleus, so it's safe even if SQL doesn't work for all queries on your datastore. If all SQL queries fail (for example, your metastore is backed by MongoDB), you might want to disable this to save the try-and-fall-back cost. -This can be configured on a per client basis by using the "set metaconf:hive.metastore.try.direct.sql=" command, starting with Hive 0.14.0 ( [HIVE-7532](https://issues.apache.org/jira/browse/HIVE-7532)). +This can be configured on a per client basis by using the `set metaconf:hive.metastore.try.direct.sql=` command, starting with Hive 0.14.0 ( [HIVE-7532](https://issues.apache.org/jira/browse/HIVE-7532)). ##### hive.metastore.try.direct.sql.ddl @@ -2353,7 +2353,7 @@ This can be configured on a per client basis by using the "set metaconf:hive.met Same as **[hive.metastore.try.direct.sql]({{< ref "#hivemetastoretrydirectsql" >}})** , for read statements within a transaction that modifies metastore data. Due to non-standard behavior in Postgres, if a direct SQL select query has incorrect syntax or something similar inside a transaction, the entire transaction will fail and fall-back to DataNucleus will not be possible. You should disable the usage of direct SQL inside [transactions]({{< ref "hive-transactions" >}}) if that happens in your case. -This can be configured on a per client basis by using the "set metaconf:hive.metastore.try.direct.sql.ddl=" command, starting with Hive 0.14.0 ( [HIVE-7532](https://issues.apache.org/jira/browse/HIVE-7532)). +This can be configured on a per client basis by using the `set metaconf:hive.metastore.try.direct.sql.ddl=` command, starting with Hive 0.14.0 ( [HIVE-7532](https://issues.apache.org/jira/browse/HIVE-7532)). ##### **hive.metastore.orm.retrieveMapNullsAsEmptyStrings** @@ -2499,7 +2499,7 @@ Maximum number of Thrift worker threads. * Default Value: `60` * Added in: Hive 0.14.0 with [HIVE-7353](https://issues.apache.org/jira/browse/HIVE-7353) - Keepalive time (in seconds) for an idle worker thread. When number of workers > min workers, excess threads are killed after this time interval. + Keepalive time (in seconds) for an idle worker thread. When number of workers \> min workers, excess threads are killed after this time interval. ##### hive.server2.thrift.max.message.size @@ -2576,7 +2576,7 @@ LDAP base DN (distinguished name). * Default Value: `uid` * Added In: Hive 2.1.0 with [HIVE-13295](https://issues.apache.org/jira/browse/HIVE-13295) -This property is to indicate what prefix to use when building the bindDN for LDAP connection (when using just baseDN). So bindDN will be "=,". If userDNPattern and/or groupDNPattern is used in the configuration, the guidKey is not needed. Primarily required when just baseDN is being used. +This property is to indicate what prefix to use when building the bindDN for LDAP connection (when using just baseDN). So bindDN will be `=,`. If userDNPattern and/or groupDNPattern is used in the configuration, the guidKey is not needed. Primarily required when just baseDN is being used. ##### hive.server2.authentication.ldap.Domain @@ -2672,7 +2672,7 @@ The password for the bind domain name. This password may be specified in the con ##### hive.server2.global.init.file.location -* Default Value: $HIVE_CONF_DIR  (typically /conf) +* Default Value: $HIVE_CONF_DIR  (typically \/conf) * Added in Hive 0.14.0 with [HIVE-5160](https://issues.apache.org/jira/browse/HIVE-5160), [HIVE-7497](https://issues.apache.org/jira/browse/HIVE-7497), and [HIVE-8138](https://issues.apache.org/jira/browse/HIVE-8138) Either the location of a HiveServer2 global init file or a directory containing a .hiverc file. If the property is set, the value must be a valid path to an init file or directory where the init file is located. @@ -2726,7 +2726,7 @@ Maximum idle time for a connection on the server when in HTTP mode. * Default Value: 60 * Added In: Hive 0.14.0 in [HIVE-7353](https://issues.apache.org/jira/browse/HIVE-7353) -Keepalive time (in seconds) for an idle http worker thread. When number of workers > min workers, excess threads are killed after this time interval. +Keepalive time (in seconds) for an idle http worker thread. When number of workers \> min workers, excess threads are killed after this time interval. ##### hive.server2.thrift.sasl.qop @@ -3361,7 +3361,7 @@ This is the location that Hive in Tez mode will look for to find a site-wide in * Default Value: `hdfs:///user/` * Added In: Hive 0.13.0 with [HIVE-5003](https://issues.apache.org/jira/browse/HIVE-5003) and [HIVE-6098](https://issues.apache.org/jira/browse/HIVE-6098) -If Hive (in Tez mode only) cannot find a usable Hive jar in **[hive.jar.directory]({{< ref "#hivejardirectory" >}})** , it will upload the Hive jar to <**hive.user.install.directory**>/<*user_name*> and use it to run queries. +If Hive (in Tez mode only) cannot find a usable Hive jar in **[hive.jar.directory]({{< ref "#hivejardirectory" >}})** , it will upload the Hive jar to <**hive.user.install.directory**>/\<*user_name*\> and use it to run queries. ##### [hive.compute.splits.in.am](http://hive.compute.splits.in.am) @@ -3675,7 +3675,7 @@ Maximum allocation possible from LLAP buddy allocator. For ORC, should be as lar * Default Value: 8 * Added In: Hive 2.0.0 with [HIVE-12597](https://issues.apache.org/jira/browse/HIVE-12597) -Arena count for LLAP low-level cache; cache will be allocated in the steps of (size/arena_count) bytes. This size must be <= 1Gb and >= max allocation; if it is not the case, an adjusted size will be used. Using powers of 2 is recommended. +Arena count for LLAP low-level cache; cache will be allocated in the steps of (size/arena_count) bytes. This size must be \<= 1Gb and \>= max allocation; if it is not the case, an adjusted size will be used. Using powers of 2 is recommended. ##### hive.llap.io.memory.size @@ -4280,7 +4280,7 @@ Whether column accesses are tracked in the QueryPlan. This is useful to identif * Default Value: `200` (Hive 0.11 and 0.12) or ``150``  ([Hive 0.13](https://issues.apache.org/jira/browse/HIVE-5559) and later) * Added In: Hive 0.11 with [HIVE-3750](https://issues.apache.org/jira/browse/HIVE-3750) -Determines if, when the prefix of the key used for intermediate statistics collection exceeds a certain length, a hash of the key is used instead. If the value < 0 then hashing is never used, if the value >= 0 then hashing is used only when the key prefixes' length exceeds that value. The key prefix is defined as everything preceding the task ID in the key. For counter type statistics, it's maxed by **[mapreduce.job.counters.group.name.max](https://hadoop.apache.org/docs/r1.2.1/mapred-default.html)** , which is by default 128. +Determines if, when the prefix of the key used for intermediate statistics collection exceeds a certain length, a hash of the key is used instead. If the value \< 0 then hashing is never used, if the value \>= 0 then hashing is used only when the key prefixes' length exceeds that value. The key prefix is defined as everything preceding the task ID in the key. For counter type statistics, it's maxed by **[mapreduce.job.counters.group.name.max](https://hadoop.apache.org/docs/r1.2.1/mapred-default.html)** , which is by default 128. ##### hive.stats.key.prefix.reserve.length @@ -5023,7 +5023,7 @@ Set this to true to enable the use of scratch directories directly on blob stora * Default value: `0` (disabled) * Added In: Hive 2.2.0 with [HIVE-15881](https://issues.apache.org/jira/browse/HIVE-15881) -Set this to a maximum number of threads that Hive will use to list file information from file systems, such as file size and number of files per table (recommended > 1 for blobstore). +Set this to a maximum number of threads that Hive will use to list file information from file systems, such as file size and number of files per table (recommended \> 1 for blobstore). ## Test Properties diff --git a/content/docs/latest/user/hive-deprecated-authorization-mode.md b/content/docs/latest/user/hive-deprecated-authorization-mode.md index 8e8b5e92..5b6a6e84 100644 --- a/content/docs/latest/user/hive-deprecated-authorization-mode.md +++ b/content/docs/latest/user/hive-deprecated-authorization-mode.md @@ -37,7 +37,7 @@ Note that, by default, the [hive.security.authorization.createtable.owner.grants At the core of Hive's authorization system are users, groups, and roles. Roles allow administrators to give a name to a set of grants which can be easily reused. A role may be assigned to users, groups, and other roles. For example, consider a system with the following users and groups: -* : +* \: \ * user_all_dbs: group_db1, group_db2 * user_db1: group_db1 * user_db2: group_db2 @@ -50,11 +50,11 @@ Hive roles must be created manually before being used, unlike users and groups. * Group privileges (Does the user belong to any groups that the privilege has been granted to) * Role privileges (Does the user or any of the groups that the user belongs to have a role that grants the privilege) -By default, the Metastore uses the HadoopDefaultAuthenticator for determing user -> group mappings, which determines authorization by using the Unix usernames and groups on the machine where the Metastore is running. To make this more clear, consider a scenario where a user foo is a member of group bar on the machine running the Hive CLI, and connects to a Metastore running on a separate server that also has a user named foo, but on the Metastore Server, foo is a member of group baz. When an operation is executed, the Metastore will determine foo to be in the group baz. +By default, the Metastore uses the HadoopDefaultAuthenticator for determing user -\> group mappings, which determines authorization by using the Unix usernames and groups on the machine where the Metastore is running. To make this more clear, consider a scenario where a user foo is a member of group bar on the machine running the Hive CLI, and connects to a Metastore running on a separate server that also has a user named foo, but on the Metastore Server, foo is a member of group baz. When an operation is executed, the Metastore will determine foo to be in the group baz. -Taking this a step further, it is also possible for the groups that a user belongs to on the Metastore Server may differ from the groups that the same user belongs to, as determined by HDFS. This could be the case if Hive or HDFS are configured to use non-default user -> group mappers, or the Metastore and the Namenode both use the defaults, but the processes are running on different machines, and the user -> group mappings are not the same on each machine. +Taking this a step further, it is also possible for the groups that a user belongs to on the Metastore Server may differ from the groups that the same user belongs to, as determined by HDFS. This could be the case if Hive or HDFS are configured to use non-default user -\> group mappers, or the Metastore and the Namenode both use the defaults, but the processes are running on different machines, and the user -\> group mappings are not the same on each machine. -It is important to realize that Hive Metastore only controls authorization for metadata, and the underlying data is controlled by HDFS, so if permissions and privileges between the two systems are not in sync, users may have access to metadata, but not the physical data. If the user -> group mappings across the Metastore and Namenode are not in sync, as in the scenarios above, a user may have the privileges required to access a table according to the Metastore, but may not have permission to access the underlying files according to the Namenode. This could also happen due to administrator intervention, if permissions on the files were changed by hand, but Metastore grants had not been updated. +It is important to realize that Hive Metastore only controls authorization for metadata, and the underlying data is controlled by HDFS, so if permissions and privileges between the two systems are not in sync, users may have access to metadata, but not the physical data. If the user -\> group mappings across the Metastore and Namenode are not in sync, as in the scenarios above, a user may have the privileges required to access a table according to the Metastore, but may not have permission to access the underlying files according to the Namenode. This could also happen due to administrator intervention, if permissions on the files were changed by hand, but Metastore grants had not been updated. #### Names of Users and Roles diff --git a/content/docs/latest/user/hive-iceberg-integration.md b/content/docs/latest/user/hive-iceberg-integration.md index 815fafc2..fe1dc5fc 100644 --- a/content/docs/latest/user/hive-iceberg-integration.md +++ b/content/docs/latest/user/hive-iceberg-integration.md @@ -49,7 +49,7 @@ CREATE TABLE V2_TABLE (ID INT) STORED BY ICEBERG TBLPROPERTIES ('format-version' **File Formats:** -The iceberg table currently supports three file formats: PARQUET, ORC & AVRO. The default file format is Parquet. The file format can be explicitily provided by using STORED AS while creating the table +The iceberg table currently supports three file formats: PARQUET, ORC & AVRO. The default file format is Parquet. The file format can be explicitily provided by using STORED AS \ while creating the table Example-1: @@ -84,7 +84,7 @@ CREATE TABLE tbl_x (id int) STORED BY ICEBERG TBLPROPERTIES ( **Migrating existing tables to Iceberg Tables** -Any Hive external table can be converted into an iceberg tables, without actually rewriting the data files again. We can use _ALTER TABLE
CONVERT TO ICEBERG [TBLPROPERTIES]_ to convert any existing external table to an iceberg table. +Any Hive external table can be converted into an iceberg tables, without actually rewriting the data files again. We can use _ALTER TABLE \
CONVERT TO ICEBERG [TBLPROPERTIES]_ to convert any existing external table to an iceberg table. ``` ALTER TABLE TABLE1 CONVERT TO ICEBERG TBLPROPERTIES ('format-version'='2'); diff --git a/content/docs/latest/user/hive-transactions.md b/content/docs/latest/user/hive-transactions.md index f97aca10..dea8a6c5 100644 --- a/content/docs/latest/user/hive-transactions.md +++ b/content/docs/latest/user/hive-transactions.md @@ -101,7 +101,7 @@ This module is responsible for discovering which tables or partitions are due fo #### Worker -Each Worker handles a single compaction task.  A compaction is a MapReduce job with name in the following form: -compactor-.
..  Each worker submits the job to the cluster (via [hive.compactor.job.queue]({{< ref "#hive-compactor-job-queue" >}}) if defined) and waits for the job to finish.  [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) determines the number of Workers in each Metastore.  The total number of Workers in the Hive Warehouse determines the maximum number of concurrent compactions. +Each Worker handles a single compaction task.  A compaction is a MapReduce job with name in the following form: \-compactor-\.\.\.  Each worker submits the job to the cluster (via [hive.compactor.job.queue]({{< ref "#hive-compactor-job-queue" >}}) if defined) and waits for the job to finish.  [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) determines the number of Workers in each Metastore.  The total number of Workers in the Hive Warehouse determines the maximum number of concurrent compactions. #### Cleaner @@ -166,7 +166,7 @@ A number of new configuration parameters have been added to the system to suppor | [hive.txn.retryable.sqlex.regex]({{< ref "#hive-txn-retryable-sqlex-regex" >}}) | *Default:* "" (empty string) | HiveServer2/ Metastore | Comma separated list of regular expression patterns for SQL state, error code, and error message of retryable SQLExceptions, that's suitable for the Hive metastore database (as of [Hive 1.3.0 and 2.1.0](https://issues.apache.org/jira/browse/HIVE-12637)).For an example, see [Configuration Properties]({{< ref "#configuration-properties" >}}). | | [hive.compactor.initiator.on]({{< ref "#hive-compactor-initiator-on" >}}) | *Default:* false*Value required for transactions:* true (for exactly one instance of the Thrift metastore service) | Metastore | Whether to run the initiator thread on this metastore instance. Prior to [Hive 1.3.0](https://issues.apache.org/jira/browse/HIVE-11388) it's critical that this is enabled on exactly one standalone metastore service instance (not enforced yet).As of [Hive 1.3.0](https://issues.apache.org/jira/browse/HIVE-11388) this property may be enabled on any number of standalone metastore instances. | | [hive.compactor.cleaner.on]({{< ref "#hive-compactor-cleaner-on" >}}) | *Default:* false*Value required for transactions:* true (for exactly one instance of the Thrift metastore service) | Metastore | Whether to run the cleaner thread on this metastore instance. Before **Hive 4.0.0** Cleaner thread can be started/stopped with config hive.compactor.initiator.on. This config helps to enable/disable initiator/cleaner threads independently | -| [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) | *Default:* 0*Value required for transactions:* > 0 on at least one instance of the Thrift metastore service | Metastore | How many compactor worker threads to run on this metastore instance.2 | +| [hive.compactor.worker.threads]({{< ref "#hive-compactor-worker-threads" >}}) | *Default:* 0*Value required for transactions:* \> 0 on at least one instance of the Thrift metastore service | Metastore | How many compactor worker threads to run on this metastore instance.2 | | [hive.compactor.worker.timeout]({{< ref "#hive-compactor-worker-timeout" >}}) | *Default:* 86400 | Metastore | Time in seconds after which a compaction job will be declared failed and the compaction re-queued. | | [hive.compactor.cleaner.run.interval]({{< ref "#hive-compactor-cleaner-run-interval" >}}) | *Default*: 5000 | Metastore | Time in milliseconds between runs of the cleaner thread. ([Hive 0.14.0](https://issues.apache.org/jira/browse/HIVE-8258) and later.) | | [hive.compactor.check.interval]({{< ref "#hive-compactor-check-interval" >}}) | *Default:* 300 | Metastore | Time in seconds between checks to see if any tables or partitions need to be compacted.3 | @@ -219,7 +219,7 @@ If a table owner does not wish the system to automatically determine when to com Table properties are set with the TBLPROPERTIES clause when a table is created or altered, as described in the [Create Table]({{< ref "#create-table" >}}) and [Alter Table Properties]({{< ref "#alter-table-properties" >}}) sections of Hive Data Definition Language. The "`transactional`" and "`NO_AUTO_COMPACTION`" table properties are case-sensitive in Hive releases 0.x and 1.0, but they are case-insensitive starting with release 1.1.0 ([HIVE-8308](https://issues.apache.org/jira/browse/HIVE-8308)). -More compaction related options can be set via TBLPROPERTIES as of [Hive 1.3.0 and 2.1.0](https://issues.apache.org/jira/browse/HIVE-13354). They can be set at both table-level via [CREATE TABLE](/docs/latest/language/languagemanual-ddl#createdroptruncate-table), and on request-level via [ALTER TABLE/PARTITION COMPACT](/docs/latest/language/languagemanual-ddl#alter-tablepartition-compact).  These are used to override the Warehouse/table wide settings.  For example, to override an MR property to affect a compaction job, one can add "compactor.=" in either CREATE TABLE statement or when launching a compaction explicitly via ALTER TABLE.  The "=" will be set on JobConf of the compaction MR job.   Similarly, "tblprops.=" can be used to set/override any table property which is interpreted by the code running on the cluster.  Finally, "compactorthreshold.=" can be used to override properties from the "New Configuration Parameters for Transactions" table above that end with ".threshold" and control when compactions are triggered by the system.  Examples: +More compaction related options can be set via TBLPROPERTIES as of [Hive 1.3.0 and 2.1.0](https://issues.apache.org/jira/browse/HIVE-13354). They can be set at both table-level via [CREATE TABLE](/docs/latest/language/languagemanual-ddl#createdroptruncate-table), and on request-level via [ALTER TABLE/PARTITION COMPACT](/docs/latest/language/languagemanual-ddl#alter-tablepartition-compact).  These are used to override the Warehouse/table wide settings.  For example, to override an MR property to affect a compaction job, one can add "compactor.\=\" in either CREATE TABLE statement or when launching a compaction explicitly via ALTER TABLE.  The "\=\" will be set on JobConf of the compaction MR job.   Similarly, "tblprops.\=\" can be used to set/override any table property which is interpreted by the code running on the cluster.  Finally, "compactorthreshold.\=\" can be used to override properties from the "New Configuration Parameters for Transactions" table above that end with ".threshold" and control when compactions are triggered by the system.  Examples: **Example: Set compaction options in TBLPROPERTIES at table level** diff --git a/content/docs/latest/user/hiveserver2-clients.md b/content/docs/latest/user/hiveserver2-clients.md index 5e9adf55..bd283c6d 100644 --- a/content/docs/latest/user/hiveserver2-clients.md +++ b/content/docs/latest/user/hiveserver2-clients.md @@ -74,8 +74,8 @@ beeline> !connect jdbc:hive2://:/;auth=noSasl hiveuser pass | Command | Description | | --- | --- | -| ! | List of SQLLine commands available at .Example: `!quit` exits the Beeline client. | -| !delimiter | Set the delimiter for queries written in Beeline. Multi-character delimiters are allowed, but quotation marks, slashes, and -- are not allowed. Defaults to ;Usage: `!delimiter $$`Version: 3.0.0 ([HIVE-10865](https://issues.apache.org/jira/browse/HIVE-10865)) | +| `!` | List of SQLLine commands available at .Example: `!quit` exits the Beeline client. | +| `!delimiter` | Set the delimiter for queries written in Beeline. Multi-character delimiters are allowed, but quotation marks, slashes, and -- are not allowed. Defaults to ;Usage: `!delimiter $$`Version: 3.0.0 ([HIVE-10865](https://issues.apache.org/jira/browse/HIVE-10865)) | ### Beeline Properties @@ -95,20 +95,20 @@ Use "`;`" (semicolon) to terminate commands. Comments in scripts can be specifie | Command | Description | | --- | --- | -| reset | Resets the configuration to the default values. | -| reset | Resets the value of a particular configuration variable (key) to the default value.**Note:** If you misspell the variable name, Beeline will not show an error. | -| set = | Sets the value of a particular configuration variable (key). **Note:** If you misspell the variable name, Beeline will not show an error. | -| set | Prints a list of configuration variables that are overridden by the user or Hive. | -| set -v | Prints all Hadoop and Hive configuration variables. | -| add FILE[S] * add JAR[S] * add ARCHIVE[S] * | Adds one or more files, jars, or archives to the list of resources in the distributed cache. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | -| add FILE[S] * add JAR[S]  * add ARCHIVE[S] * | As of [Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-9664), adds one or more files, jars or archives to the list of resources in the distributed cache using an [Ivy](http://ant.apache.org/ivy/) URL of the form ivy://group:module:version?query_string. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | +| `reset` | Resets the configuration to the default values. | +| `reset ` | Resets the value of a particular configuration variable (key) to the default value.**Note:** If you misspell the variable name, Beeline will not show an error. | +| `set =` | Sets the value of a particular configuration variable (key). **Note:** If you misspell the variable name, Beeline will not show an error. | +| `set` | Prints a list of configuration variables that are overridden by the user or Hive. | +| `set -v` | Prints all Hadoop and Hive configuration variables. | +| `add FILE[S] *` `add JAR[S] *` `add ARCHIVE[S] *` | Adds one or more files, jars, or archives to the list of resources in the distributed cache. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | +| `add FILE[S] *`  `add JAR[S]  *`  `add ARCHIVE[S] *` | As of [Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-9664), adds one or more files, jars or archives to the list of resources in the distributed cache using an [Ivy](http://ant.apache.org/ivy/) URL of the form ivy://group:module:version?query_string. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | | list FILE[S] list JAR[S] list ARCHIVE[S] | Lists the resources already added to the distributed cache. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. (As of Hive 0.14.0: [HIVE-7592](https://issues.apache.org/jira/browse/HIVE-7592)). | -| list FILE[S] * list JAR[S] * list ARCHIVE[S] * | Checks whether the given resources are already added to the distributed cache or not. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | -| delete FILE[S] * delete JAR[S] * delete ARCHIVE[S] * | Removes the resource(s) from the distributed cache. | -| delete FILE[S] * delete JAR[S] * delete ARCHIVE[S] * | As of [Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-9664), removes the resource(s) which were added using the from the distributed cache. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | -| reload | As of [Hive 0.14.0](https://issues.apache.org/jira/browse/HIVE-7553), makes HiveServer2 aware of any jar changes in the path specified by the configuration parameter [hive.reloadable.aux.jars.path]({{< ref "#hive-reloadable-aux-jars-path" >}}) (without needing to restart HiveServer2). The changes can be adding, removing, or updating jar files. | -| dfs | Executes a dfs command. | -| | Executes a Hive query and prints results to standard output. | +| `list FILE[S] *` `list JAR[S] *` `list ARCHIVE[S] *` | Checks whether the given resources are already added to the distributed cache or not. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | +| `delete FILE[S] *` `delete JAR[S] *` `delete ARCHIVE[S] *` | Removes the resource(s) from the distributed cache. | +| `delete FILE[S] *`  `delete JAR[S] *`  `delete ARCHIVE[S] *` | As of [Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-9664), removes the resource(s) which were added using the \ from the distributed cache. See [Hive Resources]({{< ref "#hive-resources" >}}) for more information. | +| `reload` | As of [Hive 0.14.0](https://issues.apache.org/jira/browse/HIVE-7553), makes HiveServer2 aware of any jar changes in the path specified by the configuration parameter [hive.reloadable.aux.jars.path]({{< ref "#hive-reloadable-aux-jars-path" >}}) (without needing to restart HiveServer2). The changes can be adding, removing, or updating jar files. | +| `dfs ` | Executes a dfs command. | +| `` | Executes a Hive query and prints results to standard output. | ### Beeline Command Options @@ -116,45 +116,45 @@ The Beeline CLI supports these command line options: | Option | Description | | --- | --- | -| **-u** ** | The JDBC URL to connect to. Special characters in parameter values should be encoded with URL encoding if needed.Usage: `beeline -u` *db_URL*  | -| **-r** | [Reconnect]({{< ref "#reconnect" >}}) to last used URL (if a user has previously used `!connect` to a URL and used `!save` to a beeline.properties file).Usage: `beeline -r`Version: 2.1.0 ([HIVE-13670](https://issues.apache.org/jira/browse/HIVE-13670)) | -| **-n** ** | The username to connect as.Usage: `beeline -n` *valid_user* | -| **-p** ** | The password to connect as.Usage: `beeline -p` *valid_password*Optional password mode:Starting Hive 2.2.0 ([HIVE-13589](https://issues.apache.org/jira/browse/HIVE-13589)) the argument for -p option is optional.Usage : beeline -p [valid_password]If the password is not provided after -p Beeline will prompt for the password while initiating the connection. When password is provided Beeline uses it initiate the connection without prompting. | -| **-d** ** | The driver class to use.Usage: `beeline -d` *driver_class* | -| **-e** ** | Query that should be executed. Double or single quotes enclose the query string. This option can be specified multiple times.Usage: `beeline -e "`*query_string*"Support to run multiple SQL statements separated by semicolons in a single *query_string*: 1.2.0 ([HIVE-9877](https://issues.apache.org/jira/browse/HIVE-9877))Bug fix (null pointer exception): 0.13.0 ([HIVE-5765](https://issues.apache.org/jira/browse/HIVE-5765))Bug fix (--headerInterval not honored): 0.14.0 ([HIVE-7647](https://issues.apache.org/jira/browse/HIVE-7647))Bug fix (running `-e` in background): 1.3.0 and 2.0.0 ([HIVE-6758](https://issues.apache.org/jira/browse/HIVE-6758)); [workaround available](https://issues.apache.org/jira/browse/HIVE-6758?focusedCommentId=13954968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13954968) for earlier versions  | -| **-f** ** | Script file that should be executed.Usage: `beeline -f` *filepath*Version: 0.12.0 ([HIVE-4268](https://issues.apache.org/jira/browse/HIVE-4268))Note: If the script contains tabs, query compilation fails in version 0.12.0. This bug is fixed in version 0.13.0 ([HIVE-6359](https://issues.apache.org/jira/browse/HIVE-6359)).Bug fix (running `-f` in background): 1.3.0 and 2.0.0 ([HIVE-6758](https://issues.apache.org/jira/browse/HIVE-6758)); [workaround available](https://issues.apache.org/jira/browse/HIVE-6758?focusedCommentId=13954968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13954968) for earlier versions  | -| **-i** (or) **--init** | The init files for initializationUsage: `beeline -i */tmp/initfile*`Single file:Version: 0.14.0 ([HIVE-6561](https://issues.apache.org/jira/browse/HIVE-6561))Multiple files:Version: 2.1.0 ([HIVE-11336](https://issues.apache.org/jira/browse/HIVE-11336)) | -| **-w** (or) **--password-file** | The password file to read password from.Version: 1.2.0 ([HIVE-7175](https://issues.apache.org/jira/browse/HIVE-7175)) | -| **-a** (or) **--authType** | The authentication type passed to the jdbc as an auth propertyVersion: 0.13.0 ([HIVE-5155](https://issues.apache.org/jira/browse/HIVE-5155)) | -| **--property-file** | File to read configuration properties fromUsage: `beeline --property-file */tmp/a*`Version: 2.2.0 ([HIVE-13964](https://issues.apache.org/jira/browse/HIVE-13964)) | -| **--hiveconf** *property**=**value* | Use *value* for the given configuration property. Properties that are listed in hive.conf.restricted.list cannot be reset with **hiveconf** (see [Restricted List and Whitelist]({{< ref "#restricted-list-and-whitelist" >}})).Usage: `beeline --hiveconf` *prop1*`=`*value1*Version: 0.13.0 ([HIVE-6173](https://issues.apache.org/jira/browse/HIVE-6173)) | -| **--hivevar** *name**=**value* | Hive variable name and value. This is a Hive-specific setting in which variables can be set at the session level and referenced in Hive commands or queries.Usage: `beeline --hivevar` *var1*`=`*value1* | -| **--color=**[true/false] | Control whether color is used for display. Default is false.Usage: `beeline --color=true`(Not supported for Separated-Value Output formats. See [HIVE-9770](https://issues.apache.org/jira/browse/HIVE-9770)) | -| **--showHeader=**[true/false] | Show column names in query results (true) or not (false). Default is true.Usage: `beeline --showHeader=false` | -| **--headerInterval=**ROWS | The interval for redisplaying column headers, in number of rows, when **outputformat** is table. Default is 100.Usage: `beeline --headerInterval=50`(Not supported for Separated-Value Output formats. See [HIVE-9770](https://issues.apache.org/jira/browse/HIVE-9770)) | -| **--fastConnect=**[true/false] | When connecting, skip building a list of all tables and columns for tab-completion of HiveQL statements (true) or build the list (false). Default is true.Usage: `beeline --fastConnect=false` | -| **--autoCommit=**[true/false] | Enable/disable automatic transaction commit. Default is false.Usage: `beeline --autoCommit=true` | -| **--verbose=**[true/false] | Show verbose error messages and debug information (true) or do not show (false). Default is false.Usage: `beeline --verbose=true` | -| **--showWarnings=**[true/false] | Display warnings that are reported on the connection after issuing any HiveQL commands. Default is false.Usage: `beeline --showWarnings=true` | -| ****--**showDbInPrompt**=****[true/false] | Display the current database name in prompt. Default is false.Usage: `beeline --showDbInPrompt=true`Version: 2.2.0 ([HIVE-14123](https://issues.apache.org/jira/browse/HIVE-14123)) | -| **--showNestedErrs=**[true/false] | Display nested errors. Default is false.Usage: `beeline --showNestedErrs=true` | -| **--numberFormat=**[pattern] | Format numbers using a [DecimalFormat](http://docs.oracle.com/javase/7/docs/api/java/text/DecimalFormat.html) pattern.Usage: `beeline --numberFormat="#,###,##0.00"` | -| **--force=**[true/false] | Continue running script even after errors (true) or do not continue (false). Default is false.Usage: `beeline--force=true` | -| **--maxWidth=**MAXWIDTH | The maximum width to display before truncating data, in characters, when **outputformat** is table. Default is to query the terminal for current width, then fall back to 80.Usage: `beeline --maxWidth=150` | -| **--maxColumnWidth=**MAXCOLWIDTH | The maximum column width, in characters, when **outputformat** is table. Default is 50 in Hive version 2.2.0+ (see [HIVE-14135](https://issues.apache.org/jira/browse/HIVE-14135)) or 15 in earlier versions.Usage: `beeline --maxColumnWidth=25` | -| **--silent=**[true/false] | Reduce the amount of informational messages displayed (true) or not (false). It also stops displaying the log messages for the query from HiveServer2 ([Hive 0.14](https://issues.apache.org/jira/browse/HIVE-7615) and later) and the HiveQL commands ([Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-10202) and later). Default is false.Usage: `beeline --silent=true` | -| **--autosave=**[true/false] | Automatically save preferences (true) or do not autosave (false). Default is false.Usage: `beeline --autosave=true` | -| **--outputformat=**[table/vertical/csv/tsv/dsv/csv2/tsv2] | Format mode for result display. Default is table. See [Separated-Value Output Formats]({{< ref "#separated-value-output-formats" >}}) below for description of recommended sv options.Usage: `beeline --outputformat=tsv`Version: dsv/csv2/tsv2 added in 0.14.0 ([HIVE-8615](https://issues.apache.org/jira/browse/HIVE-8615)) | -| **--****truncateTable**=[true/false] | If true, truncates table column in the console when it exceeds console length.Version: 0.14.0 ([HIVE-6928](https://issues.apache.org/jira/browse/HIVE-6928)) | -| **--delimiterForDSV=** DELIMITER | The delimiter for delimiter-separated values output format. Default is '|' character.Version: 0.14.0 ([HIVE-7390](https://issues.apache.org/jira/browse/HIVE-7390)) | -| **--isolation=**LEVEL | Set the transaction isolation level to TRANSACTION_READ_COMMITTED or TRANSACTION_SERIALIZABLE. See the "Field Detail" section in the Java [Connection](http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html) documentation.Usage: `beeline --isolation=TRANSACTION_SERIALIZABLE` | -| **--nullemptystring=**[true/false] | Use historic behavior of printing null as empty string (true) or use current behavior of printing null as NULL (false). Default is false.Usage: `beeline --nullemptystring=false`Version: 0.13.0 ([HIVE-4485](https://issues.apache.org/jira/browse/HIVE-4485)) | -| **--incremental=**[true/false] | Defaults to `true` from Hive 2.3 onwards, before it defaulted to `false``.` When set to `false`, the entire result set is fetched and buffered before being displayed, yielding optimal display column sizing. When set to `true`, result rows are displayed immediately as they are fetched, yielding lower latency and memory usage at the price of extra display column padding. Setting `--incremental=true` is recommended if you encounter an OutOfMemory on the client side (due to the fetched result set size being large). | -| ****--incrementalBufferRows=****NUMROWS | The number of rows to buffer when printing rows on stdout, defaults to 1000; only applicable if `--incremental=true` and `--outputformat=table`Usage: `beeline --incrementalBufferRows=1000`Version: 2.3.0 ([HIVE-14170](https://issues.apache.org/jira/browse/HIVE-14170)) | -| **--maxHistoryRows****=******NUMROWS | The maximum number of rows to store Beeline history.Version: 2.3.0 ([HIVE-15166](https://issues.apache.org/jira/browse/HIVE-15166)) | -| **--delimiter=**; | Set the delimiter for queries written in Beeline. Multi-char delimiters are allowed, but quotation marks, slashes, and -- are not allowed. Defaults to ;Usage: `beeline --delimiter=$$`Version: 3.0.0 ([HIVE-10865](https://issues.apache.org/jira/browse/HIVE-10865)) | -| **--convertBinaryArrayToString=**[true/false] | Display binary column data as a string using the platform's default character set.The default behavior (false) is to display binary data using: `Arrays.toString(byte[] columnValue)`Version: 3.0.0 ([HIVE-14786](https://issues.apache.org/jira/browse/HIVE-14786))Display binary column data as a string using the UTF-8 character set.The default behavior (false) is to display binary data using Base64 encoding without padding.Version: 4.0.0 ([HIVE-2](https://issues.apache.org/jira/browse/HIVE-14786)[3856](https://issues.apache.org/jira/browse/HIVE-23856))Usage: `beeline --convertBinaryArrayToString=true` | -| **--help** | Display a usage message.Usage: `beeline --help` | +| `-u ` | The JDBC URL to connect to. Special characters in parameter values should be encoded with URL encoding if needed.Usage: `beeline -u` *db_URL*  | +| `-r` | [Reconnect]({{< ref "#reconnect" >}}) to last used URL (if a user has previously used `!connect` to a URL and used `!save` to a beeline.properties file).Usage: `beeline -r`Version: 2.1.0 ([HIVE-13670](https://issues.apache.org/jira/browse/HIVE-13670)) | +| `-n ` | The username to connect as.Usage: `beeline -n` valid_user | +| `-p ` | The password to connect as.Usage: `beeline -p` valid_passwordOptional password mode:Starting Hive 2.2.0 ([HIVE-13589](https://issues.apache.org/jira/browse/HIVE-13589)) the argument for -p option is optional.Usage : beeline -p [valid_password]If the password is not provided after -p Beeline will prompt for the password while initiating the connection. When password is provided Beeline uses it initiate the connection without prompting. | +| `-d ` | The driver class to use.Usage: `beeline -d` driver_class | +| `-e ` | Query that should be executed. Double or single quotes enclose the query string. This option can be specified multiple times.Usage: `beeline -e "`query_string"Support to run multiple SQL statements separated by semicolons in a single query_string: 1.2.0 ([HIVE-9877](https://issues.apache.org/jira/browse/HIVE-9877))Bug fix (null pointer exception): 0.13.0 ([HIVE-5765](https://issues.apache.org/jira/browse/HIVE-5765))Bug fix (--headerInterval not honored): 0.14.0 ([HIVE-7647](https://issues.apache.org/jira/browse/HIVE-7647))Bug fix (running `-e` in background): 1.3.0 and 2.0.0 ([HIVE-6758](https://issues.apache.org/jira/browse/HIVE-6758)); [workaround available](https://issues.apache.org/jira/browse/HIVE-6758?focusedCommentId=13954968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13954968) for earlier versions  | +| `-f ` | Script file that should be executed.Usage: `beeline -f` filepathVersion: 0.12.0 ([HIVE-4268](https://issues.apache.org/jira/browse/HIVE-4268))Note: If the script contains tabs, query compilation fails in version 0.12.0. This bug is fixed in version 0.13.0 ([HIVE-6359](https://issues.apache.org/jira/browse/HIVE-6359)).Bug fix (running `-f` in background): 1.3.0 and 2.0.0 ([HIVE-6758](https://issues.apache.org/jira/browse/HIVE-6758)); [workaround available](https://issues.apache.org/jira/browse/HIVE-6758?focusedCommentId=13954968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13954968) for earlier versions  | +| `-i` (or) `--init ` | The init files for initializationUsage: `beeline -i /tmp/initfile`Single file:Version: 0.14.0 ([HIVE-6561](https://issues.apache.org/jira/browse/HIVE-6561))Multiple files:Version: 2.1.0 ([HIVE-11336](https://issues.apache.org/jira/browse/HIVE-11336)) | +| `-w` (or) `--password-file ` | The password file to read password from.Version: 1.2.0 ([HIVE-7175](https://issues.apache.org/jira/browse/HIVE-7175)) | +| `-a` (or) `--authType ` | The authentication type passed to the jdbc as an auth propertyVersion: 0.13.0 ([HIVE-5155](https://issues.apache.org/jira/browse/HIVE-5155)) | +| `--property-file ` | File to read configuration properties fromUsage: `beeline --property-file /tmp/a`Version: 2.2.0 ([HIVE-13964](https://issues.apache.org/jira/browse/HIVE-13964)) | +| `--hiveconf property=value` | Use value for the given configuration property. Properties that are listed in hive.conf.restricted.list cannot be reset with hiveconf (see [Restricted List and Whitelist]({{< ref "#restricted-list-and-whitelist" >}})).Usage: `beeline --hiveconf` prop1`=`value1Version: 0.13.0 ([HIVE-6173](https://issues.apache.org/jira/browse/HIVE-6173)) | +| `--hivevar name=value` | Hive variable name and value. This is a Hive-specific setting in which variables can be set at the session level and referenced in Hive commands or queries.Usage: `beeline --hivevar` var1`=`value1 | +| `--color=[true/false]` | Control whether color is used for display. Default is false.Usage: `beeline --color=true`(Not supported for Separated-Value Output formats. See [HIVE-9770](https://issues.apache.org/jira/browse/HIVE-9770)) | +| `--showHeader=[true/false]` | Show column names in query results (true) or not (false). Default is true.Usage: `beeline --showHeader=false` | +| `--headerInterval=ROWS` | The interval for redisplaying column headers, in number of rows, when outputformat is table. Default is 100.Usage: `beeline --headerInterval=50`(Not supported for Separated-Value Output formats. See [HIVE-9770](https://issues.apache.org/jira/browse/HIVE-9770)) | +| `--fastConnect=[true/false]` | When connecting, skip building a list of all tables and columns for tab-completion of HiveQL statements (true) or build the list (false). Default is true.Usage: `beeline --fastConnect=false` | +| `--autoCommit=[true/false]` | Enable/disable automatic transaction commit. Default is false.Usage: `beeline --autoCommit=true` | +| `--verbose=[true/false]` | Show verbose error messages and debug information (true) or do not show (false). Default is false.Usage: `beeline --verbose=true` | +| `--showWarnings=[true/false]` | Display warnings that are reported on the connection after issuing any HiveQL commands. Default is false.Usage: `beeline --showWarnings=true` | +| `--showDbInPrompt=[true/false]` | Display the current database name in prompt. Default is false.Usage: `beeline --showDbInPrompt=true`Version: 2.2.0 ([HIVE-14123](https://issues.apache.org/jira/browse/HIVE-14123)) | +| `--showNestedErrs=[true/false]` | Display nested errors. Default is false.Usage: `beeline --showNestedErrs=true` | +| `--numberFormat=[pattern]` | Format numbers using a [DecimalFormat](http://docs.oracle.com/javase/7/docs/api/java/text/DecimalFormat.html) pattern.Usage: `beeline --numberFormat="#,###,##0.00"` | +| `--force=[true/false]` | Continue running script even after errors (true) or do not continue (false). Default is false.Usage: `beeline--force=true` | +| `--maxWidth=MAXWIDTH` | The maximum width to display before truncating data, in characters, when outputformat is table. Default is to query the terminal for current width, then fall back to 80.Usage: `beeline --maxWidth=150` | +| `--maxColumnWidth=MAXCOLWIDTH` | The maximum column width, in characters, when outputformat is table. Default is 50 in Hive version 2.2.0+ (see [HIVE-14135](https://issues.apache.org/jira/browse/HIVE-14135)) or 15 in earlier versions.Usage: `beeline --maxColumnWidth=25` | +| `--silent=[true/false]` | Reduce the amount of informational messages displayed (true) or not (false). It also stops displaying the log messages for the query from HiveServer2 ([Hive 0.14](https://issues.apache.org/jira/browse/HIVE-7615) and later) and the HiveQL commands ([Hive 1.2.0](https://issues.apache.org/jira/browse/HIVE-10202) and later). Default is false.Usage: `beeline --silent=true` | +| `--autosave=[true/false]` | Automatically save preferences (true) or do not autosave (false). Default is false.Usage: `beeline --autosave=true` | +| `--outputformat=[table/vertical/csv/tsv/dsv/csv2/tsv2]` | Format mode for result display. Default is table. See [Separated-Value Output Formats]({{< ref "#separated-value-output-formats" >}}) below for description of recommended sv options.Usage: `beeline --outputformat=tsv`Version: dsv/csv2/tsv2 added in 0.14.0 ([HIVE-8615](https://issues.apache.org/jira/browse/HIVE-8615)) | +| `--truncateTable=[true/false] ` | If true, truncates table column in the console when it exceeds console length.Version: 0.14.0 ([HIVE-6928](https://issues.apache.org/jira/browse/HIVE-6928)) | +| `--delimiterForDSV= DELIMITER` | The delimiter for delimiter-separated values output format. Default is '|' character.Version: 0.14.0 ([HIVE-7390](https://issues.apache.org/jira/browse/HIVE-7390)) | +| `--isolation=LEVEL` | Set the transaction isolation level to TRANSACTION_READ_COMMITTED or TRANSACTION_SERIALIZABLE. See the "Field Detail" section in the Java [Connection](http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html) documentation.Usage: `beeline --isolation=TRANSACTION_SERIALIZABLE` | +| `--nullemptystring=[true/false]` | Use historic behavior of printing null as empty string (true) or use current behavior of printing null as NULL (false). Default is false.Usage: `beeline --nullemptystring=false`Version: 0.13.0 ([HIVE-4485](https://issues.apache.org/jira/browse/HIVE-4485)) | +| `--incremental=[true/false]` | Defaults to `true` from Hive 2.3 onwards, before it defaulted to `false``.` When set to `false`, the entire result set is fetched and buffered before being displayed, yielding optimal display column sizing. When set to `true`, result rows are displayed immediately as they are fetched, yielding lower latency and memory usage at the price of extra display column padding. Setting `--incremental=true` is recommended if you encounter an OutOfMemory on the client side (due to the fetched result set size being large). | +| `--incrementalBufferRows=NUMROWS` | The number of rows to buffer when printing rows on stdout, defaults to 1000; only applicable if `--incremental=true` and `--outputformat=table`Usage: `beeline --incrementalBufferRows=1000`Version: 2.3.0 ([HIVE-14170](https://issues.apache.org/jira/browse/HIVE-14170)) | +| `--maxHistoryRows=NUMROWS` | The maximum number of rows to store Beeline history.Version: 2.3.0 ([HIVE-15166](https://issues.apache.org/jira/browse/HIVE-15166)) | +| `--delimiter=;` | Set the delimiter for queries written in Beeline. Multi-char delimiters are allowed, but quotation marks, slashes, and -- are not allowed. Defaults to ;Usage: `beeline --delimiter=$$`Version: 3.0.0 ([HIVE-10865](https://issues.apache.org/jira/browse/HIVE-10865)) | +| `--convertBinaryArrayToString=[true/false]` | Display binary column data as a string using the platform's default character set.The default behavior (false) is to display binary data using: `Arrays.toString(byte[] columnValue)`Version: 3.0.0 ([HIVE-14786](https://issues.apache.org/jira/browse/HIVE-14786))Display binary column data as a string using the UTF-8 character set.The default behavior (false) is to display binary data using Base64 encoding without padding.Version: 4.0.0 ([HIVE-2](https://issues.apache.org/jira/browse/HIVE-14786)[3856](https://issues.apache.org/jira/browse/HIVE-23856))Usage: `beeline --convertBinaryArrayToString=true` | +| `--help` | Display a usage message.Usage: `beeline --help` | ### Output Formats @@ -419,7 +419,7 @@ where * `**:**,**:**` is a server instance or a comma separated list of server instances to connect to (if dynamic service discovery is enabled). If empty, the embedded server will be used. * *`dbName`* is the name of the initial database. -* ** is the path of init script file ([Hive 2.2.0](https://issues.apache.org/jira/browse/HIVE-5867) and later). This script file is written with SQL statements which will be executed automatically after connection. This option can be empty. +* *``* is the path of init script file ([Hive 2.2.0](https://issues.apache.org/jira/browse/HIVE-5867) and later). This script file is written with SQL statements which will be executed automatically after connection. This option can be empty. * *`sess_var_list`* is a semicolon separated list of key=value pairs of session variables (e.g., `user=foo;password=bar`). * *`hive_conf_list`* is a semicolon separated list of key=value pairs of Hive configuration variables for this session * *`hive_var_list`* is a semicolon separated list of key=value pairs of Hive variables for this session. @@ -431,7 +431,7 @@ Special characters in *`sess_var_list, hive_conf_list, hive_var_list`*paramete The JDBC connection URL format has the prefix `jdbc:hive2://` and the Driver class is `org.apache.hive.jdbc.HiveDriver`. Note that this is different from the old [HiveServer]({{< ref "hiveserver" >}}). * For a remote server, the URL format is `jdbc:hive2://:/;initFile=` (default port for HiveServer2 is 10000). -* For an embedded server, the URL format is `jdbc:hive2:///;`initFile=``(no host or port). +* For an embedded server, the URL format is `jdbc:hive2:///;initFile=`(no host or port). The `initFile` option is available in [Hive 2.2.0](https://issues.apache.org/jira/browse/HIVE-5867) and later releases. @@ -450,8 +450,8 @@ In versions earlier than [0.14](https://issues.apache.org/jira/browse/HIVE-6972) JDBC connection URL:  `jdbc:hive2://:/;ssl=true;sslTrustStore=;trustStorePassword=`, where: -*  is the path where client's truststore file lives. -* is the password to access the truststore. +* `` is the path where client's truststore file lives. +* `` is the password to access the truststore. In HTTP mode:  `jdbc:hive2://:/;ssl=true;sslTrustStore=;trustStorePassword=;transportMode=http;httpPath=`. @@ -459,7 +459,7 @@ For versions earlier than 0.14, see the [version note]({{< ref "#version-note" ### Connection URL When ZooKeeper Service Discovery Is Enabled -ZooKeeper-based service discovery introduced in Hive 0.14.0 ([HIVE-7935](https://issues.apache.org/jira/browse/HIVE-7395)) enables high availability and rolling upgrade for HiveServer2. A JDBC URL that specifies needs to be used to make use of these features. That is, at least in `hive-site.xml` or other configuration files for HiveServer2, `hive.server2.support.dynamic.service.discovery` should be set to `true`, and `hive.zookeeper.quorum` should be defined to point to several started Zookeeper Servers. Reference [Configuration Properties]({{< ref "configuration-properties" >}}) . +ZooKeeper-based service discovery introduced in Hive 0.14.0 ([HIVE-7935](https://issues.apache.org/jira/browse/HIVE-7395)) enables high availability and rolling upgrade for HiveServer2. A JDBC URL that specifies `` needs to be used to make use of these features. That is, at least in `hive-site.xml` or other configuration files for HiveServer2, `hive.server2.support.dynamic.service.discovery` should be set to `true`, and `hive.zookeeper.quorum` should be defined to point to several started Zookeeper Servers. Reference [Configuration Properties]({{< ref "configuration-properties" >}}) . The minimal configuration example is as follows. @@ -483,15 +483,15 @@ With further changes in Hive 2.0.0 and 1.3.0 (unreleased, [HIVE-11581](https://i The JDBC connection URL: `jdbc:hive2:///;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2` . -The is the same as the value of hive.zookeeper.quorum configuration parameter in hive-site.xml/hivserver2-site.xml used by HiveServer2. +The `` is the same as the value of hive.zookeeper.quorum configuration parameter in hive-site.xml/hivserver2-site.xml used by HiveServer2. -Additional runtime parameters needed for querying can be provided within the URL as follows, by appending it as a ?