From ca811a80bf2661a41aad72480ef5cf572500c5ab Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 13:59:35 +0300
Subject: [PATCH 01/22] Created recommended_configurations_updated file

---
 guides/operations/setup/recommended_configurations_updated | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 guides/operations/setup/recommended_configurations_updated

diff --git a/guides/operations/setup/recommended_configurations_updated b/guides/operations/setup/recommended_configurations_updated
new file mode 100644
index 000000000..8b1378917
--- /dev/null
+++ b/guides/operations/setup/recommended_configurations_updated
@@ -0,0 +1 @@
+

From 4a47b969448fd04ac130c6be605219186d374479 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 14:04:43 +0300
Subject: [PATCH 02/22] Update recommended_configurations_updated

---
 .../setup/recommended_configurations_updated  | 1354 +++++++++++++++++
 1 file changed, 1354 insertions(+)

diff --git a/guides/operations/setup/recommended_configurations_updated b/guides/operations/setup/recommended_configurations_updated
index 8b1378917..508514a82 100644
--- a/guides/operations/setup/recommended_configurations_updated
+++ b/guides/operations/setup/recommended_configurations_updated
@@ -1 +1,1355 @@
+.. _recommended_configuration:
+
+*********************************************
+Recommended Post-Installation Configuration
+*********************************************
+
+Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.
+
+This page provides recommendations for production deployments of SQream DB.
+
+.. contents:: In this topic:
+   :local:
+
+# Recommended BIOS Settings
+
+
+The BIOS settings may have a variety of names, or may not exist on your system. Each system vendor has a different set of settings and variables. 
+
+It is safe to skip any and all of the configuration steps, but this may impact performance.
+
+If any doubt arises, consult the documentation for your server or your hardware vendor for the correct way to apply the settings.
+
+.. list-table::
+   :widths: 25 25 50
+   :header-rows: 1
+   
+   * - Item
+     - Setting
+     - Rationale
+   * - **Management console access - ensure the physical connection?**
+     - **Connected**
+     - Connection to OOB required to preserve continuous network uptime.
+   * - **All drives.**
+     - **Connected and displayed on RAID interface**
+     - Prerequisite for cluster or OS installation.
+   * - **RAID volumes.**
+     - **Configured according to project guidelines. Must be rebooted to take effect.**
+     - Clustered to increase logical volume and provide redundancy.
+   * - **Fan speed Thermal Configuration.**
+     - Dell fan speed: **High Maximum**. Specified minimum setting: **60**. HPe thermal configuration: **Increased cooling**.
+     - NVIDIA Tesla GPUs are passively cooled and require high airflow to operate at full performance.
+   * - Power regulator or iDRAC power unit policy   
+     - HPe: **HP static high performance** mode enabled. Dell: **iDRAC power unit policy** (power cap policy) disabled.
+     - Other power profiles (such as "balanced") throttle the CPU and diminishes performance. Throttling may also cause GPU failure.   
+   * - **System Profile**, **Power Profile**, or **Performance Profile**
+     - **High Performance**
+     - The Performance profile provides potentially increased performance by maximizing processor frequency, and the disabling certain power saving features such as C-states. Use this setting for environments that are not sensitive to power consumption.
+   * - **Power Cap Policy** or **Dynamic power capping**
+     - **Disabled**
+     - Other power profiles (like "balanced") throttle the CPU and may diminish performance or cause GPU failure. This setting may appear together with the above (Power profile or Power regulator). **(In BIOS?)** This setting allows disabling system ROM power calibration during the boot process. **Note:** Power regulator settings are named differently in BIOS and iLO/iDRAC. **Comment: is it necessary to show the different power regulator setting names in this document?**	 
+   * - **Intel Turbo Boost**
+     - **Enabled**
+     - Intel Turbo Boost enables overclocking the processor to boost CPU-bound operation performance. Overclocking may risk computational jitter due to changes in the processor's turbo frequency. This causes brief pauses in processor operation, introducing uncertainty into application processing time. Turbo operation is a function of power consumption, processor temperature, and the number of active cores.
+   * - **Logical Processor**
+     - **HPe**: Enable **Hyperthreading** **Dell**: Enable **Logical Processor**
+     - Hyperthreading doubles the amount of logical processors, which may improve performance by ~5-10% for CPU-bound operations.	 	 
+   * - **Intel Virtualization Technology** (VT-d)
+     - **Disable**
+     - VT-d is optimal for running VMs. However, when running Linux natively, disabling VT-d boosts performance by up to 10%.	 
+   * - **Logical Processor**
+     - **HPe**: Enable **Hyperthreading** **Dell**: Enable **Logical Processor**
+     - Hyperthreading doubles the amount of logical processors, which may improve performance by ~5-10% for CPU-bound operations.	 	 
+   * - **Intel Virtualization Technology** (VT-d)
+     - **Disable**
+     - VT-d is optimal for running VMs. However, when running Linux natively, disabling VT-d boosts performance by up to 10%.	  
+   * - **Processor C-States** (Minimum processor idle power core state)
+     - **Disable** 
+     - Processor C-States reduce server power when the system is in an idle state. This causes slower cold-starts when the system transitions from an idle to a load state, and may reduce query performance by up to 15%. **Comment: a hyperlinked footnote to an internal source was inserted into the Confluence doc here. Do we want to include this in the final version? Linked URL: https://www.dell.com/support/kbdoc/en-il/000060621/what-is-the-c-state**	 	 
+   * - **HPe**: **Energy/Performance bias**
+     - **Maximum performance**
+     - Configures processor sub-systems for high-performance and low-latency. Other power profiles (like "balanced") throttle the CPU and may diminish performance. Use this setting for environments that are not sensitive to power consumption.		 
+   * - **HPe**: **DIMM voltage**
+     - **Optimized for Performance**
+     - Setting a higher voltage for DIMMs may increase performance.		 
+   * - **Memory Operating Mode**
+     - **Optimizer Mode**, **Disable Node Interleaving**, **Auto Memory Operating Voltage**
+     - Memory Operating Mode is tuned for performance in **Optimizer** mode. Other modes may improve reliability, but reduce performance. **Node Interleaving** should be disabled because enabling it interleaves the memory between memory nodes, which harms NUMA-aware applications such as SQream DB.	 
+   * - **HPe**: **Memory power savings mode**
+     - **Maximum performance**
+     - This setting configures several memory parameters to optimize the performance of memory sub-systems. The default setting is **Balanced**.	 
+   * - HPe **ACPI SLIT**
+     - **Enabled**
+     - ACPI SLIT sets the relative access times between processors and memory and I/O sub-systems. ACPI SLIT enables operating systems to use this data to improve performance by more efficiently allocating resources and workloads.	 
+   * - **QPI Snoop** **Comment: should we write that it is HPe or Intel? HPe: QPI Snoop**
+     - **Cluster on Die** or **Home Snoop**
+     - QPI (QuickPath Interconnect) Snoop lets you configure different Snoop modes that impact the QPI interconnect. Changing this setting may improve the performance of certain workloads. The default setting of **Home Snoop** provides high memory bandwidth in an average NUMA environment. **Cluster on Die** may provide increased memory bandwidth in highly optimized NUMA workloads. **Early Snoop** may decrease memory latency, but may result in lower overall bandwidth compared to other modes. **Comment: Early Snoop is not defined as a Setting. Should it therefore be described here as a note, i.e., Note: Early Snoop may decrease memory...?**	
+	 
+
+	 
+
+
+	 
+Installing the Operating System
+===================================================	 
+Either the CentOS (versions 7.6-7.9) or RHEL (versions 7.6-7.9) must be installed before installing the SQream database. Either the customer or a SQream representative can perform the installation.
+
+**Comment: I recommend leaving contact information here - Please call xxx-xxx-xxxx to contact a SQream representative.**
+
+**To install the operating system:**
+
+#. Select a language (English recommended).
+#. From **Software Selection**, select **Minimal**.
+#. Select the **Development Tools** group checkbox.
+#. Continue the installation. **Comment: why don't we document the entire procedure? I.e., why do we stop here and say "Continue the installation" and "Set up the necessary drives..."?**
+#. Set up the necessary drives and users as per the installation process.
+
+.. note::
+   * Using (**Comment: enabling? I need to see the GUI.**) Debugging Tools is recommended for future problem-solving if necessary.
+
+Selecting the **Development Tools** group installs the following tools:
+
+  * autoconf
+  * automake
+  * binutils
+  * bison
+  * flex
+  * gcc
+  * gcc-c++
+  * gettext
+  * libtool
+  * make
+  * patch
+  * pkgconfig
+  * redhat-rpm-config
+  * rpm-build
+  * rpm-sign
+
+
+
+The root user is created and the OS shell is booted up.  
+
+Configuring the Operating System
+===================================================
+When configuring the operating system, several basic settings related to creating a new server are required. Configuring these as part of your basic set-up increases your server's security and usability. 
+
+Logging In to the Server
+--------------------------------
+You can log in to the server using the server's IP address and password for the **root** user. The server's IP address and **root** user were created while installing the operating system above.
+
+Automatically Creating a SQream User
+--------------------------------
+
+**To automatically create a SQream user:**
+
+#. If a SQream user was created during installation, verify that the same ID is used on every server:
+
+.. code-block:: console
+
+   $ sudo id sqream
+  
+The ID **1000** is used on each server in the following example:
+    
+.. code-block:: console
+
+   $ uid=1000(sqream) gid=1000(sqream) groups=1000(sqream)
+   
+2. If the ID's are different, delete the SQream user and SQream group from both servers:
+
+.. code-block:: console
+
+   $ sudo userdel sqream
+   
+3. Recreate it using the same ID:
+   
+.. code-block:: console
+
+   $ sudo rm /var/spool/mail/sqream
+
+Manually Creating a SQream User
+--------------------------------
+
+**To manually create a SQream user:**
+
+SQream enables you to manually create users. This section shows you how to manually create a user with the UID **1111**.
+
+.. note::
+   * You cannot manually create during the operating system installation procedure.
+   
+1. Add a user with an identical UID on all cluster nodes:
+
+.. code-block:: console
+
+   $ useradd -u 1111 sqream
+   
+2. Add the user **sqream** to the **wheel** group.
+
+.. code-block:: console
+
+   $ sudo usermod -aG wheel sqream
+   
+You can remove the SQream user from the **wheel** group when the installation and configuration are complete:
+
+.. code-block:: console
+
+   $ passwd sqream
+   
+3. Log out and log back in as **sqream**.
+
+.. warning:: If you deleted the **sqream** user and recreated it with different ID, to avoid permission errors, you must change its ownership to /home/sqream.
+
+4. Change the **sqream** user's ownership to /home/sqream:
+
+.. code-block:: console
+
+   $ sudo chown -R sqream:sqream /home/sqream
+   
+Setting Up A Locale
+--------------------------------
+
+SQream enables you to set up a locale. In this example, the locale used is your own location.
+
+**To set up a locale:**   
+
+1. Set the language of the locale:
+
+.. code-block:: console
+
+   $ sudo localectl set-locale LANG=en_US.UTF-8
+
+2. Set the time stamp (time and date) of the locale:
+
+.. code-block:: console
+
+   $ sudo timedatectl set-timezone Asia/Jerusalem
+
+.. note::
+   * If needed, you can run the **timedatectl list-timezones** command to see your current time-zone.
+  
+   
+Installing the Required Packages
+--------------------------------
+You can install the required packages by running the following command:
+
+.. code-block:: console
+
+   $ sudo yum install ntp pciutils monit zlib-devel openssl-devel kernel-devel-$(uname -r) kernel-headers-$(uname -r) gcc net-tools wget jq
+  
+   
+Installing the Recommended Tools
+--------------------------------
+You can install the recommended tools by running the following command:
+
+.. code-block:: console
+
+   $ sudo yum install bash-completion.noarch vim-enhanced vim-common net-tools iotop htop psmisc screen xfsprogs wget yum-utils deltarpm dos2unix   
+   
+
+Installing Python 3.6.7
+--------------------------------
+1. Download the Python 3.6.7 source code tarball file from the following URL into the **/home/sqream** directory:
+
+.. code-block:: console
+
+   $ wget https://www.python.org/ftp/python/3.6.7/Python-3.6.7.tar.xz
+   
+2. Extract the Python 3.6.7 source code into your current directory:
+
+.. code-block:: console
+
+   $ tar -xf Python-3.6.7.tar.xz
+   
+3. Navigate to the Python 3.6.7 directory:
+
+.. code-block:: console
+
+  $ cd Python-3.6.7/Python-3
+  
+4. Run the **./configure** script:
+
+.. code-block:: console
+
+  $ ./configure
+   
+5. Build the software:
+
+.. code-block:: console
+
+  $ make -j30
+  
+6. Install the software:
+
+.. code-block:: console
+
+  $ sudo make install
+  
+7. Verify that Python 3.6.7 has been installed:
+
+.. code-block:: console
+
+  $ python3.6.7
+  
+Installing NodeJS on CentOS 
+--------------------------------
+**To install the node.js on CentOS:**
+
+1. Download the `setup_12.x file <https://rpm.nodesource.com/setup_12.x>`__ as a root user logged in shell:
+
+.. code-block:: console
+
+  $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
+  
+2. Clear the YUM cache and update the local metadata:
+
+.. code-block:: console
+
+  $ sudo yum clean all && sudo yum makecache fast
+  
+3. Install the **node.js**) file:
+
+.. code-block:: console
+
+  $ sudo yum install -y nodejs
+
+Installing NodeJS on Ubuntu
+--------------------------------
+**To install the node.js file on Ubuntu:**
+  
+1. Download the `setup_12.x file <https://deb.nodesource.com/setup_12.x>`__ as a root user logged in shell:
+
+.. code-block:: console
+
+  $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
+  
+2. Install the node.js file:
+
+.. code-block:: console
+
+  $ sudo apt-get install -y nodejs  
+  
+3. Verify the node version: **Comment - is this step relevant only for installing on Ubuntu, or on CentOS as well?**
+  
+.. code-block:: console
+
+  $ node -v 
+  
+Configuring the Network Time Protocol (NTP)
+-------------------------------- 
+This section describes how to configure your NTP.
+
+.. note::
+   * If you don't have internet access, see `Configure NTP Client to Synchronize with NTP Server <https://www.thegeekstuff.com/2014/06/linux-ntp-server-client/>`__.
+
+*Comment - Is this the correct procedure on the linked URL: Configure NTP Client to Synchronize with NTP Server?*
+
+**To configure your NTP:**
+  
+1. Install the NTP file.
+
+.. code-block:: console
+
+  $ sudo yum install ntp
+  
+2. Enable the **ntpd** program.
+
+.. code-block:: console
+
+  $ sudo systemctl enable ntpd
+  
+3. Start the **ntdp** program.
+
+.. code-block:: console
+
+  $ sudo systemctl start ntpd
+  
+4. Print a list of peers known to the server and a summary of their states.   
+  
+.. code-block:: console
+
+  $ sudo ntpq -p
+  
+Configuring the Network Time Protocol Server
+--------------------------------
+If your organization has an NTP server, you can configure it.
+
+**To configure your NTP server:**
+
+1. Output your NTP server address and append ``/etc/ntpd.conf`` to the outuput.
+
+.. code-block:: console
+
+  $ echo -e "\nserver <your NTP server address>\n" | sudo tee -a /etc/ntp.conf
+
+2. Restart the service.
+
+.. code-block:: console
+
+  $ sudo systemctl restart ntpd
+
+3. Check that synchronization is enabled:
+
+.. code-block:: console
+
+  $ sudo timedatectl
+  
+Checking that synchronization is enabled generates the following output:
+
+.. code-block:: console
+
+  $ Local time: Sat 2019-10-12 17:26:13 EDT
+     Universal time: Sat 2019-10-12 21:26:13 UTC
+           RTC time: Sat 2019-10-12 21:26:13
+          Time zone: America/New_York (EDT, -0400)
+        NTP enabled: yes
+   NTP synchronized: yes
+    RTC in local TZ: no
+         DST active: yes
+    Last DST change: DST began at
+                     Sun 2019-03-10 01:59:59 EST
+                     Sun 2019-03-10 03:00:00 EDT
+    Next DST change: DST ends (the clock jumps one hour backwards) at
+                     Sun 2019-11-03 01:59:59 EDT
+                     Sun 2019-11-03 01:00:00 EST
+					 
+					 
+Configuring the Server to Boot Without the UI
+--------------------------------
+You can configure your server to boot without a UI in cases when it is not required (recommended) by running the following command:					 
+
+.. code-block:: console
+
+  $ sudo systemctl set-default multi-user.target	
+
+Running this command activates the **NO-UI** server mode.
+
+Configuring the Security Limits
+--------------------------------
+The security limits refers to the number of open files, processes, etc.
+
+You can configure the security limits by running the **echo -e** command as a root user logged in shell:
+
+.. code-block:: console
+
+  $ sudo bash
+
+.. code-block:: console
+
+  $ echo -e "sqream soft nproc 1000000\nsqream hard nproc 1000000\nsqream soft nofile 1000000\nsqream hard nofile 1000000\nsqream soft core unlimited\nsqream hard core unlimited" >> /etc/security/limits.conf
+  
+Configuring the Kernel Parameters
+--------------------------------
+**To configure the kernel parameters:
+
+1. Insert a new line after each kernel parameter:
+
+.. code-block:: console
+
+  $ echo -e "vm.dirty_background_ratio = 5 \n vm.dirty_ratio = 10 \n vm.swappiness = 10 \n vm.vfs_cache_pressure = 200 \n vm.zone_reclaim_mode = 0 \n" >> /etc/sysctl.conf
+  
+.. warning:: In the past, the **vm.zone_reclaim_mode** parameter was set to **7.** In the latest Sqream version, the vm.zone_reclaim_mode parameter must be set to **0**. If it is not set to **0**, when a numa node runs out of memory, the system will get stuck and will be unable to pull memory from other numa nodes.
+  
+2. Check the maximum value of the **fs.file**. 
+
+.. code-block:: console
+
+  $ sysctl -n fs.file-max
+
+3. *Optional* - If the maximum value of the **fs.file** is smaller than **2097152**, run the following command:
+
+.. code-block:: console
+
+  $ echo "fs.file-max=2097152" >> /etc/sysctl.conf
+
+.. warning:: **IP4 forward** must be enabled for Docker and K8s installation only.
+
+Configuring the Firewall
+--------------------------------
+The example in this section shows the open ports for four sqreamd sessions. If more than four are required, open the required ports as needed.
+
+.. note::
+   * In the example below, Port 8080 is a new UI port.
+
+**To configure the firewall:
+
+1. Start the service and enable FirewallID on boot:
+
+.. code-block:: console
+
+  $ systemctl start firewalld
+  
+2. Add the following ports to the permanent firewall:
+
+.. code-block:: console
+
+  $ firewall-cmd --zone=public --permanent --add-port=8080/tcp
+  $ firewallfirewall-cmd --zone=public --permanent --add-port=3105/tcp
+  $ firewall-cmd --zone=public --permanent --add-port=3108/tcp
+  $ firewall-cmd --zone=public --permanent --add-port=5000-5003/tcp
+  $ firewall-cmd --zone=public --permanent --add-port=5100-5103/tcp
+  $ firewall-cmd --permanent --list-all
+
+**Comment: does *--list-all* add the entire list of ports to the permanent firewall?**
+
+3. Reload the firewall:
+
+.. code-block:: console
+
+  $ firewall-cmd --reload
+
+4. Start the service and enable FirewallID on boot:
+
+.. code-block:: console
+
+  $ systemctl start firewalld  
+
+If you do not need the firewall, you can disable it:
+  
+.. code-block:: console
+
+  $ sudo systemctl disable firewalld
+  
+  
+Disabling selinux
+--------------------------------
+**To disable selinux:**
+
+1. Show the status of **selinux**:
+
+.. code-block:: console
+
+  $ sudo sestatus
+
+2. If the output is not **disabled**, edit the **/etc/selinux/config** file: 
+
+.. code-block:: console
+
+  $ sudo vim /etc/selinux/config
+  
+3. Change **SELINUX=enforcing** to **SELINUX=disabled**.
+  
+The above changes will only take effect after rebooting the server.
+
+You can disable selinux immediately after rebooting the server by running the following command:
+
+.. code-block:: console
+
+  $ sudo setenforce 0
+
+Configuring the /etc/hosts File
+--------------------------------
+**To configure the /etc/hosts file:**
+
+1. Edit the **/etc/hosts** file:
+
+.. code-block:: console
+
+  $ sudo vim /etc/hosts
+
+2. Call your local host:
+
+.. code-block:: console
+
+  $ 127.0.0.1	localhost
+  $ <server ip>	<server_name>
+  
+**Comment - Is the above an output or a step?**
+  
+Configuring the DNS
+--------------------------------
+**To configure the DNS:**
+
+1. Run the **ifconfig** commasnd to check your NIC name. In the following example, **eth0** is the NIC name:
+
+.. code-block:: console
+
+  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 
+
+2. Replace the DNS lines from the example above with your own DNS addresses :
+
+.. code-block:: console
+
+  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-4.4.4.4
+  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-8.8.8.8
+
+**Comment: Is the above input correct?**
+
+Installing the Nvidia CUDA Driver
+===================================================
+
+
+.. warning:: If your UI runs on the server, the server must be stopped before installing the CUDA drivers.
+
+CUDA Driver Prerequisites  
+--------------------------------
+1. Verify that the NVIDIA card has been installed and is detected by the system:
+
+ .. code-block:: console
+
+  $ lspci | grep -i nvidia
+  
+2. Check which version of gcc has been installed:
+
+ .. code-block:: console
+
+  $ gcc --version
+  
+3. If gcc has not been installed, install it for one of the following operating systems:
+
+   * On RHEL/CentOS: 
+
+    .. code-block:: console
+
+          $ sudo yum install -y gcc
+
+   * On Ubuntu: 
+
+    .. code-block:: console
+
+         $ sudo apt-get install gcc
+
+
+Updating the Kernel Headers  
+--------------------------------
+1. Update the kernel headers one of the following operating systems:
+
+   * On RHEL/CentOS:
+
+    .. code-block:: console
+
+          $ sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
+		  
+   * On Ubuntu:
+   
+    .. code-block:: console
+
+          $ sudo apt-get install linux-headers-$(uname -r)
+		  
+2. Install **wget** one of the following operating systems:
+
+   * On RHEL/CentOS:
+   
+    .. code-block:: console
+
+          $ sudo yum install wget
+		  
+   * On Ubuntu:   
+		  
+    .. code-block:: console
+
+          $ sudo apt-get install wget
+		  		  
+Disabling Nouveau  
+--------------------------------
+You can disable Nouveau, which is the default driver.
+
+**To disable Nouveau:**
+
+1. Check if the Nouveau driver has been loaded:
+
+ .. code-block:: console
+
+  $ lsmod | grep nouveau
+
+If the Nouveau driver has been loaded, the command above generates output.
+
+2. Blacklist the Nouveau drivers to disable them:
+
+ .. code-block:: console
+
+  $ cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf blacklist nouveau options nouveau modeset=0 EOF
+  
+ 
+3. Regenerate the kernel **initramfs** directory set:
+
+  1. Modify the **initramfs** directory set:
+  
+   .. code-block:: console
+
+    $ sudo dracut --force
+	
+  2. Reboot the server:
+
+   .. code-block:: console
+
+    $ sudo reboot
+
+Installing the CUDA Driver
+--------------------------------
+This section describes how to install the CUDA driver.  
+  
+.. warning:: The version of the driver installed on the customer's server must be equal or higher than the driver included in the Sqream release package. Contact a Sqream customer service representative to identify the correct version to install.
+
+Installing the CUDA Driver from the Repository
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Installing the CUDA driver from the Repository is the recommended installation method.
+
+**To install the CUDA driver from the Repository:**
+
+1. Install the CUDA dependencies for one of the following operating systems:
+
+   * For RHEL:
+
+     .. code-block:: console
+
+      $ sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+
+   * For CentOS:
+
+     .. code-block:: console
+
+      $ sudo yum install epel-release
+	
+2. Install the CUDA dependencies from the **epel** repository:
+
+ .. code-block:: console
+
+  $ sudo yum install dkms libvdpau
+
+.. note::
+   * Installing the CUDA depedendencies from the **epel** repository is only required for installing **runfile**.
+
+3. Download the required local repository:
+
+ .. code-block:: console
+
+  $ wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
+  
+4. Install the required local repository: 
+   
+ .. code-block:: console
+
+  $ sudo yum localinstall cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
+
+For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
+
+5. Install the CUDA drivers:
+
+  a. Clear the YUM cache:
+  
+   .. code-block:: console
+
+      $ sudo yum clean all
+	  
+  b. Install the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver:
+  
+   .. code-block:: console
+
+      $ sudo yum -y install nvidia-driver-latest-dkms
+
+6. Verify that the installation was successful:
+
+.. code-block:: console
+
+   $ nvidia-smi
+   
+**Comment - what is the output?**
+ 
+.. note::
+   * If you do not have an internet connection, you can set up the local repository as shown in the following example:
+ 
+**Comment: I suggest making the following example its own section, i.e., how to prepare the CUDA driver offline.** 
+
+You can prepare the CUDA driver offline from a server connected to the CUDA repo by running the following commands as a *root* user:
+	  
+1. Query all the packages installed in your system, and verify that cuda-repo has been installed:
+
+   .. code-block:: console
+
+    $ rpm -qa |grep cuda-repo
+
+2. Navigate to the correct repository:
+
+   .. code-block:: console
+
+    $ cd /etc/yum.repos.d/
+
+3. List in long format and print lines matching a pattern for the cuda file:
+
+   .. code-block:: console
+
+    $ ls -l |grep cuda
+
+The following is an example of generated output:
+
+.. code-block:: console
+
+    $ cuda-10-1-local.repo
+
+4. Edit the **/etc/yum.repos.d/cuda-10-1-local.repo** file:
+
+   .. code-block:: console
+
+    $ vim /etc/yum.repos.d/cuda-10-1-local.repo
+
+Check the **name** of the file. **Comment:; for what?**
+
+The following is an example of generated output:
+
+.. code-block:: console
+
+    $ name=cuda-10-1-local
+   
+5. Clone the repository to a location where it can be copied from:
+
+   .. code-block:: console
+
+    $ reposync -g -l -m --repoid=cuda-10-1-local --download_path=/var/cuda-repo-10.1-local
+
+6. Copy the repository to the installation server and create the repository:
+
+   .. code-block:: console
+
+    $ createrepo -g comps.xml /var/cuda-repo-10.1-local
+
+7. Add a repo configuration file in **/etc/yum.repos.d/** by editing the **/etc/yum.repos.d/cuda-10.1-local.repo** repository:
+ 
+   .. code-block:: console
+
+    $ [cuda-10.1-local]
+    $ name=cuda-10.1-local
+    $ baseurl=file:///var/cuda-repo-10.1-local
+    $ enabled=1
+    $ gpgcheck=1
+    $ gpgkey=file:///var/cuda-repo-10-1-local/7fa2af80.pub   
+   
+8. Install the CUDA drivers by installing the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver as a root user logged in shell:
+  
+   .. code-block:: console
+
+    $ sudo yum -y install nvidia-driver-latest-dkms
+	  
+Tuning Up NVIDIA Performance
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+This section describes how to tune up NVIDIA performance.
+
+.. note::
+   * The procedures in this section are relevant to Intel only.
+	
+**To tune up NVIDIA performance:**
+
+1. Change the permissions on the **rc.local** file to **executable**:
+
+   .. code-block:: console
+
+      $ sudo chmod +x /etc/rc.local	  
+	  
+2. Edit the **/etc/yum.repos.d/cuda-10-1-local.repo** file:
+
+   .. code-block:: console
+
+      $ sudo vim /etc/rc.local 
+
+3. Add the following lines:
+
+   a. **For V100**:
+
+      .. code-block:: console
+
+         $ nvidia-persistenced
+
+   b. **For IBM (mandatory)**:
+	  
+      .. code-block:: console
+
+         $ sudo systemctl enable nvidia-persistenced
+		 
+   .. note::
+      * Setting up the NVIDIA POWER9 CUDA driver includes additional set-up requirements. The NVIDIA POWER9 CUDA driver will not function properly if the additional set-up requirements are not followed. See `POWER9 Setup <https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#power9-setup>`__ for the additional set-up requirements.
+
+   
+   b. **For K80**:
+	  
+      .. code-block:: console
+
+         $ nvidia-persistenced
+         $ nvidia-smi -pm 1
+         $ nvidia-smi -acp 0
+         $ nvidia-smi --auto-boost-permission=0
+         $ nvidia-smi --auto-boost-default=0
+
+4. Reboot the server and run the **NVIDIA System Management Interface (NVIDIA SMI)**:
+
+      .. code-block:: console
+
+         $ nvidia-smi
+
+
+Disabling Automatic Bug Reporting Tools
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+**To disable automatic bug reporting tools:**
+
+1. Run the following **abort** commands:
+
+   .. code-block:: console
+
+      $ for i in abrt-ccpp.service abrtd.service abrt-oops.service abrt-pstoreoops.service abrt-vmcore.service abrt-xorg.service ; do sudo systemctl disable $i; sudo systemctl stop $i; done
+
+
+The server is ready for the SQream software installation.
+
+
+2. Run the following checks:
+
+   a. Check the OS release:
+   
+      .. code-block:: console
+	  
+         $ cat /etc/os-release
+	  
+   b. Verify that a SQream user exists and has the same ID on all cluster member services:
+   
+      .. code-block:: console
+	  
+         $ id sqream
+		 
+   c. Verify that the storage is mounted:
+   
+      .. code-block:: console
+	  
+         $ mount
+		 
+   d. Verify that the driver has been installed correctly:
+   
+      .. code-block:: console
+	  
+         $ nvidia-smi
+		 
+   e. Check the maximum value of the **fs.file**: 
+
+      .. code-block:: console
+
+         $ sysctl -n fs.file-max
+		 
+The desired output when checking the maximum value of the **fs.file** is greater or equal to **2097152**.
+
+   f. Run the following command as a SQream user: **Question - what do the following command parameter do? -c?**		 
+		 
+      .. code-block:: console
+
+         $ ulimit -c -u -n	
+
+**Comment: See https://ss64.com/bash/ulimit.html**		 
+		 
+The following shows the desired output when **<fill in>**:
+
+.. code-block:: console
+
+         $ core file size (blocks, -c) unlimited
+         $ max user processes (-u) 1000000
+         $ open files (-n) 1000000
+
+3. Configure the security limits by running the **echo -e** command as a root user logged in shell:
+
+   .. code-block:: console
+
+      $ sudo bash
+      $ echo -e "sqream soft nproc 1000000\nsqream hard nproc 1000000\nsqream soft nofile 1000000\nsqream hard nofile 1000000\nsqream soft core unlimited\nsqream hard core unlimited" >> /etc/security/limits.conf
+	  
+Enabling Core Dumps
+===================================================
+
+Enabling core dumps is recommended, but optional.
+
+**To enable core dumps:**
+
+1. Check the abrtd Status
+
+2. Set the limits
+
+3. Create the core dumps directory.
+
+
+Checking the abrtd Status
+---------------------------------------------------
+
+**To check the abrtd status:**
+
+1. Check if **abrtd** is running:
+
+   .. code-block:: console
+
+      $ sudo ps -ef |grep abrt
+
+2. If **abrtd** is running, stop it:	  
+	  
+   .. code-block:: console
+
+      $ sudo service abrtd stop
+      $ sudo chkconfig abrt-ccpp off
+      $ sudo chkconfig abrt-oops off
+      $ sudo chkconfig abrt-vmcore off
+      $ sudo chkconfig abrt-xorg off
+      $ sudo chkconfig abrtd off
+
+
+Setting the Limits
+---------------------------------------------------	
+
+**To set the limits:**  
+	  
+1. Set the limits:
+
+   .. code-block:: console
+
+      $ ulimit -c
+	  
+2. If the output is **0**, add the following lines to the **limits.conf** file (/etc/security):	  
+	  
+   .. code-block:: console
+
+      $ *          soft     core           unlimited
+      $ *          hard     core           unlimited	  
+	  
+3. Log out and log in to apply the limit changes.	
+
+Creating the Core Dumps Directory
+---------------------------------------------------	
+
+**To set the core dumps directory:** 
+
+1. Make the **/tmp/core_dumps** directory:
+
+   .. code-block:: console
+
+      $ mkdir /tmp/core_dumps
+
+2. Set the ownership of the **/tmp/core_dumps** directory:
+
+   .. code-block:: console
+
+      $ sudo chown sqream.sqream /tmp/core_dumps
+  
+3. Grant read, write, and execute permissions to all users:
+
+   .. code-block:: console
+
+      $ sudo chmod -R 777 /tmp/core_dumps	
+
+Setting the Output Directory of the /etc/sysctl.conf File 
+---------------------------------------------------	
+
+**To set the output directory of the /etc/sysctl.conf file:** 
+
+1. Edit the **/etc/sysctl.conf** file:
+
+   .. code-block:: console
+
+      $ sudo vim /etc/sysctl.conf
+
+2. Add the following to the bottom of the file:
+
+   .. code-block:: console
+
+      $ kernel.core_uses_pid = 1
+      $ kernel.core_pattern = /<tmp/core_dumps>/core-%e-%s-%u-%g-%p-%t
+      $ fs.suid_dumpable = 2
+
+**Comment - leave a note that the user can choose his correct location of the folder.**
+
+3. To apply the changes without rebooting the server, run:
+	  
+  .. code-block:: console
+
+      $ sudo sysctl -p
+
+4. Check that the core output directory points to the following:
+
+  .. code-block:: console
+
+      $ sudo cat /proc/sys/kernel/core_pattern
+	  
+The following shows the correct generated output:
+	  
+.. code-block:: console
+
+   $ /tmp/core_dumps/core-%e-%s-%u-%g-%p-%t	  
+	  
+5. Verify that the core dumping works:	  
+	  
+  .. code-block:: console
+
+      $ select abort_server();
+
+	  
+Verifying that the Core Dumps Work 
+---------------------------------------------------	
+**To verify that the core dumps work:**
+
+1. Stop and restart all SQream services.
+
+2. Connect to SQream with ClientCmd and run the following command:
+
+  .. code-block:: console
+
+      $ select abort_server();
+
+.. note::
+   * You can verify that the core dumps work only after installing and running SQream.
+   
+This causes the server to crash and a new core.xxx file to be included in the folder that is written in **/etc/sysctl.conf**
+
+**Question: what does "the folder that is written..." mean.**
+**Question: what did the author mean by "Stage 3"?**
+   
+Troubleshooting Core Dumping 
+---------------------------------------------------	
+This section describes the troubleshooting procedure to be followed if all parameters have been configured correctly, but the cores have not been created.
+
+**To troubleshoot core dumping:**
+
+1. Reboot the server.
+
+2. Verify that you have folder permissions:
+
+   .. code-block:: console
+
+      $ sudo chmod -R 777 /tmp/core_dumps   
+   
+3. Verify that the limits have been set correctly:
+
+   .. code-block:: console
+
+      $ ulimit -c
+
+If all parameters have been configured correctly, the correct output is:
+
+.. code-block:: console
+
+      $ unlimited	
+
+4. If all parameters have been configured correctly, but running **ulimit -c** outputs **0**, run the following:
+
+   .. code-block:: console
+
+      $ sudo vim /etc/profile
+
+5. Search for line and tag it with the **hash** symbol: **Search for which line?**
+
+   .. code-block:: console
+
+      $ ulimit -S -c 0 > /dev/null 2>&1	 
+	  
+
+6. If the line is not found in **/etc/profile** directory, do the following:	  
+	  
+   a. Run the following command:
+
+      .. code-block:: console
+
+         $ sudo vim /etc/init.d/functions
+
+   b. Search for the following:
+   
+      .. code-block:: console
+
+         $ ulimit -S -c ${DAEMON_COREFILE_LIMIT:-0} >/dev/null 2>&1
+
+   c. If the line is found, tag it with the **hash** symbol and reboot the server.
+
+**Comment: The rest of the document is the old content. Can it all be removed?**   
+	  
+	  
+	  
+	 
+Use a dedicated SQream DB administration account
+===================================================
+
+Create a user for SQream DB, and optionally assign it to the ``wheel`` group for ``sudo`` access.
+
+.. code-block:: console
+
+   $ useradd -m -u 1132 -U sqream
+   $ passwd sqream 
+   $ usermod -aG wheel sqream 
+
+.. note::
+   * The UID (1132 in the example above) is set to ensure all shared files are accessible by all workers.
+
+Configure the OS locale and timezone
+=====================================
+
+#. Set your OS to use UTF-8, which SQream DB uses for non-English language support.
+   
+   .. code-block:: console
+   
+      $ sudo localectl set-locale LANG=en_US.UTF-8
+
+#. Set the correct timezone for your server.
+   Refer to `the list of available timezones <https://en.wikipedia.org/wiki/List_of_tz_database_time_zones>`_ to find a timezone that matches your location.
+   
+   .. code-block:: console
+   
+      $ sudo timedatectl set-timezone America/New_York
+
+
+Configure NTP for clock synchronization
+=========================================
+
+SQream DB clusters rely on clock synchronization to function correctly.
+
+
+   .. code-block:: console
+   
+      $ sudo yum install -y ntp ntpdate
+      $ sudo systemctl enable ntpd
+      $ sudo systemctl start ntpd
+
+If your organization has an NTP server, configure it by adding records to ``/etc/ntpd.conf``, reloading the service, and checking that synchronization is enabled:
+
+   .. code-block:: console
+   
+      $ echo -e "\nserver <your NTP server address>\n" | sudo tee -a /etc/ntp.conf
+      $ sudo systemctl restart ntpd
+      $ sudo timedatectl
+               Local time: Sat 2019-10-12 17:26:13 EDT
+           Universal time: Sat 2019-10-12 21:26:13 UTC
+                 RTC time: Sat 2019-10-12 21:26:13
+                Time zone: America/New_York (EDT, -0400)
+              NTP enabled: yes
+         NTP synchronized: yes
+          RTC in local TZ: no
+               DST active: yes
+          Last DST change: DST began at
+                           Sun 2019-03-10 01:59:59 EST
+                           Sun 2019-03-10 03:00:00 EDT
+          Next DST change: DST ends (the clock jumps one hour backwards) at
+                           Sun 2019-11-03 01:59:59 EDT
+                           Sun 2019-11-03 01:00:00 EST
+
+
+
+Install recommended utilities
+===============================
+
+The following packages contain tools that are recommended but not required for using SQream DB.
+
+   .. code-block:: console
+   
+      $ sudo yum install -y bash-completion.noarch vim-enhanced.x86_64 vim-common.x86_64 net-tools iotop htop psmisc screen xfsprogs wget yum-utils deltarpm dos2unix tuned  pciutils
+
+
+Tuning OS parameters for performance and stability
+===================================================
+
+SQream DB requires certain OS parameters to be set on all hosts in your cluster.
+
+These settings affect:
+
+* Shared memory - Most OS installations may try to limit high throughput software like SQream DB.
+* Network - On high throughput operations like ingest, optimizing network connection parameters can boost performance
+* User limits - SQream DB may open a large amount of files. The default OS settings may cause some statements to fail if the system runs out of file descriptors.
+* Core dump creation rules
+
+#. Create a directory for core dumps
+
+   In this step, you will create a directory for writing core dumps - which you will configure in the next step.
+   
+   .. code-block:: console
+      
+      $ sudo mkdir /tmp/core_dumps
+
+   .. note::
+      Core dumps can be large - up to the size of the system memory (i.e. for a machine with 512GB of RAM, the size of the core dump will be 512GB).
+
+      Make sure the directory has enough space for writing a core dump.
+
+
+#. Set ``sysctl`` overrides to tune system performance
+   
+   .. code-block:: console
+      :linenos:
+      
+      $ sudo tee /etc/sysctl.d/sqreamdb.conf > /dev/null <<EOT
+      kernel.sysrq = 1
+      kernel.core_uses_pid = 1
+      kernel.core_pattern = /tmp/core_dumps/%f-core-%e-%s-%u-%g-%p-%t
+      kernel.pid_max = 524288
+      vm.max_map_count = 2042292
+      vm.dirty_background_ratio = 5
+      vm.dirty_ratio = 3
+      vm.swappiness = 1
+      vm.vfs_cache_pressure = 200
+      vm.zone_reclaim_mode = 0
+      fs.suid_dumpable = 2
+      fs.file-max = 2097152
+      EOT
+
+   .. note:: 
+      The settings above include provisioning for core dumps. Core dumps can be a valuable source of information in some scenarios, where stack traces and error logs are not enough.
+      
+      By default, the kernel writes core dump files in the current working directory of the process. SQream recommends overriding this setting and write the core dump files to a fixed directory.
+      
+      The setting on line 4 uses the directory you created in the previous step (``/tmp/core_dumps``).
+
+
+
+
+#. Increase the limit of open files and processes 
+
+   .. code-block:: console
+      
+         $ sudo tee -a /etc/security/limits.conf > /dev/null <<EOT
+         * soft nproc 524288
+         * hard nproc 524288
+         * soft nofile 524288
+         * hard nofile 524288
+         * soft core unlimited
+         * hard core unlimited
+         EOT
+
+#. Verify mount options for drives
+
+   SQream recommends XFS for local data storage.
+   The recommended XFS mount options are:
+   
+   ``rw,nodev,noatime,nobarrier,inode64``
+
+.. note:: Reboot your system for the above settings to take effect.
+
+Disable SELinux
+=================
+
+SELinux may interfere with NVIDIA driver installation and some SQream DB operations. Unless absolutely necessary, we recommend disabling it.
+
+#. Check if SELinux is enabled
+
+   .. code-block:: console
+      
+      $ sudo sestatus
+      SELinux status:                 disabled
+
+#. You can disable SELinux by changing the value of ``SELINUX`` parameter to ``disabled`` in ``/etc/selinux/config`` and rebooting.
+
+Secure the server with a firewall
+===================================
+
+Opening up ports in the firewall
+---------------------------------
+
+The example below shows how to open up all ports required by SQream DB and related management interfaces. The example also takes into account up to 4 workers on the host.
+
+   .. code-block:: console
+      
+      $ sudo systemctl start firewalld
+      $ sudo systemctl enable firewalld
+      $ for p in {2812,3000,3001,3105,3108,5000-5003,5100-5103}; do sudo firewall-cmd --zone=public --permanent --add-port=${p}/tcp; done
+      $ sudo firewalld --reloadi
+
+
+Disabling the built in firewall
+---------------------------------
+
+If not required, you can disable the server's firewall. This will reduce connectivity issues, but should only be done inside your internal network.
+
+   .. code-block:: console
+      
+      $ sudo systemctl disable firewalld
+      $ sudo systemctl stop firewalld
+
+
+.. rubric:: What's next?
+
+* :ref:`Test your SQream DB installation by creating your first table<first_steps>`
+
+* :ref:`Connect an external tool to SQream DB <third_party_tools>`
 

From 246be6f0c5ad4d69247b4e800c0f35a6d04f6c8b Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 14:06:53 +0300
Subject: [PATCH 03/22] Update recommended_configurations_updated

---
 guides/operations/setup/recommended_configurations_updated | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated b/guides/operations/setup/recommended_configurations_updated
index 508514a82..c8bb41bf2 100644
--- a/guides/operations/setup/recommended_configurations_updated
+++ b/guides/operations/setup/recommended_configurations_updated
@@ -1,7 +1,7 @@
 .. _recommended_configuration:
 
 *********************************************
-Recommended Post-Installation Configuration
+Recommended Post-Installation Configurations
 *********************************************
 
 Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.

From 2d27247735c0ec9cf189b672508fabbea4343270 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 14:16:32 +0300
Subject: [PATCH 04/22] Update and rename recommended_configurations_updated to
 recommended_configurations_updated.rst

---
 ...gurations_updated => recommended_configurations_updated.rst} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename guides/operations/setup/{recommended_configurations_updated => recommended_configurations_updated.rst} (99%)

diff --git a/guides/operations/setup/recommended_configurations_updated b/guides/operations/setup/recommended_configurations_updated.rst
similarity index 99%
rename from guides/operations/setup/recommended_configurations_updated
rename to guides/operations/setup/recommended_configurations_updated.rst
index c8bb41bf2..508514a82 100644
--- a/guides/operations/setup/recommended_configurations_updated
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -1,7 +1,7 @@
 .. _recommended_configuration:
 
 *********************************************
-Recommended Post-Installation Configurations
+Recommended Post-Installation Configuration
 *********************************************
 
 Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.

From 51cbe009fdc71047ea8da70fa25301790494824b Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 15:11:00 +0300
Subject: [PATCH 05/22] Updated recommended_configurations_updated.rst

Made formatting changes.
---
 .../recommended_configurations_updated.rst    | 57 +++++++------------
 1 file changed, 19 insertions(+), 38 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 508514a82..349380706 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -1,4 +1,4 @@
-.. _recommended_configuration:
+.. _recommended_configurations_updated:
 
 *********************************************
 Recommended Post-Installation Configuration
@@ -9,11 +9,10 @@ Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and
 This page provides recommendations for production deployments of SQream DB.
 
 .. contents:: In this topic:
-   :local:
-
-# Recommended BIOS Settings
-
+   :local:   
 
+Recommended BIOS Settings
+==========================
 The BIOS settings may have a variety of names, or may not exist on your system. Each system vendor has a different set of settings and variables. 
 
 It is safe to skip any and all of the configuration steps, but this may impact performance.
@@ -47,7 +46,7 @@ If any doubt arises, consult the documentation for your server or your hardware
      - The Performance profile provides potentially increased performance by maximizing processor frequency, and the disabling certain power saving features such as C-states. Use this setting for environments that are not sensitive to power consumption.
    * - **Power Cap Policy** or **Dynamic power capping**
      - **Disabled**
-     - Other power profiles (like "balanced") throttle the CPU and may diminish performance or cause GPU failure. This setting may appear together with the above (Power profile or Power regulator). **(In BIOS?)** This setting allows disabling system ROM power calibration during the boot process. **Note:** Power regulator settings are named differently in BIOS and iLO/iDRAC. **Comment: is it necessary to show the different power regulator setting names in this document?**	 
+     - Other power profiles (like "balanced") throttle the CPU and may diminish performance or cause GPU failure. This setting may appear together with the above (Power profile or Power regulator). **(In BIOS?)** This setting allows disabling system ROM power calibration during the boot process. Power regulator settings are named differently in BIOS and iLO/iDRAC. **Comment: is it necessary to show the different power regulator setting names in this document?**	 
    * - **Intel Turbo Boost**
      - **Enabled**
      - Intel Turbo Boost enables overclocking the processor to boost CPU-bound operation performance. Overclocking may risk computational jitter due to changes in the processor's turbo frequency. This causes brief pauses in processor operation, introducing uncertainty into application processing time. Turbo operation is a function of power consumption, processor temperature, and the number of active cores.
@@ -83,7 +82,7 @@ If any doubt arises, consult the documentation for your server or your hardware
      - ACPI SLIT sets the relative access times between processors and memory and I/O sub-systems. ACPI SLIT enables operating systems to use this data to improve performance by more efficiently allocating resources and workloads.	 
    * - **QPI Snoop** **Comment: should we write that it is HPe or Intel? HPe: QPI Snoop**
      - **Cluster on Die** or **Home Snoop**
-     - QPI (QuickPath Interconnect) Snoop lets you configure different Snoop modes that impact the QPI interconnect. Changing this setting may improve the performance of certain workloads. The default setting of **Home Snoop** provides high memory bandwidth in an average NUMA environment. **Cluster on Die** may provide increased memory bandwidth in highly optimized NUMA workloads. **Early Snoop** may decrease memory latency, but may result in lower overall bandwidth compared to other modes. **Comment: Early Snoop is not defined as a Setting. Should it therefore be described here as a note, i.e., Note: Early Snoop may decrease memory...?**	
+     - QPI (QuickPath Interconnect) Snoop lets you configure different Snoop modes that impact the QPI interconnect. Changing this setting may improve the performance of certain workloads. The default setting of **Home Snoop** provides high memory bandwidth in an average NUMA environment. **Cluster on Die** may provide increased memory bandwidth in highly optimized NUMA workloads. **Early Snoop** may decrease memory latency, but may result in lower overall bandwidth compared to other modes.	
 	 
 
 	 
@@ -104,8 +103,7 @@ Either the CentOS (versions 7.6-7.9) or RHEL (versions 7.6-7.9) must be installe
 #. Continue the installation. **Comment: why don't we document the entire procedure? I.e., why do we stop here and say "Continue the installation" and "Set up the necessary drives..."?**
 #. Set up the necessary drives and users as per the installation process.
 
-.. note::
-   * Using (**Comment: enabling? I need to see the GUI.**) Debugging Tools is recommended for future problem-solving if necessary.
+   Using Debugging Tools is recommended for future problem-solving if necessary.
 
 Selecting the **Development Tools** group installs the following tools:
 
@@ -144,9 +142,9 @@ Automatically Creating a SQream User
 
 #. If a SQream user was created during installation, verify that the same ID is used on every server:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo id sqream
+      $ sudo id sqream
   
 The ID **1000** is used on each server in the following example:
     
@@ -171,10 +169,7 @@ Manually Creating a SQream User
 
 **To manually create a SQream user:**
 
-SQream enables you to manually create users. This section shows you how to manually create a user with the UID **1111**.
-
-.. note::
-   * You cannot manually create during the operating system installation procedure.
+SQream enables you to manually create users. This section shows you how to manually create a user with the UID **1111**. You cannot manually create during the operating system installation procedure.
    
 1. Add a user with an identical UID on all cluster nodes:
 
@@ -223,8 +218,7 @@ SQream enables you to set up a locale. In this example, the locale used is your
 
    $ sudo timedatectl set-timezone Asia/Jerusalem
 
-.. note::
-   * If needed, you can run the **timedatectl list-timezones** command to see your current time-zone.
+If needed, you can run the **timedatectl list-timezones** command to see your current time-zone.
   
    
 Installing the Required Packages
@@ -337,8 +331,7 @@ Configuring the Network Time Protocol (NTP)
 -------------------------------- 
 This section describes how to configure your NTP.
 
-.. note::
-   * If you don't have internet access, see `Configure NTP Client to Synchronize with NTP Server <https://www.thegeekstuff.com/2014/06/linux-ntp-server-client/>`__.
+If you don't have internet access, see `Configure NTP Client to Synchronize with NTP Server <https://www.thegeekstuff.com/2014/06/linux-ntp-server-client/>`__.
 
 *Comment - Is this the correct procedure on the linked URL: Configure NTP Client to Synchronize with NTP Server?*
 
@@ -464,10 +457,7 @@ Configuring the Kernel Parameters
 
 Configuring the Firewall
 --------------------------------
-The example in this section shows the open ports for four sqreamd sessions. If more than four are required, open the required ports as needed.
-
-.. note::
-   * In the example below, Port 8080 is a new UI port.
+The example in this section shows the open ports for four sqreamd sessions. If more than four are required, open the required ports as needed. Port 8080 in the example below is a new UI port.
 
 **To configure the firewall:
 
@@ -705,8 +695,7 @@ Installing the CUDA driver from the Repository is the recommended installation m
 
   $ sudo yum install dkms libvdpau
 
-.. note::
-   * Installing the CUDA depedendencies from the **epel** repository is only required for installing **runfile**.
+Installing the CUDA depedendencies from the **epel** repository is only required for installing **runfile**.
 
 3. Download the required local repository:
 
@@ -744,8 +733,7 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
    
 **Comment - what is the output?**
  
-.. note::
-   * If you do not have an internet connection, you can set up the local repository as shown in the following example:
+If you do not have an internet connection, you can set up the local repository as shown in the following example:
  
 **Comment: I suggest making the following example its own section, i.e., how to prepare the CUDA driver offline.** 
 
@@ -820,10 +808,7 @@ The following is an example of generated output:
 	  
 Tuning Up NVIDIA Performance
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-This section describes how to tune up NVIDIA performance.
-
-.. note::
-   * The procedures in this section are relevant to Intel only.
+This section describes how to tune up NVIDIA performance. The procedures in this section are relevant to Intel only.
 	
 **To tune up NVIDIA performance:**
 
@@ -853,8 +838,7 @@ This section describes how to tune up NVIDIA performance.
 
          $ sudo systemctl enable nvidia-persistenced
 		 
-   .. note::
-      * Setting up the NVIDIA POWER9 CUDA driver includes additional set-up requirements. The NVIDIA POWER9 CUDA driver will not function properly if the additional set-up requirements are not followed. See `POWER9 Setup <https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#power9-setup>`__ for the additional set-up requirements.
+**Notice**: Setting up the NVIDIA POWER9 CUDA driver includes additional set-up requirements. The NVIDIA POWER9 CUDA driver will not function properly if the additional set-up requirements are not followed. See `POWER9 Setup <https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#power9-setup>`__ for the additional set-up requirements.
 
    
    b. **For K80**:
@@ -1083,12 +1067,10 @@ Verifying that the Core Dumps Work
 
       $ select abort_server();
 
-.. note::
-   * You can verify that the core dumps work only after installing and running SQream.
+You can verify that the core dumps work only after installing and running SQream.
    
 This causes the server to crash and a new core.xxx file to be included in the folder that is written in **/etc/sysctl.conf**
 
-**Question: what does "the folder that is written..." mean.**
 **Question: what did the author mean by "Stage 3"?**
    
 Troubleshooting Core Dumping 
@@ -1162,8 +1144,7 @@ Create a user for SQream DB, and optionally assign it to the ``wheel`` group for
    $ passwd sqream 
    $ usermod -aG wheel sqream 
 
-.. note::
-   * The UID (1132 in the example above) is set to ensure all shared files are accessible by all workers.
+The UID (1132 in the example above) is set to ensure all shared files are accessible by all workers.
 
 Configure the OS locale and timezone
 =====================================

From 72e535e81a06bc83cf92f61aaf6a839c25dcfd4c Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 15:28:37 +0300
Subject: [PATCH 06/22] Update recommended_configurations_updated.rst

---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 349380706..91167056f 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -148,7 +148,7 @@ Automatically Creating a SQream User
   
 The ID **1000** is used on each server in the following example:
     
-.. code-block:: console
+   .. code-block:: console
 
    $ uid=1000(sqream) gid=1000(sqream) groups=1000(sqream)
    

From 1092b2261a4913725a763444e2135dc8f12112cd Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 15:35:33 +0300
Subject: [PATCH 07/22] Update recommended_configurations_updated.rst

---
 .../setup/recommended_configurations_updated.rst   | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 91167056f..d452ecdcf 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -150,19 +150,19 @@ The ID **1000** is used on each server in the following example:
     
    .. code-block:: console
 
-   $ uid=1000(sqream) gid=1000(sqream) groups=1000(sqream)
+      $ uid=1000(sqream) gid=1000(sqream) groups=1000(sqream)
    
 2. If the ID's are different, delete the SQream user and SQream group from both servers:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo userdel sqream
+      $ sudo userdel sqream
    
 3. Recreate it using the same ID:
    
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo rm /var/spool/mail/sqream
+      $ sudo rm /var/spool/mail/sqream
 
 Manually Creating a SQream User
 --------------------------------
@@ -173,9 +173,9 @@ SQream enables you to manually create users. This section shows you how to manua
    
 1. Add a user with an identical UID on all cluster nodes:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ useradd -u 1111 sqream
+      $ useradd -u 1111 sqream
    
 2. Add the user **sqream** to the **wheel** group.
 

From a131be6485e521b7df5c2742c41866bd674b9fdd Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 15:50:56 +0300
Subject: [PATCH 08/22] Update recommended_configurations_updated.rst

---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index d452ecdcf..9986745e3 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -4,7 +4,7 @@
 Recommended Post-Installation Configuration
 *********************************************
 
-Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.
+Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability!
 
 This page provides recommendations for production deployments of SQream DB.
 

From f335d5be32f4d6197a0e96f4693823694992ef58 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 15:58:32 +0300
Subject: [PATCH 09/22] Update recommended_configurations_updated.rst

---
 .../operations/setup/recommended_configurations_updated.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 9986745e3..7994084e6 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -4,7 +4,7 @@
 Recommended Post-Installation Configuration
 *********************************************
 
-Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability!
+Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.
 
 This page provides recommendations for production deployments of SQream DB.
 
@@ -179,9 +179,9 @@ SQream enables you to manually create users. This section shows you how to manua
    
 2. Add the user **sqream** to the **wheel** group.
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo usermod -aG wheel sqream
+      $ sudo usermod -aG wheel sqream
    
 You can remove the SQream user from the **wheel** group when the installation and configuration are complete:
 

From ebe23aaaa3decee5de2b14a4cc6fc8a8aa74c328 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:15:23 +0300
Subject: [PATCH 10/22] Made formatting changes

Made changes to instances of code block formatting.
---
 .../recommended_configurations_updated.rst    | 312 +++++++++---------
 1 file changed, 156 insertions(+), 156 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 7994084e6..858ef7435 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -185,19 +185,19 @@ SQream enables you to manually create users. This section shows you how to manua
    
 You can remove the SQream user from the **wheel** group when the installation and configuration are complete:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ passwd sqream
+      $ passwd sqream
    
 3. Log out and log back in as **sqream**.
 
-.. warning:: If you deleted the **sqream** user and recreated it with different ID, to avoid permission errors, you must change its ownership to /home/sqream.
+**Note:** If you deleted the **sqream** user and recreated it with different ID, to avoid permission errors, you must change its ownership to /home/sqream.
 
 4. Change the **sqream** user's ownership to /home/sqream:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo chown -R sqream:sqream /home/sqream
+      $ sudo chown -R sqream:sqream /home/sqream
    
 Setting Up A Locale
 --------------------------------
@@ -208,15 +208,15 @@ SQream enables you to set up a locale. In this example, the locale used is your
 
 1. Set the language of the locale:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo localectl set-locale LANG=en_US.UTF-8
+      $ sudo localectl set-locale LANG=en_US.UTF-8
 
 2. Set the time stamp (time and date) of the locale:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ sudo timedatectl set-timezone Asia/Jerusalem
+      $ sudo timedatectl set-timezone Asia/Jerusalem
 
 If needed, you can run the **timedatectl list-timezones** command to see your current time-zone.
   
@@ -243,45 +243,45 @@ Installing Python 3.6.7
 --------------------------------
 1. Download the Python 3.6.7 source code tarball file from the following URL into the **/home/sqream** directory:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ wget https://www.python.org/ftp/python/3.6.7/Python-3.6.7.tar.xz
+      $ wget https://www.python.org/ftp/python/3.6.7/Python-3.6.7.tar.xz
    
 2. Extract the Python 3.6.7 source code into your current directory:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ tar -xf Python-3.6.7.tar.xz
+      $ tar -xf Python-3.6.7.tar.xz
    
 3. Navigate to the Python 3.6.7 directory:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ cd Python-3.6.7/Python-3
+      $ cd Python-3.6.7/Python-3
   
 4. Run the **./configure** script:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ ./configure
+      $ ./configure
    
 5. Build the software:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ make -j30
+      $ make -j30
   
 6. Install the software:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo make install
+      $ sudo make install
   
 7. Verify that Python 3.6.7 has been installed:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ python3.6.7
+      $ python3.6.7
   
 Installing NodeJS on CentOS 
 --------------------------------
@@ -289,21 +289,21 @@ Installing NodeJS on CentOS
 
 1. Download the `setup_12.x file <https://rpm.nodesource.com/setup_12.x>`__ as a root user logged in shell:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
+      $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
   
 2. Clear the YUM cache and update the local metadata:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo yum clean all && sudo yum makecache fast
+      $ sudo yum clean all && sudo yum makecache fast
   
 3. Install the **node.js**) file:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo yum install -y nodejs
+      $ sudo yum install -y nodejs
 
 Installing NodeJS on Ubuntu
 --------------------------------
@@ -311,21 +311,21 @@ Installing NodeJS on Ubuntu
   
 1. Download the `setup_12.x file <https://deb.nodesource.com/setup_12.x>`__ as a root user logged in shell:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
+      $ curl -sL https://rpm.nodesource.com/setup_12.x | sudo bash -
   
 2. Install the node.js file:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo apt-get install -y nodejs  
+      $ sudo apt-get install -y nodejs  
   
 3. Verify the node version: **Comment - is this step relevant only for installing on Ubuntu, or on CentOS as well?**
   
-.. code-block:: console
+   .. code-block:: console
 
-  $ node -v 
+      $ node -v 
   
 Configuring the Network Time Protocol (NTP)
 -------------------------------- 
@@ -339,27 +339,27 @@ If you don't have internet access, see `Configure NTP Client to Synchronize with
   
 1. Install the NTP file.
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo yum install ntp
+      $ sudo yum install ntp
   
 2. Enable the **ntpd** program.
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo systemctl enable ntpd
+      $ sudo systemctl enable ntpd
   
 3. Start the **ntdp** program.
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo systemctl start ntpd
+      $ sudo systemctl start ntpd
   
 4. Print a list of peers known to the server and a summary of their states.   
   
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo ntpq -p
+      $ sudo ntpq -p
   
 Configuring the Network Time Protocol Server
 --------------------------------
@@ -369,27 +369,27 @@ If your organization has an NTP server, you can configure it.
 
 1. Output your NTP server address and append ``/etc/ntpd.conf`` to the outuput.
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ echo -e "\nserver <your NTP server address>\n" | sudo tee -a /etc/ntp.conf
+      $ echo -e "\nserver <your NTP server address>\n" | sudo tee -a /etc/ntp.conf
 
 2. Restart the service.
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo systemctl restart ntpd
+      $ sudo systemctl restart ntpd
 
 3. Check that synchronization is enabled:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo timedatectl
+      $ sudo timedatectl
   
 Checking that synchronization is enabled generates the following output:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ Local time: Sat 2019-10-12 17:26:13 EDT
+      $ Local time: Sat 2019-10-12 17:26:13 EDT
      Universal time: Sat 2019-10-12 21:26:13 UTC
            RTC time: Sat 2019-10-12 21:26:13
           Time zone: America/New_York (EDT, -0400)
@@ -435,25 +435,25 @@ Configuring the Kernel Parameters
 
 1. Insert a new line after each kernel parameter:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ echo -e "vm.dirty_background_ratio = 5 \n vm.dirty_ratio = 10 \n vm.swappiness = 10 \n vm.vfs_cache_pressure = 200 \n vm.zone_reclaim_mode = 0 \n" >> /etc/sysctl.conf
+      $ echo -e "vm.dirty_background_ratio = 5 \n vm.dirty_ratio = 10 \n vm.swappiness = 10 \n vm.vfs_cache_pressure = 200 \n vm.zone_reclaim_mode = 0 \n" >> /etc/sysctl.conf
   
-.. warning:: In the past, the **vm.zone_reclaim_mode** parameter was set to **7.** In the latest Sqream version, the vm.zone_reclaim_mode parameter must be set to **0**. If it is not set to **0**, when a numa node runs out of memory, the system will get stuck and will be unable to pull memory from other numa nodes.
+**Notice:** In the past, the **vm.zone_reclaim_mode** parameter was set to **7.** In the latest Sqream version, the vm.zone_reclaim_mode parameter must be set to **0**. If it is not set to **0**, when a numa node runs out of memory, the system will get stuck and will be unable to pull memory from other numa nodes.
   
 2. Check the maximum value of the **fs.file**. 
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sysctl -n fs.file-max
+      $ sysctl -n fs.file-max
 
 3. *Optional* - If the maximum value of the **fs.file** is smaller than **2097152**, run the following command:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ echo "fs.file-max=2097152" >> /etc/sysctl.conf
+      $ echo "fs.file-max=2097152" >> /etc/sysctl.conf
 
-.. warning:: **IP4 forward** must be enabled for Docker and K8s installation only.
+**IP4 forward** must be enabled for Docker and K8s installation only.
 
 Configuring the Firewall
 --------------------------------
@@ -463,40 +463,40 @@ The example in this section shows the open ports for four sqreamd sessions. If m
 
 1. Start the service and enable FirewallID on boot:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ systemctl start firewalld
+      $ systemctl start firewalld
   
 2. Add the following ports to the permanent firewall:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ firewall-cmd --zone=public --permanent --add-port=8080/tcp
-  $ firewallfirewall-cmd --zone=public --permanent --add-port=3105/tcp
-  $ firewall-cmd --zone=public --permanent --add-port=3108/tcp
-  $ firewall-cmd --zone=public --permanent --add-port=5000-5003/tcp
-  $ firewall-cmd --zone=public --permanent --add-port=5100-5103/tcp
-  $ firewall-cmd --permanent --list-all
+      $ firewall-cmd --zone=public --permanent --add-port=8080/tcp
+      $ firewallfirewall-cmd --zone=public --permanent --add-port=3105/tcp
+      $ firewall-cmd --zone=public --permanent --add-port=3108/tcp
+      $ firewall-cmd --zone=public --permanent --add-port=5000-5003/tcp
+      $ firewall-cmd --zone=public --permanent --add-port=5100-5103/tcp
+      $ firewall-cmd --permanent --list-all
 
 **Comment: does *--list-all* add the entire list of ports to the permanent firewall?**
 
 3. Reload the firewall:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ firewall-cmd --reload
+      $ firewall-cmd --reload
 
 4. Start the service and enable FirewallID on boot:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ systemctl start firewalld  
+      $ systemctl start firewalld  
 
 If you do not need the firewall, you can disable it:
   
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo systemctl disable firewalld
+      $ sudo systemctl disable firewalld
   
   
 Disabling selinux
@@ -505,15 +505,15 @@ Disabling selinux
 
 1. Show the status of **selinux**:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo sestatus
+      $ sudo sestatus
 
 2. If the output is not **disabled**, edit the **/etc/selinux/config** file: 
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo vim /etc/selinux/config
+      $ sudo vim /etc/selinux/config
   
 3. Change **SELINUX=enforcing** to **SELINUX=disabled**.
   
@@ -531,16 +531,16 @@ Configuring the /etc/hosts File
 
 1. Edit the **/etc/hosts** file:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo vim /etc/hosts
+      $ sudo vim /etc/hosts
 
 2. Call your local host:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ 127.0.0.1	localhost
-  $ <server ip>	<server_name>
+      $ 127.0.0.1	localhost
+      $ <server ip>	<server_name>
   
 **Comment - Is the above an output or a step?**
   
@@ -550,16 +550,16 @@ Configuring the DNS
 
 1. Run the **ifconfig** commasnd to check your NIC name. In the following example, **eth0** is the NIC name:
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 
+      $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 
 
 2. Replace the DNS lines from the example above with your own DNS addresses :
 
-.. code-block:: console
+   .. code-block:: console
 
-  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-4.4.4.4
-  $ sudo vim /etc/sysconfig/network-scripts/ifcfg-8.8.8.8
+      $ sudo vim /etc/sysconfig/network-scripts/ifcfg-4.4.4.4
+      $ sudo vim /etc/sysconfig/network-scripts/ifcfg-8.8.8.8
 
 **Comment: Is the above input correct?**
 
@@ -567,35 +567,35 @@ Installing the Nvidia CUDA Driver
 ===================================================
 
 
-.. warning:: If your UI runs on the server, the server must be stopped before installing the CUDA drivers.
+**Warning:** If your UI runs on the server, the server must be stopped before installing the CUDA drivers.
 
 CUDA Driver Prerequisites  
 --------------------------------
 1. Verify that the NVIDIA card has been installed and is detected by the system:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ lspci | grep -i nvidia
+      $ lspci | grep -i nvidia
   
 2. Check which version of gcc has been installed:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ gcc --version
+      $ gcc --version
   
 3. If gcc has not been installed, install it for one of the following operating systems:
 
    * On RHEL/CentOS: 
 
-    .. code-block:: console
+     .. code-block:: console
 
-          $ sudo yum install -y gcc
+        $ sudo yum install -y gcc
 
    * On Ubuntu: 
 
-    .. code-block:: console
+     .. code-block:: console
 
-         $ sudo apt-get install gcc
+        $ sudo apt-get install gcc
 
 
 Updating the Kernel Headers  
@@ -604,29 +604,29 @@ Updating the Kernel Headers
 
    * On RHEL/CentOS:
 
-    .. code-block:: console
+     .. code-block:: console
 
-          $ sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
+        $ sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
 		  
    * On Ubuntu:
    
-    .. code-block:: console
+     .. code-block:: console
 
-          $ sudo apt-get install linux-headers-$(uname -r)
+        $ sudo apt-get install linux-headers-$(uname -r)
 		  
 2. Install **wget** one of the following operating systems:
 
    * On RHEL/CentOS:
    
-    .. code-block:: console
+     .. code-block:: console
 
-          $ sudo yum install wget
+        $ sudo yum install wget
 		  
    * On Ubuntu:   
 		  
-    .. code-block:: console
+     .. code-block:: console
 
-          $ sudo apt-get install wget
+        $ sudo apt-get install wget
 		  		  
 Disabling Nouveau  
 --------------------------------
@@ -636,38 +636,38 @@ You can disable Nouveau, which is the default driver.
 
 1. Check if the Nouveau driver has been loaded:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ lsmod | grep nouveau
+      $ lsmod | grep nouveau
 
 If the Nouveau driver has been loaded, the command above generates output.
 
 2. Blacklist the Nouveau drivers to disable them:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf blacklist nouveau options nouveau modeset=0 EOF
+      $ cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf blacklist nouveau options nouveau modeset=0 EOF
   
  
 3. Regenerate the kernel **initramfs** directory set:
 
   1. Modify the **initramfs** directory set:
   
-   .. code-block:: console
+     .. code-block:: console
 
-    $ sudo dracut --force
+        $ sudo dracut --force
 	
   2. Reboot the server:
 
-   .. code-block:: console
+     .. code-block:: console
 
-    $ sudo reboot
+        $ sudo reboot
 
 Installing the CUDA Driver
 --------------------------------
 This section describes how to install the CUDA driver.  
   
-.. warning:: The version of the driver installed on the customer's server must be equal or higher than the driver included in the Sqream release package. Contact a Sqream customer service representative to identify the correct version to install.
+**Notice:** The version of the driver installed on the customer's server must be equal or higher than the driver included in the Sqream release package. Contact a Sqream customer service representative to identify the correct version to install.
 
 Installing the CUDA Driver from the Repository
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -681,55 +681,55 @@ Installing the CUDA driver from the Repository is the recommended installation m
 
      .. code-block:: console
 
-      $ sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+        $ sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
 
    * For CentOS:
 
      .. code-block:: console
 
-      $ sudo yum install epel-release
+        $ sudo yum install epel-release
 	
 2. Install the CUDA dependencies from the **epel** repository:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ sudo yum install dkms libvdpau
+      $ sudo yum install dkms libvdpau
 
 Installing the CUDA depedendencies from the **epel** repository is only required for installing **runfile**.
 
 3. Download the required local repository:
 
- .. code-block:: console
+   .. code-block:: console
 
-  $ wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
+      $ wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
   
 4. Install the required local repository: 
    
- .. code-block:: console
+   .. code-block:: console
 
-  $ sudo yum localinstall cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
+      $ sudo yum localinstall cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
 
 For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-  a. Clear the YUM cache:
+   a. Clear the YUM cache:
   
-   .. code-block:: console
+      .. code-block:: console
 
-      $ sudo yum clean all
+         $ sudo yum clean all
 	  
-  b. Install the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver:
+   b. Install the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver:
   
-   .. code-block:: console
+      .. code-block:: console
 
-      $ sudo yum -y install nvidia-driver-latest-dkms
+         $ sudo yum -y install nvidia-driver-latest-dkms
 
 6. Verify that the installation was successful:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ nvidia-smi
+      $ nvidia-smi
    
 **Comment - what is the output?**
  
@@ -743,31 +743,31 @@ You can prepare the CUDA driver offline from a server connected to the CUDA repo
 
    .. code-block:: console
 
-    $ rpm -qa |grep cuda-repo
+      $ rpm -qa |grep cuda-repo
 
 2. Navigate to the correct repository:
 
    .. code-block:: console
 
-    $ cd /etc/yum.repos.d/
+      $ cd /etc/yum.repos.d/
 
 3. List in long format and print lines matching a pattern for the cuda file:
 
    .. code-block:: console
 
-    $ ls -l |grep cuda
+      $ ls -l |grep cuda
 
 The following is an example of generated output:
 
 .. code-block:: console
 
-    $ cuda-10-1-local.repo
+   $ cuda-10-1-local.repo
 
 4. Edit the **/etc/yum.repos.d/cuda-10-1-local.repo** file:
 
    .. code-block:: console
 
-    $ vim /etc/yum.repos.d/cuda-10-1-local.repo
+      $ vim /etc/yum.repos.d/cuda-10-1-local.repo
 
 Check the **name** of the file. **Comment:; for what?**
 
@@ -775,36 +775,36 @@ The following is an example of generated output:
 
 .. code-block:: console
 
-    $ name=cuda-10-1-local
+   $ name=cuda-10-1-local
    
 5. Clone the repository to a location where it can be copied from:
 
    .. code-block:: console
 
-    $ reposync -g -l -m --repoid=cuda-10-1-local --download_path=/var/cuda-repo-10.1-local
+      $ reposync -g -l -m --repoid=cuda-10-1-local --download_path=/var/cuda-repo-10.1-local
 
 6. Copy the repository to the installation server and create the repository:
 
    .. code-block:: console
 
-    $ createrepo -g comps.xml /var/cuda-repo-10.1-local
+      $ createrepo -g comps.xml /var/cuda-repo-10.1-local
 
 7. Add a repo configuration file in **/etc/yum.repos.d/** by editing the **/etc/yum.repos.d/cuda-10.1-local.repo** repository:
  
    .. code-block:: console
 
-    $ [cuda-10.1-local]
-    $ name=cuda-10.1-local
-    $ baseurl=file:///var/cuda-repo-10.1-local
-    $ enabled=1
-    $ gpgcheck=1
-    $ gpgkey=file:///var/cuda-repo-10-1-local/7fa2af80.pub   
+      $ [cuda-10.1-local]
+      $ name=cuda-10.1-local
+      $ baseurl=file:///var/cuda-repo-10.1-local
+      $ enabled=1
+      $ gpgcheck=1
+      $ gpgkey=file:///var/cuda-repo-10-1-local/7fa2af80.pub   
    
 8. Install the CUDA drivers by installing the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver as a root user logged in shell:
   
    .. code-block:: console
 
-    $ sudo yum -y install nvidia-driver-latest-dkms
+      $ sudo yum -y install nvidia-driver-latest-dkms
 	  
 Tuning Up NVIDIA Performance
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -841,7 +841,7 @@ This section describes how to tune up NVIDIA performance. The procedures in this
 **Notice**: Setting up the NVIDIA POWER9 CUDA driver includes additional set-up requirements. The NVIDIA POWER9 CUDA driver will not function properly if the additional set-up requirements are not followed. See `POWER9 Setup <https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#power9-setup>`__ for the additional set-up requirements.
 
    
-   b. **For K80**:
+   c. **For K80**:
 	  
       .. code-block:: console
 
@@ -853,9 +853,9 @@ This section describes how to tune up NVIDIA performance. The procedures in this
 
 4. Reboot the server and run the **NVIDIA System Management Interface (NVIDIA SMI)**:
 
-      .. code-block:: console
+   .. code-block:: console
 
-         $ nvidia-smi
+      $ nvidia-smi
 
 
 Disabling Automatic Bug Reporting Tools
@@ -918,9 +918,9 @@ The following shows the desired output when **<fill in>**:
 
 .. code-block:: console
 
-         $ core file size (blocks, -c) unlimited
-         $ max user processes (-u) 1000000
-         $ open files (-n) 1000000
+   $ core file size (blocks, -c) unlimited
+   $ max user processes (-u) 1000000
+   $ open files (-n) 1000000
 
 3. Configure the security limits by running the **echo -e** command as a root user logged in shell:
 
@@ -1034,13 +1034,13 @@ Setting the Output Directory of the /etc/sysctl.conf File
 	  
   .. code-block:: console
 
-      $ sudo sysctl -p
+     $ sudo sysctl -p
 
 4. Check that the core output directory points to the following:
 
   .. code-block:: console
 
-      $ sudo cat /proc/sys/kernel/core_pattern
+     $ sudo cat /proc/sys/kernel/core_pattern
 	  
 The following shows the correct generated output:
 	  
@@ -1052,7 +1052,7 @@ The following shows the correct generated output:
 	  
   .. code-block:: console
 
-      $ select abort_server();
+     $ select abort_server();
 
 	  
 Verifying that the Core Dumps Work 
@@ -1065,7 +1065,7 @@ Verifying that the Core Dumps Work
 
   .. code-block:: console
 
-      $ select abort_server();
+     $ select abort_server();
 
 You can verify that the core dumps work only after installing and running SQream.
    
@@ -1097,7 +1097,7 @@ If all parameters have been configured correctly, the correct output is:
 
 .. code-block:: console
 
-      $ unlimited	
+   $ unlimited	
 
 4. If all parameters have been configured correctly, but running **ulimit -c** outputs **0**, run the following:
 

From b5b3278e0bfe91e23498ca32dbf1a76673dec6bd Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:18:35 +0300
Subject: [PATCH 11/22] Update recommended_configurations_updated.rst

Fixed code block issue in line 397.
---
 guides/operations/setup/recommended_configurations_updated.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 858ef7435..3b6f38072 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -403,7 +403,8 @@ Checking that synchronization is enabled generates the following output:
     Next DST change: DST ends (the clock jumps one hour backwards) at
                      Sun 2019-11-03 01:59:59 EDT
                      Sun 2019-11-03 01:00:00 EST
-					 
+ 
+ 
 					 
 Configuring the Server to Boot Without the UI
 --------------------------------

From c10622556fe6a1c4e4cd1fac29d7c0fc970830b5 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:19:44 +0300
Subject: [PATCH 12/22] Update recommended_configurations_updated.rst

Code block error in Line 397.
---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 3b6f38072..fbfb5e473 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -394,7 +394,7 @@ Checking that synchronization is enabled generates the following output:
            RTC time: Sat 2019-10-12 21:26:13
           Time zone: America/New_York (EDT, -0400)
         NTP enabled: yes
-   NTP synchronized: yes
+    NTP synchronized: yes
     RTC in local TZ: no
          DST active: yes
     Last DST change: DST began at

From 5db89558a10a0d970ffb1847b694b789a903c5ed Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:23:23 +0300
Subject: [PATCH 13/22] Update recommended_configurations_updated.rst

Fixed minor formatting.
---
 .../setup/recommended_configurations_updated.rst | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index fbfb5e473..bc0cbde9d 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -136,7 +136,7 @@ Logging In to the Server
 You can log in to the server using the server's IP address and password for the **root** user. The server's IP address and **root** user were created while installing the operating system above.
 
 Automatically Creating a SQream User
---------------------------------
+------------------------------------
 
 **To automatically create a SQream user:**
 
@@ -328,7 +328,7 @@ Installing NodeJS on Ubuntu
       $ node -v 
   
 Configuring the Network Time Protocol (NTP)
--------------------------------- 
+------------------------------------------- 
 This section describes how to configure your NTP.
 
 If you don't have internet access, see `Configure NTP Client to Synchronize with NTP Server <https://www.thegeekstuff.com/2014/06/linux-ntp-server-client/>`__.
@@ -362,7 +362,7 @@ If you don't have internet access, see `Configure NTP Client to Synchronize with
       $ sudo ntpq -p
   
 Configuring the Network Time Protocol Server
---------------------------------
+--------------------------------------------
 If your organization has an NTP server, you can configure it.
 
 **To configure your NTP server:**
@@ -407,7 +407,7 @@ Checking that synchronization is enabled generates the following output:
  
 					 
 Configuring the Server to Boot Without the UI
---------------------------------
+---------------------------------------------
 You can configure your server to boot without a UI in cases when it is not required (recommended) by running the following command:					 
 
 .. code-block:: console
@@ -431,7 +431,7 @@ You can configure the security limits by running the **echo -e** command as a ro
   $ echo -e "sqream soft nproc 1000000\nsqream hard nproc 1000000\nsqream soft nofile 1000000\nsqream hard nofile 1000000\nsqream soft core unlimited\nsqream hard core unlimited" >> /etc/security/limits.conf
   
 Configuring the Kernel Parameters
---------------------------------
+---------------------------------
 **To configure the kernel parameters:
 
 1. Insert a new line after each kernel parameter:
@@ -671,7 +671,7 @@ This section describes how to install the CUDA driver.
 **Notice:** The version of the driver installed on the customer's server must be equal or higher than the driver included in the Sqream release package. Contact a Sqream customer service representative to identify the correct version to install.
 
 Installing the CUDA Driver from the Repository
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Installing the CUDA driver from the Repository is the recommended installation method.
 
 **To install the CUDA driver from the Repository:**
@@ -860,7 +860,7 @@ This section describes how to tune up NVIDIA performance. The procedures in this
 
 
 Disabling Automatic Bug Reporting Tools
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 **To disable automatic bug reporting tools:**
 
 1. Run the following **abort** commands:
@@ -1011,7 +1011,7 @@ Creating the Core Dumps Directory
       $ sudo chmod -R 777 /tmp/core_dumps	
 
 Setting the Output Directory of the /etc/sysctl.conf File 
----------------------------------------------------	
+-----------------------------------------------------------------	
 
 **To set the output directory of the /etc/sysctl.conf file:** 
 

From e79177e75c88dfcee99de9e6859700a298d77cf6 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:28:04 +0300
Subject: [PATCH 14/22] Update recommended_configurations_updated.rst

Made minor formatting change.
---
 .../operations/setup/recommended_configurations_updated.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index bc0cbde9d..2394e386a 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -714,11 +714,11 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-   a. Clear the YUM cache:
+a. Clear the YUM cache:
   
-      .. code-block:: console
+   .. code-block:: console
 
-         $ sudo yum clean all
+      $ sudo yum clean all
 	  
    b. Install the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver:
   

From 509a02f98d9ec42f471cd8e63c7675f4255f01b7 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:28:56 +0300
Subject: [PATCH 15/22] Update recommended_configurations_updated.rst

Made minor formatting change.
---
 .../operations/setup/recommended_configurations_updated.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 2394e386a..bc0cbde9d 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -714,11 +714,11 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-a. Clear the YUM cache:
+   a. Clear the YUM cache:
   
-   .. code-block:: console
+      .. code-block:: console
 
-      $ sudo yum clean all
+         $ sudo yum clean all
 	  
    b. Install the most current DKMS (Dynamic Kernel Module Support) NVIDIA driver:
   

From 0c0d3ee3856db4a75d873c698703608c30812565 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:30:37 +0300
Subject: [PATCH 16/22] Update recommended_configurations_updated.rst

Tested numbering style.
---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index bc0cbde9d..1e4fd79c4 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -714,7 +714,7 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-   a. Clear the YUM cache:
+   #. Clear the YUM cache:
   
       .. code-block:: console
 

From 2e0baab6ce926144d08cec8a32217997c3a59724 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:31:14 +0300
Subject: [PATCH 17/22] Update recommended_configurations_updated.rst

Tested numbering style.
---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 1e4fd79c4..2eb2232b1 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -714,7 +714,7 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-   #. Clear the YUM cache:
+   1. Clear the YUM cache:
   
       .. code-block:: console
 

From 6d6f2cb6bd3b3b00209d587ea9d6c2ae8bf9a926 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Mon, 26 Apr 2021 16:33:26 +0300
Subject: [PATCH 18/22] Update recommended_configurations_updated.rst

Tested numbering style. Conclusion: use lowercase letters (a, b, c) for second level lists. In GitHub preview, these appear as lower case Roman Numerals (i, ii, iii). I have not checked how they appear on the front end as this branch has not been merged yet.
---
 guides/operations/setup/recommended_configurations_updated.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations_updated.rst
index 2eb2232b1..bc0cbde9d 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations_updated.rst
@@ -714,7 +714,7 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
 5. Install the CUDA drivers:
 
-   1. Clear the YUM cache:
+   a. Clear the YUM cache:
   
       .. code-block:: console
 

From 2c8663504afa4c839a1b14d1ec47585af2c39342 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Sun, 9 May 2021 13:45:26 +0300
Subject: [PATCH 19/22] Update and rename
 recommended_configurations_updated.rst to recommended_configurations.rst

---
 ...onfigurations_updated.rst => recommended_configurations.rst} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename guides/operations/setup/{recommended_configurations_updated.rst => recommended_configurations.rst} (99%)

diff --git a/guides/operations/setup/recommended_configurations_updated.rst b/guides/operations/setup/recommended_configurations.rst
similarity index 99%
rename from guides/operations/setup/recommended_configurations_updated.rst
rename to guides/operations/setup/recommended_configurations.rst
index bc0cbde9d..49af27332 100644
--- a/guides/operations/setup/recommended_configurations_updated.rst
+++ b/guides/operations/setup/recommended_configurations.rst
@@ -1,4 +1,4 @@
-.. _recommended_configurations_updated:
+.. _recommended_configurations:
 
 *********************************************
 Recommended Post-Installation Configuration

From fd5a361b84ef2abf107805ce9708ebbeb20017c5 Mon Sep 17 00:00:00 2001
From: YanivGerowitz <82020437+YanivGerowitz@users.noreply.github.com>
Date: Sun, 9 May 2021 13:46:59 +0300
Subject: [PATCH 20/22] Update and rename recommended_configurations.rst to
 recommended_pre-installation_configurations.rst

---
 ...ns.rst => recommended_pre-installation_configurations.rst} | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename guides/operations/setup/{recommended_configurations.rst => recommended_pre-installation_configurations.rst} (99%)

diff --git a/guides/operations/setup/recommended_configurations.rst b/guides/operations/setup/recommended_pre-installation_configurations.rst
similarity index 99%
rename from guides/operations/setup/recommended_configurations.rst
rename to guides/operations/setup/recommended_pre-installation_configurations.rst
index 49af27332..e1c85e8ff 100644
--- a/guides/operations/setup/recommended_configurations.rst
+++ b/guides/operations/setup/recommended_pre-installation_configurations.rst
@@ -1,7 +1,7 @@
-.. _recommended_configurations:
+.. _recommended_pre-installation_configurations:
 
 *********************************************
-Recommended Post-Installation Configuration
+Recommended Pre-Installation Configuration
 *********************************************
 
 Once you've :ref:`installed SQream DB<installing_sqream_db_docker>`, you can and should tune your system for better performance and stability.

From f26657d811bbfcda343e38b4f603912c2064f168 Mon Sep 17 00:00:00 2001
From: Yaniv Gerowitz <yanivg@sqreamtech.com>
Date: Mon, 10 May 2021 09:38:56 +0300
Subject: [PATCH 21/22] Replaced "recommended_configuration" with updated
 "recommended"pre-installation_configurations" link

The link **pre-installation_configurations** goes to the updated page based on the Confluence doc.
---
 guides/features/external_data/hdfs_yaniv.rst  | 260 +++++++++++++++++
 guides/operations/setup/index.rst             |   2 +-
 .../setup/installing_sqream_with_binary.rst   | 264 ++++++++++++++++++
 3 files changed, 525 insertions(+), 1 deletion(-)
 create mode 100644 guides/features/external_data/hdfs_yaniv.rst
 create mode 100644 guides/operations/setup/installing_sqream_with_binary.rst

diff --git a/guides/features/external_data/hdfs_yaniv.rst b/guides/features/external_data/hdfs_yaniv.rst
new file mode 100644
index 000000000..4c1a49c81
--- /dev/null
+++ b/guides/features/external_data/hdfs_yaniv.rst
@@ -0,0 +1,260 @@
+.. _hdfs_yaniv.rst:
+
+.. _back_to_top:
+
+Launching SQream in an HDFS Environment
+=======================================
+This page describes how to:
+
+
+
+* :ref:`Configure an HDFS environment for the user sqream <configuring_an_hdfs_environment_for_the_user_sqream>`
+* :ref:`Authenticate Hadoop servers that require Kerberos <authenticate_hadoop_servers_that_require_kerberos>`
+
+.. _configuring_an_hdfs_environment_for_the_user_sqream:
+
+Configuring an HDFS Environment for the User **sqream**
+----------------------------------------------------------
+
+This section describes how to configure an HDFS environment for the user **sqream** and is only relevant for users with an HDFS environment.
+
+**To configure an HDFS environment for the user sqream:**
+
+1. Open your **bash_profile** configuration file for editing:
+
+   .. code-block:: console
+     
+       $ vim /home/sqream/.bash_profile
+       
+2. Make the following edits:
+
+..
+   Comment: - see below; do we want to be a bit more specific on what changes we're talking about?
+
+   .. code-block:: console
+     
+      $ #PATH=$PATH:$HOME/.local/bin:$HOME/bin
+
+      $ #export PATH
+
+      $ # PS1
+      $ #MYIP=$(curl -s -XGET "http://ip-api.com/json" | python -c 'import json,sys; jstr=json.load(sys.stdin); print jstr["query"]')
+      $ #PS1="\[\e[01;32m\]\D{%F %T} \[\e[01;33m\]\u@\[\e[01;36m\]$MYIP \[\e[01;31m\]\w\[\e[37;36m\]\$ \[\e[1;37m\]"
+
+      $ SQREAM_HOME=/usr/local/sqream
+      $ export SQREAM_HOME
+
+      $ export JAVA_HOME=${SQREAM_HOME}/hdfs/jdk
+      $ export HADOOP_INSTALL=${SQREAM_HOME}/hdfs/hadoop
+      $ export CLASSPATH=`${HADOOP_INSTALL}/bin/hadoop classpath --glob`
+      $ export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_INSTALL}/lib/native
+      $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${SQREAM_HOME}/lib:$HADOOP_COMMON_LIB_NATIVE_DIR
+
+
+      $ PATH=$PATH:$HOME/.local/bin:$HOME/bin:${SQREAM_HOME}/bin/:${JAVA_HOME}/bin:$HADOOP_INSTALL/bin
+      $ export PATH
+
+3. Verify that the edits have been made:
+
+   .. code-block:: console
+     
+      source /home/sqream/.bash_profile
+       
+4. Check if you can access Hadoop from your machine:       
+       
+  .. code-block:: console
+     
+     $ hadoop fs -ls hdfs://<hadoop server name or ip>:8020/
+      
+..
+   Comment: - 
+   **NOTICE:** If you cannot access Hadoop from your machine because it uses Kerberos, see `Connecting a SQream Server to Cloudera Hadoop with Kerberos <https://sqream.atlassian.net/wiki/spaces/DOC/pages/822902789/How+to+connect+sqream+server+to+Cloudera+Hadoop+with+kerberos>`_
+
+
+5. Verify that an HDFS environment exists for SQream services:
+
+   .. code-block:: console
+     
+      $ ls -l /etc/sqream/sqream_env.sh
+	  
+.. _step_6:
+
+      
+6. If an HDFS environment does not exist for SQream services, create one (sqream_env.sh):
+   
+   .. code-block:: console
+     
+      $ #!/bin/bash
+
+      $ SQREAM_HOME=/usr/local/sqream
+      $ export SQREAM_HOME
+
+      $ export JAVA_HOME=${SQREAM_HOME}/hdfs/jdk
+      $ export HADOOP_INSTALL=${SQREAM_HOME}/hdfs/hadoop
+      $ export CLASSPATH=`${HADOOP_INSTALL}/bin/hadoop classpath --glob`
+      $ export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_INSTALL}/lib/native
+      $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${SQREAM_HOME}/lib:$HADOOP_COMMON_LIB_NATIVE_DIR
+
+
+      $ PATH=$PATH:$HOME/.local/bin:$HOME/bin:${SQREAM_HOME}/bin/:${JAVA_HOME}/bin:$HADOOP_INSTALL/bin
+      $ export PATH
+	  
+:ref:`Back to top <back_to_top>`
+
+	  
+.. _authenticate_hadoop_servers_that_require_kerberos:
+
+Authenticate Hadoop Servers that Require Kerberos
+---------------------------------------------------
+
+If your Hadoop server requires Kerberos authentication, do the following:
+
+1. Create a principal for the user **sqream**.
+
+   .. code-block:: console
+   
+      $ kadmin -p root/admin@SQ.COM
+      $ addprinc sqream@SQ.COM
+      
+2. If you do not know yor Kerberos root credentials, connect to the Kerberos server as a root user with ssh and run **kadmin.local**:
+
+   .. code-block:: console
+   
+      $ kadmin.local
+      
+   Running **kadmin.local** does not require a password.
+
+3. If a password is not required, change your password to **sqream@SQ.COM**.
+
+   .. code-block:: console
+   
+      $ change_password sqream@SQ.COM
+
+4. Connect to the hadoop name node using ssh:
+
+   .. code-block:: console
+   
+      $ cd /var/run/cloudera-scm-agent/process
+
+5. Check the most recently modified content of the directory above:
+
+   .. code-block:: console
+   
+      $ ls -lrt
+
+5. Look for a recently updated folder containing the text **hdfs**.
+
+The following is an example of the correct folder name:
+
+   .. code-block:: console
+   
+      cd <number>-hdfs-<something>
+	  
+   This folder should contain a file named **hdfs.keytab** or another similar .keytab file.
+   
+
+ 
+..
+   Comment: - Does "something" need to be replaced with "file name"
+   
+
+6. Copy the .keytab file to user **sqream's** Home directory on the remote machines that you are planning to use Hadoop on.
+
+7. Copy the following files to the **sqream sqream@server:<sqream folder>/hdfs/hadoop/etc/hadoop:** directory:
+
+   * core-site.xml
+   * hdfs-site.xml
+
+8. Connect to the sqream server and verify that the .keytab file's owner is a user sqream and is granted the correct permissions:
+
+   .. code-block:: console
+   
+      $ sudo chown sqream:sqream /home/sqream/hdfs.keytab
+      $ sudo chmod 600 /home/sqream/hdfs.keytab
+
+9. Log into the sqream server.
+
+10. Log in as the user **sqream**.
+
+11. Navigate to the Home directory and check the name of a Kerberos principal represented by the following .keytab file:
+
+   .. code-block:: console
+   
+      $ klist -kt hdfs.keytab
+
+   The following is an example of the correct output:
+
+   .. code-block:: console
+   
+      $ sqream@Host-121 ~ $ klist -kt hdfs.keytab
+      $ Keytab name: FILE:hdfs.keytab
+      $ KVNO Timestamp           Principal
+      $ ---- ------------------- ------------------------------------------------------
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 HTTP/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+      $    5 09/15/2020 18:03:05 hdfs/nn1@SQ.COM
+
+12. Verify that the hdfs service named **hdfs/nn1@SQ.COM** is shown in the generated output above.
+
+13. Run the following:
+
+   .. code-block:: console
+   
+      $ kinit -kt hdfs.keytab hdfs/nn1@SQ.COM
+
+ 13. Check the output:
+  
+   .. code-block:: console
+   
+      $ klist
+      
+   The following is an example of the correct output:
+
+   .. code-block:: console
+   
+      $ Ticket cache: FILE:/tmp/krb5cc_1000
+      $ Default principal: sqream@SQ.COM
+      $ 
+      $ Valid starting       Expires              Service principal
+      $ 09/16/2020 13:44:18  09/17/2020 13:44:18  krbtgt/SQ.COM@SQ.COM
+
+14. List the files located at the defined server name or IP address:
+
+   .. code-block:: console
+   
+      $ hadoop fs -ls hdfs://<hadoop server name or ip>:8020/
+
+15. Do one of the following:
+
+    * If the list below is output, continue with Step 16.
+    * If the list is not output, verify that your environment has been set up correctly.
+	
+If any of the following are empty, verify that you followed :ref:`Step 6 <step_6>` in the **Configuring an HDFS Environment for the User sqream** section above correctly:
+
+  .. code-block:: console
+   
+      $ echo $JAVA_HOME
+      $ echo $SQREAM_HOME
+      $ echo $CLASSPATH
+      $ echo $HADOOP_COMMON_LIB_NATIVE_DIR
+      $ echo $LD_LIBRARY_PATH
+      $ echo $PATH
+
+16. Verify that you copied the correct keytab file.
+
+17. Review this procedure to verify that you have followed each step.
+
+:ref:`Back to top <back_to_top>`
diff --git a/guides/operations/setup/index.rst b/guides/operations/setup/index.rst
index b422d9c5b..173c8ae26 100644
--- a/guides/operations/setup/index.rst
+++ b/guides/operations/setup/index.rst
@@ -14,6 +14,6 @@ The guides below cover installing SQream DB.
    
    before_you_begin
    local_docker
-   recommended_configuration
+   recommended_pre-installation_configurations
 
 
diff --git a/guides/operations/setup/installing_sqream_with_binary.rst b/guides/operations/setup/installing_sqream_with_binary.rst
new file mode 100644
index 000000000..1d715f96b
--- /dev/null
+++ b/guides/operations/setup/installing_sqream_with_binary.rst
@@ -0,0 +1,264 @@
+.. _installing_sqream_with_binary:
+
+*********************************************
+Installing SQream with Binary
+*********************************************
+This procedure describes how to install SQream using Binary.
+
+**To install SQream with Binary:**
+
+1. Copy the SQream package to the **/home/sqream** directory for the current version:
+
+   .. code-block:: console
+   
+      $ tar -xf sqream-db-v<2020.2>.tar.gz
+
+2. Append the version number to the name of the SQream folder. The version number in the following example is **v2020.2**:
+
+   .. code-block:: console
+   
+      $ mv sqream sqream-db-v<2020.2>
+
+3. Move the new version of the SQream folder to the **/usr/local/** directory:
+
+   .. code-block:: console
+   
+      $ sudo mv sqream-db-v<2020.2> /usr/local/
+      
+4. Change the ownership of the folder to **sqream folder**:
+
+   .. code-block:: console
+   
+      $ sudo chown -R sqream:sqream  /usr/local/sqream-db-v<2020.2>
+
+5. Navigate to the **/usr/local/** directory and create a symbolic link to SQream:
+
+   .. code-block:: console
+   
+      $ cd /usr/local
+      $ sudo ln -s sqream-db-v<2020.2> sqream
+      
+6. Verify that the symbolic link that you created points to the folder that you created:
+
+   .. code-block:: console
+   
+      $ ls -l
+      
+7. Verify that the symbolic link that you created points to the folder that you created:
+
+   .. code-block:: console
+   
+      $ sqream -> sqream-db-v<2020.2>
+      
+8. Create the SQream configuration file destination folders and set their ownership to **sqream**:
+
+   .. code-block:: console
+   
+      $ sudo mkdir /etc/sqream
+      $ sudo chown -R sqream:sqream /etc/sqream
+      
+9. Create the SQream service log destination folders and set their ownership to **sqream**:
+
+   .. code-block:: console
+   
+      $ sudo mkdir /var/log/sqream
+      $ sudo chown -R sqream:sqream /var/log/sqream
+
+10. Navigate to the **/usr/local/** directory and copy the SQream configuration files from them:
+
+   .. code-block:: console
+   
+      $ cd /usr/local/sqream/etc/
+      $ cp * /etc/sqream
+      
+The configuration files are **service configuration files**, and the JSON files are **SQream configuration files**, for a total of four files. The number of SQream configuration files and JSON files must be identical.
+      
+**NOTICE** - Verify that the JSON files have been configured correctly and that all required flags have been set to the correct values.
+
+In each JSON file, the following parameters **must be updated**:
+
+* instanceId
+* machineIP
+* metadataServerIp
+* spoolMemoryGB
+* limitQueryMemoryGB
+* gpu
+* port
+* ssl_port
+
+Note the following:
+
+* The value of the **metadataServerIp** parameter must point to the IP that the metadata is running on.
+* The value of the **machineIP** parameter must point to the IP of your local machine.
+
+It would be same on server running metadataserver and different on other server nodes.
+
+11. **Optional** - To run additional SQream services, copy the required configuration files and create additional JSON files:
+
+   .. code-block:: console
+   
+      $ cp sqream2_config.json sqream3_config.json
+      $ vim sqream3_config.json
+      
+**NOTICE:** A unique **instanceID** must be used in each JSON file. IN the example above, the instanceID **sqream_2** is changed to **sqream_3**.
+
+12. **Optional** - If you created additional services in **Step 11**, verify that you have also created their additional configuration files:
+
+    .. code-block:: console
+   
+       $ cp sqream2-service.conf sqream3-service.conf
+       $ vim sqream3-service.conf
+      
+13. For each SQream service configuration file, do the following:
+
+    1. Change the **SERVICE_NAME=sqream2** value to **SERVICE_NAME=sqream3**.
+    
+    2. Change **LOGFILE=/var/log/sqream/sqream2.log** to **LOGFILE=/var/log/sqream/sqream3.log**.
+    
+14. Set up **servicepicker**:
+
+    1. Do the following:
+
+       .. code-block:: console
+   
+          $ vim /etc/sqream/server_picker.conf
+    
+    2. Change the IP **127.0.0.1** to the IP of the server that the **metadataserver** service is running on.    
+    
+    3. Change the **CLUSTER** to the value of the cluster path.
+     
+15. Set up your service files:      
+      
+    .. code-block:: console
+   
+       $ cd /usr/local/sqream/service/
+       $ cp sqream2.service sqream3.service
+       $ vim sqream3.service      
+       
+16. Increment each **EnvironmentFile=/etc/sqream/sqream2-service.conf** configuration file for each SQream service file, as shown below:
+
+    .. code-block:: console
+     
+       $ EnvironmentFile=/etc/sqream/sqream<3>-service.conf
+       
+17. Copy and register your service files into systemd:       
+       
+    .. code-block:: console
+     
+       $ sudo cp metadataserver.service /usr/lib/systemd/system/
+       $ sudo cp serverpicker.service /usr/lib/systemd/system/
+       $ sudo cp sqream*.service /usr/lib/systemd/system/
+       
+18. Verify that your service files have been copied into systemd:
+
+    .. code-block:: console
+     
+       $ ls -l /usr/lib/systemd/system/sqream*
+       $ ls -l /usr/lib/systemd/system/metadataserver.service
+       $ ls -l /usr/lib/systemd/system/serverpicker.service
+       $ sudo systemctl daemon-reload       
+       
+19. Copy the license into the **/etc/license** directory:
+
+    .. code-block:: console
+     
+       $ cp license.enc /etc/sqream/   
+
+       
+If you have an HDFS environment, see Configuring an HDFS Environment for the User sqream :ref:`.. _hdfs_yaniv.rst:`.
+
+Upgrading SQream Version
+-------------------------
+Upgrading your SQream version requires stopping all running services while you manually upgrade SQream.
+
+**To upgrade your version of SQream:**
+
+1. Stop all actively running SQream services.
+      
+2. Verify that SQream has stopped listening on ports **500X**, **510X**, and **310X**:
+
+   .. code-block:: console
+
+      $ sudo netstat -nltp    #to make sure sqream stopped listening on 500X, 510X and 310X ports.
+
+3. Replace the old version ``sqream-db-v2020.2``, with the new version ``sqream-db-v2021.1``:
+
+   .. code-block:: console
+    
+      $ cd /home/sqream
+      $ mkdir tempfolder
+      $ mv sqream-db-v2021.1.tar.gz tempfolder/
+      $ tar -xf sqream-db-v2021.1.tar.gz
+      $ sudo mv sqream /usr/local/sqream-db-v2021.1
+      $ cd /usr/local
+      $ sudo chown -R sqream:sqream sqream-db-v2021.1
+   
+4. Remove the symbolic link:
+
+   .. code-block:: console
+   
+      $ sudo rm sqream
+   
+5. Create a new symbolic link named "sqream" pointing to the new version:
+
+   .. code-block:: console  
+
+      $ sudo ln -s sqream-db-v2021.1 sqream
+
+6. Verify that the symbolic SQream link points to the real folder:
+
+   .. code-block:: console  
+
+      $ ls -l
+	 
+   The following is an example of the correct output:
+
+   .. code-block:: console
+    
+      $ sqream -> sqream-db-v2021.1
+
+5. **Optional-** (For major versions) Upgrade your version of SQream storage cluster, as shown in the following example:
+
+   .. code-block:: console  
+
+      $ ./upgrade_storage </home/rhendricks/raviga_database>
+	  
+   The following is an example of the correct output:
+	  
+   .. code-block:: console  
+
+	  get_leveldb_version path{/home/rhendricks/raviga_database}
+	  current storage version 23
+      upgrade_v24
+      upgrade_storage to 24
+	  upgrade_storage to 24 - Done
+	  upgrade_v25
+	  upgrade_storage to 25
+	  upgrade_storage to 25 - Done
+	  upgrade_v26
+	  upgrade_storage to 26
+	  upgrade_storage to 26 - Done
+	  validate_leveldb
+	  ...
+      upgrade_v37
+	  upgrade_storage to 37
+	  upgrade_storage to 37 - Done
+	  validate_leveldb
+      storage has been upgraded successfully to version 37
+ 
+6. Verify that the latest version has been installed:
+
+   .. code-block:: console
+    
+      $ ./sqream sql --username sqream --password sqream --host localhost --databasename master -c "SELECT SHOW_VERSION();"
+      
+   The following is an example of the correct output:
+ 
+   .. code-block:: console
+    
+      v2021.1
+      1 row
+      time: 0.050603s 
+ 
+For more information, see the `upgrade_storage <https://docs.sqream.com/en/latest/reference/cli/upgrade_storage.html>`_ command line program.
+

From e567c28228a5a6f5236c13becb068befd5e9839b Mon Sep 17 00:00:00 2001
From: Yaniv Gerowitz <yanivg@sqreamtech.com>
Date: Mon, 10 May 2021 10:14:56 +0300
Subject: [PATCH 22/22] Commented out all remaining comments

---
 ...mended_pre-installation_configurations.rst | 310 ++++--------------
 1 file changed, 64 insertions(+), 246 deletions(-)

diff --git a/guides/operations/setup/recommended_pre-installation_configurations.rst b/guides/operations/setup/recommended_pre-installation_configurations.rst
index e1c85e8ff..330fbab57 100644
--- a/guides/operations/setup/recommended_pre-installation_configurations.rst
+++ b/guides/operations/setup/recommended_pre-installation_configurations.rst
@@ -46,7 +46,10 @@ If any doubt arises, consult the documentation for your server or your hardware
      - The Performance profile provides potentially increased performance by maximizing processor frequency, and the disabling certain power saving features such as C-states. Use this setting for environments that are not sensitive to power consumption.
    * - **Power Cap Policy** or **Dynamic power capping**
      - **Disabled**
-     - Other power profiles (like "balanced") throttle the CPU and may diminish performance or cause GPU failure. This setting may appear together with the above (Power profile or Power regulator). **(In BIOS?)** This setting allows disabling system ROM power calibration during the boot process. Power regulator settings are named differently in BIOS and iLO/iDRAC. **Comment: is it necessary to show the different power regulator setting names in this document?**	 
+     - Other power profiles (like "balanced") throttle the CPU and may diminish performance or cause GPU failure. This setting may appear together with the above (Power profile or Power regulator). **(In BIOS?)** This setting allows disabling system ROM power calibration during the boot process. Power regulator settings are named differently in BIOS and iLO/iDRAC.
+	 
+..
+	 **Comment: is it necessary to show the different power regulator setting names in this document?**	 
    * - **Intel Turbo Boost**
      - **Enabled**
      - Intel Turbo Boost enables overclocking the processor to boost CPU-bound operation performance. Overclocking may risk computational jitter due to changes in the processor's turbo frequency. This causes brief pauses in processor operation, introducing uncertainty into application processing time. Turbo operation is a function of power consumption, processor temperature, and the number of active cores.
@@ -93,17 +96,21 @@ Installing the Operating System
 ===================================================	 
 Either the CentOS (versions 7.6-7.9) or RHEL (versions 7.6-7.9) must be installed before installing the SQream database. Either the customer or a SQream representative can perform the installation.
 
-**Comment: I recommend leaving contact information here - Please call xxx-xxx-xxxx to contact a SQream representative.**
+..
+	 **Comment: I recommend leaving contact information here - Please call xxx-xxx-xxxx to contact a SQream representative.**
 
 **To install the operating system:**
 
 #. Select a language (English recommended).
 #. From **Software Selection**, select **Minimal**.
 #. Select the **Development Tools** group checkbox.
-#. Continue the installation. **Comment: why don't we document the entire procedure? I.e., why do we stop here and say "Continue the installation" and "Set up the necessary drives..."?**
+#. Continue the installation.
 #. Set up the necessary drives and users as per the installation process.
 
    Using Debugging Tools is recommended for future problem-solving if necessary.
+   
+   ..
+	 **Comment: In Step 4, why don't we document the entire procedure? I.e., why do we stop here and say "Continue the installation" and "Set up the necessary drives..."?**
 
 Selecting the **Development Tools** group installs the following tools:
 
@@ -321,7 +328,10 @@ Installing NodeJS on Ubuntu
 
       $ sudo apt-get install -y nodejs  
   
-3. Verify the node version: **Comment - is this step relevant only for installing on Ubuntu, or on CentOS as well?**
+3. Verify the node version:
+
+..
+	 **Comment: - is this step relevant only for installing on Ubuntu, or on CentOS as well?**
   
    .. code-block:: console
 
@@ -333,7 +343,8 @@ This section describes how to configure your NTP.
 
 If you don't have internet access, see `Configure NTP Client to Synchronize with NTP Server <https://www.thegeekstuff.com/2014/06/linux-ntp-server-client/>`__.
 
-*Comment - Is this the correct procedure on the linked URL: Configure NTP Client to Synchronize with NTP Server?*
+..
+	 **Comment: - Is this the correct procedure on the linked URL: Configure NTP Client to Synchronize with NTP Server?*
 
 **To configure your NTP:**
   
@@ -460,7 +471,7 @@ Configuring the Firewall
 --------------------------------
 The example in this section shows the open ports for four sqreamd sessions. If more than four are required, open the required ports as needed. Port 8080 in the example below is a new UI port.
 
-**To configure the firewall:
+**To configure the firewall:**
 
 1. Start the service and enable FirewallID on boot:
 
@@ -479,7 +490,9 @@ The example in this section shows the open ports for four sqreamd sessions. If m
       $ firewall-cmd --zone=public --permanent --add-port=5100-5103/tcp
       $ firewall-cmd --permanent --list-all
 
-**Comment: does *--list-all* add the entire list of ports to the permanent firewall?**
+
+..
+	 **Comment: - does *--list-all* add the entire list of ports to the permanent firewall?**
 
 3. Reload the firewall:
 
@@ -493,7 +506,7 @@ The example in this section shows the open ports for four sqreamd sessions. If m
 
       $ systemctl start firewalld  
 
-If you do not need the firewall, you can disable it:
+   If you do not need the firewall, you can disable it:
   
    .. code-block:: console
 
@@ -543,7 +556,8 @@ Configuring the /etc/hosts File
       $ 127.0.0.1	localhost
       $ <server ip>	<server_name>
   
-**Comment - Is the above an output or a step?**
+..
+	 **Comment: - Is the above an output or a step?**
   
 Configuring the DNS
 --------------------------------
@@ -562,7 +576,9 @@ Configuring the DNS
       $ sudo vim /etc/sysconfig/network-scripts/ifcfg-4.4.4.4
       $ sudo vim /etc/sysconfig/network-scripts/ifcfg-8.8.8.8
 
-**Comment: Is the above input correct?**
+
+..
+	 **Comment: - is the above input correct?**
 
 Installing the Nvidia CUDA Driver
 ===================================================
@@ -710,7 +726,10 @@ Installing the CUDA depedendencies from the **epel** repository is only required
 
       $ sudo yum localinstall cuda-repo-rhel7-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm
 
-For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
+For example, RHEL7 for cuda 10.1.
+
+..
+	 **Comment: - Confirm.**
 
 5. Install the CUDA drivers:
 
@@ -732,11 +751,14 @@ For example, RHEL7 for cuda 10.1. **Comment: Confirm.**
 
       $ nvidia-smi
    
-**Comment - what is the output?**
+
+..
+	 **Comment: - what is the output?**
  
 If you do not have an internet connection, you can set up the local repository as shown in the following example:
  
-**Comment: I suggest making the following example its own section, i.e., how to prepare the CUDA driver offline.** 
+..
+	 **Comment: - I suggest making the following example its own section, i.e., how to prepare the CUDA driver offline.** 
 
 You can prepare the CUDA driver offline from a server connected to the CUDA repo by running the following commands as a *root* user:
 	  
@@ -770,13 +792,11 @@ The following is an example of generated output:
 
       $ vim /etc/yum.repos.d/cuda-10-1-local.repo
 
-Check the **name** of the file. **Comment:; for what?**
+   The following is an example of the correct output:
 
-The following is an example of generated output:
-
-.. code-block:: console
+   .. code-block:: console
 
-   $ name=cuda-10-1-local
+      $ name=cuda-10-1-local
    
 5. Clone the repository to a location where it can be copied from:
 
@@ -905,23 +925,27 @@ The server is ready for the SQream software installation.
 
          $ sysctl -n fs.file-max
 		 
-The desired output when checking the maximum value of the **fs.file** is greater or equal to **2097152**.
+    The desired output when checking the maximum value of the **fs.file** is greater or equal to **2097152**.
 
-   f. Run the following command as a SQream user: **Question - what do the following command parameter do? -c?**		 
+   f. Run the following command as a SQream user:
+   
+..
+	 **Comment: - **Question - what do the following command parameter do? -c?**		 
 		 
       .. code-block:: console
 
          $ ulimit -c -u -n	
 
-**Comment: See https://ss64.com/bash/ulimit.html**		 
+..
+	 **Comment: - See https://ss64.com/bash/ulimit.html**		 
 		 
-The following shows the desired output when **<fill in>**:
+    The following shows the desired output when **<fill in>**:
 
-.. code-block:: console
+    .. code-block:: console
 
-   $ core file size (blocks, -c) unlimited
-   $ max user processes (-u) 1000000
-   $ open files (-n) 1000000
+    $ core file size (blocks, -c) unlimited
+    $ max user processes (-u) 1000000
+    $ open files (-n) 1000000
 
 3. Configure the security limits by running the **echo -e** command as a root user logged in shell:
 
@@ -937,7 +961,7 @@ Enabling core dumps is recommended, but optional.
 
 **To enable core dumps:**
 
-1. Check the abrtd Status
+1. Check the *8abrtd** Status
 
 2. Set the limits
 
@@ -1029,7 +1053,8 @@ Setting the Output Directory of the /etc/sysctl.conf File
       $ kernel.core_pattern = /<tmp/core_dumps>/core-%e-%s-%u-%g-%p-%t
       $ fs.suid_dumpable = 2
 
-**Comment - leave a note that the user can choose his correct location of the folder.**
+..
+	 **Comment: - leave a note that the user can choose his correct location of the folder.**
 
 3. To apply the changes without rebooting the server, run:
 	  
@@ -1043,11 +1068,11 @@ Setting the Output Directory of the /etc/sysctl.conf File
 
      $ sudo cat /proc/sys/kernel/core_pattern
 	  
-The following shows the correct generated output:
+  The following shows the correct generated output:
 	  
-.. code-block:: console
+  .. code-block:: console
 
-   $ /tmp/core_dumps/core-%e-%s-%u-%g-%p-%t	  
+     $ /tmp/core_dumps/core-%e-%s-%u-%g-%p-%t	  
 	  
 5. Verify that the core dumping works:	  
 	  
@@ -1058,6 +1083,8 @@ The following shows the correct generated output:
 	  
 Verifying that the Core Dumps Work 
 ---------------------------------------------------	
+You can verify that the core dumps work only after installing and running SQream. This causes the server to crash and a new core.xxx file to be included in the folder that is written in **/etc/sysctl.conf**
+
 **To verify that the core dumps work:**
 
 1. Stop and restart all SQream services.
@@ -1068,11 +1095,9 @@ Verifying that the Core Dumps Work
 
      $ select abort_server();
 
-You can verify that the core dumps work only after installing and running SQream.
-   
-This causes the server to crash and a new core.xxx file to be included in the folder that is written in **/etc/sysctl.conf**
 
-**Question: what did the author mean by "Stage 3"?**
+..
+	 **Comment: - what did the author mean by "Stage 3"?**
    
 Troubleshooting Core Dumping 
 ---------------------------------------------------	
@@ -1094,11 +1119,11 @@ This section describes the troubleshooting procedure to be followed if all param
 
       $ ulimit -c
 
-If all parameters have been configured correctly, the correct output is:
+   If all parameters have been configured correctly, the correct output is:
 
-.. code-block:: console
+   .. code-block:: console
 
-   $ unlimited	
+      $ unlimited	
 
 4. If all parameters have been configured correctly, but running **ulimit -c** outputs **0**, run the following:
 
@@ -1127,211 +1152,4 @@ If all parameters have been configured correctly, the correct output is:
 
          $ ulimit -S -c ${DAEMON_COREFILE_LIMIT:-0} >/dev/null 2>&1
 
-   c. If the line is found, tag it with the **hash** symbol and reboot the server.
-
-**Comment: The rest of the document is the old content. Can it all be removed?**   
-	  
-	  
-	  
-	 
-Use a dedicated SQream DB administration account
-===================================================
-
-Create a user for SQream DB, and optionally assign it to the ``wheel`` group for ``sudo`` access.
-
-.. code-block:: console
-
-   $ useradd -m -u 1132 -U sqream
-   $ passwd sqream 
-   $ usermod -aG wheel sqream 
-
-The UID (1132 in the example above) is set to ensure all shared files are accessible by all workers.
-
-Configure the OS locale and timezone
-=====================================
-
-#. Set your OS to use UTF-8, which SQream DB uses for non-English language support.
-   
-   .. code-block:: console
-   
-      $ sudo localectl set-locale LANG=en_US.UTF-8
-
-#. Set the correct timezone for your server.
-   Refer to `the list of available timezones <https://en.wikipedia.org/wiki/List_of_tz_database_time_zones>`_ to find a timezone that matches your location.
-   
-   .. code-block:: console
-   
-      $ sudo timedatectl set-timezone America/New_York
-
-
-Configure NTP for clock synchronization
-=========================================
-
-SQream DB clusters rely on clock synchronization to function correctly.
-
-
-   .. code-block:: console
-   
-      $ sudo yum install -y ntp ntpdate
-      $ sudo systemctl enable ntpd
-      $ sudo systemctl start ntpd
-
-If your organization has an NTP server, configure it by adding records to ``/etc/ntpd.conf``, reloading the service, and checking that synchronization is enabled:
-
-   .. code-block:: console
-   
-      $ echo -e "\nserver <your NTP server address>\n" | sudo tee -a /etc/ntp.conf
-      $ sudo systemctl restart ntpd
-      $ sudo timedatectl
-               Local time: Sat 2019-10-12 17:26:13 EDT
-           Universal time: Sat 2019-10-12 21:26:13 UTC
-                 RTC time: Sat 2019-10-12 21:26:13
-                Time zone: America/New_York (EDT, -0400)
-              NTP enabled: yes
-         NTP synchronized: yes
-          RTC in local TZ: no
-               DST active: yes
-          Last DST change: DST began at
-                           Sun 2019-03-10 01:59:59 EST
-                           Sun 2019-03-10 03:00:00 EDT
-          Next DST change: DST ends (the clock jumps one hour backwards) at
-                           Sun 2019-11-03 01:59:59 EDT
-                           Sun 2019-11-03 01:00:00 EST
-
-
-
-Install recommended utilities
-===============================
-
-The following packages contain tools that are recommended but not required for using SQream DB.
-
-   .. code-block:: console
-   
-      $ sudo yum install -y bash-completion.noarch vim-enhanced.x86_64 vim-common.x86_64 net-tools iotop htop psmisc screen xfsprogs wget yum-utils deltarpm dos2unix tuned  pciutils
-
-
-Tuning OS parameters for performance and stability
-===================================================
-
-SQream DB requires certain OS parameters to be set on all hosts in your cluster.
-
-These settings affect:
-
-* Shared memory - Most OS installations may try to limit high throughput software like SQream DB.
-* Network - On high throughput operations like ingest, optimizing network connection parameters can boost performance
-* User limits - SQream DB may open a large amount of files. The default OS settings may cause some statements to fail if the system runs out of file descriptors.
-* Core dump creation rules
-
-#. Create a directory for core dumps
-
-   In this step, you will create a directory for writing core dumps - which you will configure in the next step.
-   
-   .. code-block:: console
-      
-      $ sudo mkdir /tmp/core_dumps
-
-   .. note::
-      Core dumps can be large - up to the size of the system memory (i.e. for a machine with 512GB of RAM, the size of the core dump will be 512GB).
-
-      Make sure the directory has enough space for writing a core dump.
-
-
-#. Set ``sysctl`` overrides to tune system performance
-   
-   .. code-block:: console
-      :linenos:
-      
-      $ sudo tee /etc/sysctl.d/sqreamdb.conf > /dev/null <<EOT
-      kernel.sysrq = 1
-      kernel.core_uses_pid = 1
-      kernel.core_pattern = /tmp/core_dumps/%f-core-%e-%s-%u-%g-%p-%t
-      kernel.pid_max = 524288
-      vm.max_map_count = 2042292
-      vm.dirty_background_ratio = 5
-      vm.dirty_ratio = 3
-      vm.swappiness = 1
-      vm.vfs_cache_pressure = 200
-      vm.zone_reclaim_mode = 0
-      fs.suid_dumpable = 2
-      fs.file-max = 2097152
-      EOT
-
-   .. note:: 
-      The settings above include provisioning for core dumps. Core dumps can be a valuable source of information in some scenarios, where stack traces and error logs are not enough.
-      
-      By default, the kernel writes core dump files in the current working directory of the process. SQream recommends overriding this setting and write the core dump files to a fixed directory.
-      
-      The setting on line 4 uses the directory you created in the previous step (``/tmp/core_dumps``).
-
-
-
-
-#. Increase the limit of open files and processes 
-
-   .. code-block:: console
-      
-         $ sudo tee -a /etc/security/limits.conf > /dev/null <<EOT
-         * soft nproc 524288
-         * hard nproc 524288
-         * soft nofile 524288
-         * hard nofile 524288
-         * soft core unlimited
-         * hard core unlimited
-         EOT
-
-#. Verify mount options for drives
-
-   SQream recommends XFS for local data storage.
-   The recommended XFS mount options are:
-   
-   ``rw,nodev,noatime,nobarrier,inode64``
-
-.. note:: Reboot your system for the above settings to take effect.
-
-Disable SELinux
-=================
-
-SELinux may interfere with NVIDIA driver installation and some SQream DB operations. Unless absolutely necessary, we recommend disabling it.
-
-#. Check if SELinux is enabled
-
-   .. code-block:: console
-      
-      $ sudo sestatus
-      SELinux status:                 disabled
-
-#. You can disable SELinux by changing the value of ``SELINUX`` parameter to ``disabled`` in ``/etc/selinux/config`` and rebooting.
-
-Secure the server with a firewall
-===================================
-
-Opening up ports in the firewall
----------------------------------
-
-The example below shows how to open up all ports required by SQream DB and related management interfaces. The example also takes into account up to 4 workers on the host.
-
-   .. code-block:: console
-      
-      $ sudo systemctl start firewalld
-      $ sudo systemctl enable firewalld
-      $ for p in {2812,3000,3001,3105,3108,5000-5003,5100-5103}; do sudo firewall-cmd --zone=public --permanent --add-port=${p}/tcp; done
-      $ sudo firewalld --reloadi
-
-
-Disabling the built in firewall
----------------------------------
-
-If not required, you can disable the server's firewall. This will reduce connectivity issues, but should only be done inside your internal network.
-
-   .. code-block:: console
-      
-      $ sudo systemctl disable firewalld
-      $ sudo systemctl stop firewalld
-
-
-.. rubric:: What's next?
-
-* :ref:`Test your SQream DB installation by creating your first table<first_steps>`
-
-* :ref:`Connect an external tool to SQream DB <third_party_tools>`
-
+   c. If the line is found, tag it with the **hash** symbol and reboot the server.
\ No newline at end of file