Multi-Process ArcViz

In a multi-process configuration for the Arcadia Visualization Server (Arcviz), one master and multiple worker processes combine to serve many url requests, smoothly scaling with the number of users without loss of performance.

Availability Note. Multi-process ArcViz is only available on Cloudera Installations.

In Release 4.2.0.0, Arcadia Enterprise introduced the multi-process Arcviz server that routes URL requests to the multiple processes through a proxy based on specified rules; some of the requests run on the master process, while others are distributed among the worker processes.

Routing URL requests through proxy server — Architecture for Multi-Process ArcViz

The preceding diagram illustrates the implementation of the multi-process ArcViz:

The Proxy Server listens on port 38888, the only external port, to URL requests from Web Browsers.
The ArcViz Master and ArcViz Worker processes listen on internal ports (by default, 30140 and up) for URL requests that the Proxy Server routs to them.
The Proxy Server also services Static Assets such as images, Javascript files, and so on.

Multi-process set-up may be configured during ArcViz start-up, as described in Setting up Multi-Process ArcViz. If you change the number of processes, you must subsequently re-start ArcViz.

This section includes the following topics:

Best Practices for Multi-Process ArcViz

Consider the following recommended allocation of resources for achieving the best performance to in multi-process ArcViz setup:

Memory
We suggest an additional 8 GB of memory allocation for each additional process.

This means a total increase of 24 GB more for the default ArcViz configuration of 3 processes, compared to the legacy single-process approach (default prior to Release 4.2.0.0).
ArcEngine Sessions
We recommend that you allocate 40 ArcEngine sessions for each additional ArcViz process, for a total of 120.

For systems that use multiple connections that reference the same ArcEngine system, we recommend that you adjust the number of ArcEngine sessions based on the following formula: 40 x number_processes x number_connections.

You can adjust these parameters through the Cloudera Manager interface, by searching for the Arcadia Analytics Engine Advanced Configuration Snippet, and setting the following values in the Arcadia Analytics Engine Default Group:
```
-be_service_threads=120
-fe_service_threads=120
```
Metadata Connections
We recommend an allocation of 60 metadata connections for each ArcViz process, for a total of 180.
File Handles for Arcadia users
We recommend that you plan for approximately 50 active users for each process, for a total of approximately 150 users on the default configuration of multi-threaded ArcViz.

We also suggest that you make the combined number of open files (sessions, connections, libraries and other resources) sufficiently large to accommodate a large number of users.

To test this, run the following command on the host of the Arcadia Visualization Server:
```
$ su -arcadia
$ unlimit -n
```
If the number is significantly less than 32,768, we recommend that you add the following line to the limits.conf file in the /etc/security/ directory:
```
arcadia  soft  nofile  32768
```

Limitations of Multi-Process ArcViz

Consider the following limitations of the multi-process ArcViz:

If your configuration uses the SQLite Metastore, the ArcViz reverts to the single process (legacy) mode, regardless of the configuration.
If using TLS/SSL and an encrypted private key, the key must be in traditional RSA format, containing both Proc and DEK headers.