<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE chapter SYSTEM "chapter.dtd">

<chapter>
  <header>
    <copyright>
      <year>2000</year><year>2020</year>
      <holder>Ericsson AB. All Rights Reserved.</holder>
    </copyright>
    <legalnotice>
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at
 
          http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License.
    
    </legalnotice>

    <title>Using TLS for Erlang Distribution</title>
    <prepared>P Nyblom</prepared>
    <responsible></responsible>
    <docno></docno>
    <approved></approved>
    <checked></checked>
    <date>2003-04-01</date>
    <rev>B</rev>
    <file>ssl_distribution.xml</file>
  </header>
  <p>This section describes how the Erlang distribution can use 
    TLS to get extra verification and security.</p>

    <p>The Erlang distribution can in theory use almost any 
      connection-based protocol as bearer. However, a module that 
      implements the protocol-specific parts of the connection setup is 
      needed. The default distribution module is <c>inet_tcp_dist</c> 
      in the Kernel application. When starting an
      Erlang node distributed, <c>net_kernel</c> uses this module to
      set up listen ports and connections.</p>

      <p>In the SSL application, an extra distribution
      module, <c>inet_tls_dist</c>, can be used as an
      alternative. All distribution connections will use TLS and
      all participating Erlang nodes in a distributed system must use
      this distribution module.</p>

      <p>The security level depends on the parameters provided to the
      TLS connection setup. Erlang node cookies are however always
      used, as they can be used to differentiate between two different
      Erlang networks.</p>

    <p>To set up Erlang distribution over TLS:</p>

      <list type="bulleted">
      <item><em>Step 1:</em> Build boot scripts including the
      SSL application.</item>
      <item><em>Step 2:</em> Specify the distribution module for 
      <c>net_kernel</c>.</item>
      <item><em>Step 3:</em> Specify the security options and other 
      SSL options.</item>
      <item><em>Step 4:</em> Set up the environment to always use TLS.</item>
    </list>

    <p>The following sections describe these steps.</p>

  <section>
    <title>Building Boot Scripts Including the SSL Application</title>
    <p>Boot scripts are built using the <c>systools</c> utility in the
      SASL application. For more information on <c>systools</c>,
      see the SASL documentation. This is only an example of
      what can be done.</p>

      <p>The simplest boot script possible includes only the Kernel 
      and STDLIB applications. Such a script is located in the 
      <c>bin</c> directory of the Erlang distribution. The source for the 
      script is found under the Erlang installation top directory under
      <c><![CDATA[releases/<OTP version>/start_clean.rel]]></c>.</p>

      <p>Do the following:</p>
      <list type="bulleted">
	<item><p>Copy that script to another location (and preferably another 
      name).</p></item>
        <item><p>Add the applications Crypto, Public Key, and 
	SSL with their current version numbers after the 
	STDLIB application.</p></item>
      </list>

    <p>The following shows an example <c>.rel</c> file with TLS 
    added:</p>
    <code type="none">
      {release, {"OTP  APN 181 01","R15A"}, {erts, "5.9"},
      [{kernel,"2.15"},
      {stdlib,"1.18"},
      {crypto, "2.0.3"},
      {public_key, "0.12"},
      {asn1, "4.0"},
      {ssl, "5.0"}
      ]}.
   </code>

   <p>The version numbers differ in your system. Whenever one of the 
   applications included in the script is upgraded, change the script.</p>

      <p>Do the following:</p>
      <list type="bulleted">
	<item><p>Build the boot script.</p>
	<p>Assuming the <c>.rel file</c> is stored in a file
	<c>start_ssl.rel</c> in the current directory, a boot script
	can be built as follows:</p></item>
      </list>
   <code type="none">
   1> systools:make_script("start_ssl",[]).    </code>

   <p>There is now a <c>start_ssl.boot</c> file in the current
   directory.</p>

   <p>Do the following:</p>
   <list type="bulleted">
     <item><p>Test the boot script. To do this, start Erlang with the
     <c>-boot</c> command-line parameter specifying this boot script
     (with its full path, but without the <c>.boot</c> suffix). In
     UNIX it can look as follows:</p></item>
   </list>
   <code type="none"><![CDATA[
$ erl -boot /home/me/ssl/start_ssl
Erlang (BEAM) emulator version 5.0
 
Eshell V5.0  (abort with ^G)
1> whereis(ssl_manager).
<0.41.0>    ]]></code>

    <p>The <c>whereis</c> function-call verifies that the SSL 
      application is started.</p>

      <p>As an alternative to building a bootscript, you can explicitly
      add the path to the SSL <c>ebin</c> directory on the command
      line. This is done with command-line option <c>-pa</c>. This
      works as the SSL application does not need to be started for the
      distribution to come up, as a clone of the SSL application is
      hooked into the Kernel application. So, as long as the
      SSL application code can be reached, the distribution starts. 
      The <c>-pa</c> method is only recommended for testing purposes.</p>

      <note><p>The clone of the SSL application must
      enable the use of the SSL code in such an early bootstage as
      needed to set up the distribution. However, this makes it
      impossible to soft upgrade the SSL application.</p></note>
  </section>

  <section>
    <title>Specifying Distribution Module for net_kernel</title>
    <p>The distribution module for TLS is named <c>inet_tls_dist</c>
      and is specified on the command line with option <c>-proto_dist</c>.
      The argument to <c>-proto_dist</c> is to be the module
      name without suffix <c>_dist</c>. So, this distribution
      module is specified with <c>-proto_dist inet_tls</c> on the
      command line.</p>

    <p>Extending the command line gives the following:</p>
    <code type="none">
$ erl -boot /home/me/ssl/start_ssl -proto_dist inet_tls    </code>

<p>For the distribution to be started, give the emulator a name as well:</p>
    <code type="none">
$ erl -boot /home/me/ssl/start_ssl -proto_dist inet_tls -sname ssl_test
Erlang (BEAM) emulator version 5.0 [source]
 
Eshell V5.0  (abort with ^G)
(ssl_test@myhost)1>     </code>

    <p>However, a node started in this way refuses to talk
      to other nodes, as no TLS parameters are supplied
      (see the next section).</p>
  </section>

  <section>
    <title>Specifying TLS Options</title>

    <p>
      The TLS distribution options can be written into a file
      that is consulted when the node is started.  This file name
      is then specified with the command line argument
      <c>-ssl_dist_optfile</c>.
    </p>
    <p>
      Any available TLS option can be specified in an options file,
      but note that options that take a <c>fun()</c> has to use
      the syntax <c>fun Mod:Func/Arity</c> since a function
      body cannot be compiled when consulting a file.
    </p>
    <p>
      Do not tamper with the socket options
      <c>list</c>, <c>binary</c>, <c>active</c>, <c>packet</c>,
      <c>nodelay</c> and <c>deliver</c> since they are used
      by the distribution protocol handler itself.
      Other raw socket options such as <c>packet_size</c> may
      interfere severely, so beware!
    </p>
    <p>
      For TLS to work, at least a public key and a certificate
      must be specified for the server side.
      In the following example, the PEM file
      <c>"/home/me/ssl/erlserver.pem"</c> contains both
      the server certificate and its private key.
    </p>
    <p>
      Create a file named for example
      <c>"/home/me/ssl/ssl_test@myhost.conf"</c>:
    </p>
    <code type="none"><![CDATA[
[{server,
  [{certfile, "/home/me/ssl/erlserver.pem"},
   {secure_renegotiate, true}]},
 {client,
  [{secure_renegotiate, true}]}].]]>
    </code>
    <p>
      And then start the node like this
      (line breaks in the command are for readability,
      and shall not be there when typed):
    </p>
    <code type="none"><![CDATA[
$ erl -boot /home/me/ssl/start_ssl -proto_dist inet_tls
  -ssl_dist_optfile "/home/me/ssl/ssl_test@myhost.conf"
  -sname ssl_test]]>
    </code>
    <p>
      The options in the <c>{server, Opts}</c> tuple are used
      when calling <c>ssl:handshake/3</c>, and the options in the
      <c>{client, Opts}</c> tuple are used when calling
      <c>ssl:connect/4</c>.
    </p>
    <p>
      For the client, the option
      <c>{server_name_indication, atom_to_list(TargetNode)}</c>
      is added when connecting.
      This makes it possible to use the client option
      <c>{verify, verify_peer}</c>,
      and the client will verify that the certificate matches
      the node name you are connecting to.
      This only works if the the server certificate is issued
      to the name <c>atom_to_list(TargetNode)</c>.
    </p>
    <p>
      For the server it is also possible to use the option
      <c>{verify, verify_peer}</c> and the server will only accept
      client connections with certificates that are trusted by
      a root certificate that the server knows.
      A client that presents an untrusted certificate will be rejected.
      This option is preferably combined with
      <c>{fail_if_no_peer_cert, true}</c> or a client will
      still be accepted if it does not present any certificate.
    </p>
    <p>
      A node started in this way is fully functional, using TLS
      as the distribution protocol.
    </p>
  </section>

  <section>
    <title>Specifying TLS Options (Legacy)</title>

    <p>
      As in the previous section the PEM file
      <c>"/home/me/ssl/erlserver.pem"</c> contains both
      the server certificate and its private key.
    </p>

    <p>On the <c>erl</c> command line you can specify options that the
    TLS distribution adds when creating a socket.</p>

    <p>The simplest TLS options in the following list can be specified 
    by adding the 
    prefix <c>server_</c> or <c>client_</c> to the option name:</p>
    <list type="bulleted">
      <item><c>certfile</c></item>
      <item><c>keyfile</c></item>
      <item><c>password</c></item>
      <item><c>cacertfile</c></item>
      <item><c>verify</c></item>
      <item><c>verify_fun</c> (write as <c>{Module, Function, InitialUserState}</c>)</item>
      <item><c>crl_check</c></item>
      <item><c>crl_cache</c> (write as Erlang term)</item>
      <item><c>reuse_sessions</c></item>
      <item><c>secure_renegotiate</c></item>
      <item><c>depth</c></item>
      <item><c>hibernate_after</c></item>
      <item><c>ciphers</c> (use old string format)</item>
    </list>

    <p>Note that <c>verify_fun</c> needs to be written in a different
    form than the corresponding TLS option, since funs are not
    accepted on the command line.</p>

    <p>The server can also take the options <c>dhfile</c> and
    <c>fail_if_no_peer_cert</c> (also prefixed).</p>

    <p><c>client_</c>-prefixed options are used when the distribution 
    initiates a connection to another node. <c>server_</c>-prefixed 
    options are used when accepting a connection from a remote node.</p>

    <p>Raw socket options, such as <c>packet</c> and <c>size</c> must not 
    be specified on the command line.</p>

    <p>The command-line argument for specifying the TLS options is named
    <c>-ssl_dist_opt</c> and is to be followed by pairs of
    SSL options and their values. Argument <c>-ssl_dist_opt</c> can
    be repeated any number of times.</p>

    <p>
      An example command line doing the same as the example
      in the previous section can now look as follows
      (line breaks in the command are for readability, 
      and shall not be there when typed):
    </p>
    <code type="none"><![CDATA[
$ erl -boot /home/me/ssl/start_ssl -proto_dist inet_tls
  -ssl_dist_opt server_certfile "/home/me/ssl/erlserver.pem"
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true
  -sname ssl_test
Erlang (BEAM) emulator version 5.0 [source]

Eshell V5.0  (abort with ^G)
(ssl_test@myhost)1>]]>
    </code>
  </section>

  <section>
    <title>Setting up Environment to Always Use TLS (Legacy)</title>
    <p>A convenient way to specify arguments to Erlang is to use environment 
      variable <c>ERL_FLAGS</c>. All the flags needed to
      use the TLS distribution can be specified in that variable and are
      then interpreted as command-line arguments for all
      subsequent invocations of Erlang.</p>

    <p>In a Unix (Bourne) shell, it can look as follows (line breaks are for
      readability, they are not to be there when typed):</p>
    <code type="none">
$ ERL_FLAGS="-boot /home/me/ssl/start_ssl -proto_dist inet_tls
  -ssl_dist_opt server_certfile /home/me/ssl/erlserver.pem
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"
$ export ERL_FLAGS
$ erl -sname ssl_test
Erlang (BEAM) emulator version 5.0 [source]
 
Eshell V5.0  (abort with ^G)
(ssl_test@myhost)1> init:get_arguments().
[{root,["/usr/local/erlang"]},
 {progname,["erl "]},
 {sname,["ssl_test"]},
 {boot,["/home/me/ssl/start_ssl"]},
 {proto_dist,["inet_tls"]},
 {ssl_dist_opt,["server_certfile","/home/me/ssl/erlserver.pem"]},
 {ssl_dist_opt,["server_secure_renegotiate","true",
                "client_secure_renegotiate","true"]
 {home,["/home/me"]}]    </code>

    <p>The <c>init:get_arguments()</c> call verifies that the correct
      arguments are supplied to the emulator.</p>
  </section>

  <section>
    <title>Using TLS distribution over IPv6</title>
    <p>It is possible to use TLS distribution over IPv6 instead of
    IPv4. To do this, pass the option <c>-proto_dist inet6_tls</c>
    instead of <c>-proto_dist inet_tls</c> when starting Erlang,
    either on the command line or in the <c>ERL_FLAGS</c> environment
    variable.</p>

    <p>An example command line with this option would look like this:</p>
    <code type="none"><![CDATA[
$ erl -boot /home/me/ssl/start_ssl -proto_dist inet6_tls
  -ssl_dist_optfile "/home/me/ssl/ssl_test@myhost.conf"
  -sname ssl_test]]>
    </code>

    <p>A node started in this way will only be able to communicate with
    other nodes using TLS distribution over IPv6.</p>
  </section>
</chapter>
