Skip to content

Getting Started

Sunlight platform can run on top of the AWS bare-metal instances to fully take advantage of the high performance offered by Sunlight platform and also the flexibility and availability provided by AWS.

Sunlight platform on AWS is running as a multi-node cluster in a similar topology as the on-premise setup. On the AWS environment, each Sunlight Node is provisioned on top of the bare-metal instance and multiple Nodes are connected and communicating using the AWS networking infrastructure. Each Sunlight Node is equipped with a number of network interfaces used for both management and data. The total aggregate network bandwidth that can be achieved by a Node is capped by the AWS instance capability. The storage capacity is also based on the bare-metal instance specification.

If it is the first time you know Sunlight and would like to try Sunlight, please submit a request via our online request support portal to obtain a Sunlight license to be used for the deployment of Sunlight on AWS.

Deploying a Sunlight platform on AWS would require the following three major steps.

  1. AWS Environment Preparation

  2. Sunlight on AWS Manager (SAM) Setup

  3. Sunlight Platform Deployment

Supported AWS Bare-metal Instances

The following instances have been validated and officially supported to run the Sunlight Enterprise Software platform. More types of bare-metal instances will be supported in the future.

AWS Instance Type CPU Memory Network Storage More Info
i3.metal Intel Xeon E5-2686 v4 1 512 GB 25 Gbps 8 x 1.7TB NVMe AWS Guide

Note

  1. Intel Xeon E5-2686 v4 (Broadwell) has CPU frequency at 2.3 GHz and provides 72 logical processors on 36 physical cores with hyper-threading enabled.

AWS Environment Preparation

  1. Check if your AWS account has the limitation to create the bare-metal instance. If it does you can go to AWS Support Centre and request to increase your limit for the metal instance (e.g. i3.metal) in the region (e.g. EU (Frankfurt)) where you are going to deploy Sunlight. AWS instance limit

  2. Obtain the access to use the AWS resources. When you deploy the Sunlight platform on AWS, you specify your AWS security credentials to verify who you are and whether you have permission to access the resources that you are requesting. More information is available at AWS Security Credentials. Clicking "Create New Access Key" as shown below to generate a new access key and have the Access Key ID and Secret Access Key ready to be used later. AWS credential setup

  3. Prepare a key pair to be used for the instances on AWS. You can use Amazon EC2 to create your key pair. For more information, see Creating a Key Pair Using Amazon EC2. Alternatively, you could use a third-party tool and then import the public key to Amazon EC2. For more information, see Importing Your Own Public Key to Amazon EC2. AWS key pair

  4. Create an AWS t2.micro instance using any Linux image (Ubuntu is tested and recommended), which servers as a manager to manage the Sunlight cluster on AWS. This instance type is selected because it is under the free tier on AWS. This manager instance does not require massive storage, e.g. 5 GiB EBS volume.

Sunlight on AWS Manager (SAM) Setup

Install the required packages to run SAM

In order to enable the SAM tool to work with AWS, the following packages need to be installed on the SAM instance.

  • Python 2.7

  • AWS Python API library boto3

The following CLI could be used to install these packages on Ubuntu.

sudo apt update
sudo apt install python2.7 python-pip
sudo pip install boto3

Download and configure SAM

For management of the Sunlight clusters on AWS, Sunlight has developed a minimal CLI tool, namely Sunlight on AWS Manager (SAM). SAM is a python CLI tool that has a set of commands for creating, terminating (deleting) and listing Sunlight Clusters. The tool will also be updated and improved to ensure a smooth deployment on AWS. The plan for the SAM is to become an open-source server that can be controlled through a Web-based UI.

Download the SAM tool to the home directory of manager instance and extract the package.

wget https://assets.sunlight.io/aws/sunlight_sam.tar.gz
tar -xzvf sunlight_sam.tar.gz

Each Sunlight cluster is defined by a JSON template file that describes the configuration of the cluster, i.e. how many nodes, which AMI images to use for the master and slave nodes, etc. There is a cluster.template.json file in ~/sunlight_sam/data/ as follows, which is a basic two-node cluster setup.

{
  "Name": "SunlightCluster",
  "NVMasterImage": "ami-0db02a0cb41bd227f",
  "NVSlaveImage": "ami-031cd48aed510a25f",
  "CIDR": "172.35.1.0/24",
  "NVNodes": [
      {
        "Name": "Master1",
        "NVNodeType": "master",
        "SpotInstance": true,
        "InstanceType": "i3.metal",
        "NetworkInterfaces": [
          {"Name": "Sunlight Management", "PublicIp": false},
          {"Name": "Sunlight Controller PRIV", "PublicIp": false},
          {"Name": "Sunlight Controller PUB", "PublicIp": true},
          {"Name": "Sunlight Management", "PublicIp": false},
          {"Name": "Sunlight Management", "PublicIp": false}
        ]
      },
    {
      "Name": "Slave1",
      "NVNodeType": "slave",
      "SpotInstance": true,
      "InstanceType": "i3.metal",
      "NetworkInterfaces": [
        {"Name": "Sunlight Management", "PublicIp": false},
        {"Name": "Sunlight Management", "PublicIp": false},
        {"Name": "Sunlight Management", "PublicIp": false},
        {"Name": "Sunlight Management", "PublicIp": false}
      ]
    }
    ]
}
  • Name: The name could be changed to describe your cluster if more than one Sunlight cluster is deployed.
  • NVMasterImage/NVSlaveImage: The AMI image ID included in the template file doesn't need to be changed unless special image and instruction is provided from Sunlight.
  • CIDR: The IPv4 CIDR block defines the subnet which the network interfaces of the nodes are attached to. This subnet is used for Sunlight management traffic of a single Sunlight cluster, so different subnet should be used to avoid conflicts when multiple Sunlight clusters are created in the same VPC. If there are other servers running in the same VPC, please also make sure they are not conflicting with each other.
  • NVNodes: The specification of the Sunlight Nodes.
  • NVNodeType: The Node type could be either master or slave. In each cluster one master node is required and one or more slave nodes could be optionally added.
  • SpotInstance: The SpotInstance parameter defines whether to request the metal instances to be spawned as AWS Spot Instances, which could reduce the cost significantly. The AWS Spot Instance option currently integrated with SAM could only request Spot Instance of six-hour lifecycle. More Spot Instance options will be added to SAM in the future.
  • InstanceType: The AWS bare-metal instance type to be used for this cluster. Please refer to the Supported AWS Bare-metal Instances.
  • NetworkInterfaces: It defines how many network interfaces are created and attached to the node. The default settings should be used for both master and slave nodes.

Provide the credential of your AWS account

Provide your AWS credential generated previously in the 2nd step of AWS Environment Preparation. Go to the home directory of the user who would run the SAM tool in the manager instance. Create a directory named ".aws" and the following two files.

mkdir ~/.aws
echo -e "[default]\nregion=<YOUR_SELECTED_REGION>\noutput=json" >> ~/.aws/config
echo -e "[default]\naws_access_key_id = <YOUR_KEY_ID>\naws_secret_access_key = <YOUR_KEY>" >> ~/.aws/credentials

Please replace the following parameters with your specific values. For example:

  • <YOUR_SELECTED_REGION>: eu-central-1

  • <YOUR_KEY_ID>: AKIAIWSXCBRCHAAK7UD7Q

  • <YOUR_KEY>: RpO+ZEJGVMshdVxszsdf+sdfnRaqQashnTpZawh

Provide the key pair on your AWS EC2

Add your key pair prepared previously at step 3 of AWS Environment Preparation under the directory "~/sunlight_sam/data/keypairs/". Please name your private and public key file as "aws_manager_rsa" and "aws_manager_rsa.pub".

Provide the Sunlight license

Add your Sunlight license as file named license.lic under the directory "~/sunlight_sam/data/".

Sunlight Platform Deployment

Creating a Sunlight Cluster on AWS

For each Sunlight cluster a template file need to be prepared. Once the template file is ready, run the following command to create the cluster. Please make sure you run it using the same user who has the ".aws" directory under its home directory.

cd sunlight_sam
python ./aws_manager.py cluster create --file data/cluster.template.json

Note

This creation process takes around 20 minutes to deploy the Sunlight cluster on AWS. Please do not interrupt the process, otherwise there would be residue and you would have to perform a cluster terminate action.

When the creation is finished and the cluster is ready to use, the SAM tool will short summary of the cluster created as the example below.

Cluster NV-5784976794906921 Description:
{'ControllerPrivateIP': '172.35.1.217',
 'ControllerPublicIP': '18.185.235.220',
 'ManagerVM': {'AWSID': 'i-0b6111c98773fd2a4',
               'NVClusterID': 'NV-5784976794906921',
               'Name': 'manager',
               'PublicIP': '35.159.28.33'},
 'NVClusterID': 'NV-5784976794906921'}

The important information here is the ControllerPublicIP entry, which is the IP address to access the Sunlight Dashboard of this freshly created cluster. The other important entry is the NVClusterID, which is used to list the attributes of the cluster or terminating it.

Listing the cluster(s)

The user can also view the attributes of one or more clusters:

python ./aws_manager.py cluster list --json
python ./aws_manager.py cluster list --cluster_id=NV-XXXXXXXXXXXX --json
python ./aws_manager.py cluster list --cluster_id=NV-XXXXXXXXXXXX --json --filters=ControllerPublicIP

Those commands provide a JSON-based overview of the cluster(s). The user can provide a NVClusterID (for viewing a specific cluster) and/or filters for the attributes. For example:

# python ./aws_manager.py cluster list --json
{
    "NVClusterID": "NV-8692013723906030",
    "ManagerVM": {
        "PublicIP": "18.185.144.83",
        "NVClusterID": "NV-8692013723906030",
        "AWSID": "i-021937f6a6b138172",
        "Name": "manager"
    },
    "NVNodes": [
        {
            "NVNodeType": "master",
            "NVNodeID": "0e7bf73a",
            "NVNodeMAC": "0ab80e7bf73a",
            "AWSID": "i-07aef5f61cacae4c8",
            "StorageNodeIP": "7.247.58.1",
            "ControllerPublicIP": "18.194.246.234",
            "AWSSpotID": "sir-2cj8jt8q",
            "NVClusterID": "NV-8692013723906030",
            "ControllerPrivateIP": "172.36.3.130",
            "NetworkInterfaces": [
                {
                    "MAC": "0a:b8:0e:7b:f7:3a",
                    "PrivateIP": "172.36.3.122",
                    "AWSID": "eni-002ecd92e2339bfbf",
                    "Name": "Integrated ENA"
                },
                {
                    "PublicIP": "18.194.246.234",
                    "MAC": "0a:e2:70:1c:20:2c",
                    "PrivateIP": "172.36.3.130",
                    "AWSID": "eni-0e33d4ea8f3d84584",
                    "Name": "Openstack PUB"
                },
                {
                    "MAC": "0a:a8:36:11:41:70",
                    "PrivateIP": "172.36.3.11",
                    "AWSID": "eni-05a476183154cd714",
                    "Name": "Integrated ENA"
                },
                {
                    "MAC": "0a:a4:eb:57:dd:00",
                    "PrivateIP": "172.36.3.99",
                    "AWSID": "eni-0ebaf53a0b6105f7c",
                    "Name": "Integrated ENA"
                },
                {
                    "MAC": "0a:bf:d7:6d:6a:bc",
                    "PrivateIP": "172.36.3.25",
                    "AWSID": "eni-0f14145761ead4a6b",
                    "Name": "Openstack PRIV"
                }
            ]
        },
        {
            "NVNodeType": "slave",
            "NVNodeID": "bbc64e36",
            "NVNodeMAC": "0a10bbc64e36",
            "AWSID": "i-010608b0ae989f8dc",
            "AWSSpotID": "sir-3ak8jxjm",
            "NVClusterID": "NV-8692013723906030",
            "StorageNodeIP": "7.78.54.1",
            "NetworkInterfaces": [
                {
                    "MAC": "0a:23:0f:90:2b:b0",
                    "PrivateIP": "172.36.3.235",
                    "AWSID": "eni-0ed92b0df0b45483f",
                    "Name": "NetDD"
                },
                {
                    "MAC": "0a:35:fb:47:a0:8e",
                    "PrivateIP": "172.36.3.24",
                    "AWSID": "eni-0ea761f2e0d062f34",
                    "Name": "Integrated ENA"
                },
                {
                    "MAC": "0a:10:bb:c6:4e:36",
                    "PrivateIP": "172.36.3.114",
                    "AWSID": "eni-0eff0008934eab438",
                    "Name": "Integrated ENA"
                },
                {
                    "MAC": "0a:61:7f:c3:51:92",
                    "PrivateIP": "172.36.3.186",
                    "AWSID": "eni-0cba20de1db3c2fe5",
                    "Name": "Integrated ENA"
                }
            ]
        }
    ],
    "ControllerPublicIP": "18.194.246.234",
    "ControllerPrivateIP": "172.36.3.130"
}

Terminating a cluster

To terminate a Sunlight cluster on AWS, run the following command with the NVClusterID.

python ./aws_manager.py cluster terminate --cluster_id=NV-XXXXXXXXXXX

Note

This terminating process takes around 12 minutes. Please do not interrupt this process to avoid any residue.

For more info about the SAM tool:

python ./aws_manager.py cluster [list] --help

The Sunlight support team is on stand by to assist with any installation and configuration issues you may encounter. Link : support portal