Skip to content

Commit

Permalink
[CN-571] Restructure AWS discovery documentation (#797)
Browse files Browse the repository at this point in the history 
* [CN-571] restucture AWS discovery documentation

* change formating

* remove leftover part

* add exmaple configs for clients

* small clean up

* Minor edits.

* Formatting fix

* Update deploying-on-aws.adoc

* Fix attempt for incorrect URL parsing

---------

Co-authored-by: Serdar Ozmen <[email protected]>
  • Loading branch information
@SeriyBg @Serdaro
SeriyBg and Serdaro authored Aug 23, 2023
1 parent cc515da commit 65b8df9
Showing 1 changed file with 235 additions and 49 deletions.
284 changes: 235 additions & 49 deletions docs/modules/deploy/pages/deploying-on-aws.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,28 @@ Before deploying Hazelcast on AWS, you must have the following:

* Linux Kernel 3.19+
+
TCP connections may get stuck when used with older Kernel versions, resulting in undefined timeouts.
WARNING: TCP connections may get stuck when used with older Kernel versions, resulting in undefined timeouts.

== Discovering Members Automatically

To make it easier to set up clusters on Amazon AWS, Hazelcast allows members to discover each other automatically, using discovery strategies.

When a member starts on AWS, it fetches a list of all running instances filtered by the member's AWS configuration settings. Then, each instance is checked one-by-one with its IP and each of the ports defined in the `hz-port` property. When a member is discovered under `IP:PORT`, then it joins the cluster.
When a member starts on AWS, it fetches a list of all running instances filtered by the member's AWS configuration settings. Then, each instance is checked one-by-one with its IP and each of the ports defined in the `hz-port` property (see the example configurations in this section for the property's description). When a member is discovered under `IP:PORT`, then it joins the cluster.

Choose from one of the following environments to configure a discovery strategy:

- EC2
- ECS/Fargate
- ECS/EC2
- AWS Elastic Beanstalk
- <<ec2-configuration, EC2>>
- <<ecsfargate-configuration, ECS/Fargate>>
- <<ecs-environment-with-ec2-discovery, ECS/EC2>>
- <<aws-elastic-beanstalk, AWS Elastic Beanstalk>>

== EC2 Configuration
== Configuring Hazelcast for EC2

You can configure both Hazelcast members and Hazelcast clients to discover clusters automatically.
You can configure both Hazelcast members and Hazelcast clients to automatically discover clusters deployed in Amazon EC2.

=== EC2 Hazelcast Member Discovery

You can configure Hazelcast in one of the following manners.
The following is an example AWS EC2 configuration snippet in a member, to enable automatic member discoveries in EC2. Each member of your cluster should have the same configuration.

[tabs]
====
Expand All @@ -41,10 +41,10 @@ XML::
<hazelcast>
<network>
<join>
<multicast enabled="false"/>
<aws enabled="true">
<tag-key>my-ec2-instance-tag-key</tag-key>
<tag-value>my-ec2-instance-tag-value</tag-value>
<multicast enabled="false"/> <1>
<aws enabled="true"> <2>
<tag-key>my-ec2-instance-tag-key</tag-key> <3>
<tag-value>my-ec2-instance-tag-value</tag-value> <4>
</aws>
</join>
</network>
Expand All @@ -59,52 +59,156 @@ hazelcast:
network:
join:
multicast:
enabled: false
enabled: false <1>
aws:
enabled: true
tag-key: my-ec2-instance-tag-key
tag-value: my-ec2-instance-tag-value
enabled: true <2>
tag-key: my-ec2-instance-tag-key <3>
tag-value: my-ec2-instance-tag-value <4>
```
--
Java::
+
--
```java
config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
config.getNetworkConfig().getJoin().getAwsConfig().setEnabled(true)
config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false); <1>
config.getNetworkConfig().getJoin().getAwsConfig().setEnabled(true) <2>
.setProperty("tag-key", "my-ec2-instance-tag-key") <3>
.setProperty("tag-value", "my-ec2-instance-tag-value"); <4>
```
--
====
<1> Multicast discovery strategy must be disabled. It is mandatory to set.
<2> AWS discovery strategy must be enabled. It is mandatory to set.
<3> Filter to look only for EC2 Instances with the given `tag-key`; comma-separated multiple values are supported, e.g., `KeyA,KeyB`; comma-separated values behave as `AND` conditions. It is mandatory to set.
<4> Filter to look only for EC2 Instances with the given `tag-value`; comma-separated multiple values are supported, e.g., `ValueA,ValueB`; comma-separated values behave as `AND` conditions. It is mandatory to set.


The following are the optional properties to be configured[[ec2_properties]]:

* `access-key`, `secret-key`: Access and secret keys of your AWS account; if not set, `iam-role` is used.
* `iam-role`: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html[IAM Role^] attached to the EC2 instance used to fetch credentials (if `access-key`/`secret-key` are not specified); if not set, the default IAM Role attached to the EC2 instance is used.
* `region`: Region where the Hazelcast members are running; default is the current region.
* `host-header`: `ec2`, `ecs`, or the URL of a EC2/ECS API endpoint; automatically detected by default.
* `security-group-name`: Filter to look only for the EC2 instances with the given security group.
* `connection-timeout-seconds`, `read-timeout-seconds`: Connection and read timeouts when making a call to the AWS API; its default is `10` seconds.
* `connection-retries`: Number of retries while connecting to AWS API; its default is `3`.
* `hz-port`: Range of ports where the discovery strategy looks for Hazelcast members; its default is `5701-5708`.

Note that if you don't specify any of the optional properties, then Hazelcast uses the IAM Role assigned to the EC2 Instance to form a cluster from all Hazelcast members running in the same region.

=== Client Discovery of EC2 Hazelcast Cluster

Hazelcast clients can automatically discover the Hazelcast cluster running in EC2 environment.
This section is for the Hazelcast Java clients. See https://github.com/hazelcast/hazelcast-cpp-client/blob/v5.3.0/Reference_Manual.md#57-enabling-hazelcast-aws-discovery[{cpp} client] and https://github.com/hazelcast/hazelcast-go-client-discovery[Go client] documentations to learn how to configure Hazelcast C++ and Go clients to automatically discover members running in EC2 environment.

- *Client running in EC2:*

To discover the Hazelcast cluster running on EC2 using the client running on EC2, you can use the same discovery parameters as mentioned in the previous section.

The following is an example configuration snippet in a Java client. Each client should have the same configuration.

[tabs]
====
XML::
+
--
```xml
<hazelcast-client>
<network>
<aws enabled="true">
<tag-key>my-ec2-instance-tag-key</tag-key>
<tag-value>my-ec2-instance-tag-value</tag-value>
</aws>
</network>
</hazelcast>
```
--
YAML::
+
--
```yaml
hazelcast-client:
network:
aws:
enabled: true
tag-key: my-ec2-instance-tag-key
tag-value: my-ec2-instance-tag-value
```
--
Java::
+
--
```java
config.getNetworkConfig().getAwsConfig().setEnabled(true)
.setProperty("tag-key", "my-ec2-instance-tag-key")
.setProperty("tag-value", "my-ec2-instance-tag-value");
```
--
====

The following optional properties can be configured[[ec2_properties]]:
- *Client running in ECS:*

* `access-key`, `secret-key`: access and secret keys of your AWS account; if not set, `iam-role` is used
* `iam-role`: IAM Role attached to EC2 instance used to fetch credentials (if `access-key`/`secret-key` not specified); if not set, default IAM Role attached to EC2 instance is used
* `region`: region where Hazelcast members are running; default is the current region
* `host-header`: `ec2`, `ecs`, or the URL of a EC2/ECS API endpoint; automatically detected by default
* `security-group-name`: filter to look only for EC2 instances with the given security group
* `tag-key`, `tag-value`: filter to look only for EC2 Instances with the given `tag-key`/`tag-value`; comma-separated multiple values are supported, e.g., `KeyA,KeyB`; comma-separated values behave as AND conditions
* `connection-timeout-seconds`, `read-timeout-seconds`: connection and read timeouts when making a call to AWS API; default to `10`
* `connection-retries`: number of retries while connecting to AWS API; default to `3`
* `hz-port`: a range of ports where the plugin looks for Hazelcast members; default is `5701-5708`
To discover the Hazelcast cluster running on EC2 using the client running on ECS, you need to specify the needed <<ec2_properties,EC2 related properties>> (`iam-role`, `security-group-name`) and to set the `ec2:DescribeInstances` permission.
If none of the ECS or EC2 related properties are specified, the AWS discovery strategy tries to discover the ECS members. If none is found, then it tries to discover EC2 members.

Note that if you don't specify any of the properties, then Hazelcast uses the IAM Role assigned to the EC2 Instance to form a cluster from all Hazelcast members running in same region.
The following is an example configuration snippet in a Java client. Each client should have the same configuration.

=== EC2 Hazelcast Client Configuration
[tabs]
====
XML::
+
--
```xml
<hazelcast-client>
<network>
<aws enabled="true">
<iam-role>my-iam-role</iam-role>
<security-group-name>my-security-group-name</security-group-name>
<tag-key>my-ec2-instance-tag-key</tag-key>
<tag-value>my-ec2-instance-tag-value</tag-value>
</aws>
</network>
</hazelcast>
```
--
YAML::
+
--
```yaml
hazelcast-client:
network:
aws:
enabled: true
iam-role: my-iam-role
security-group-name: my-security-group-name
tag-key: my-ec2-instance-tag-key
tag-value: my-ec2-instance-tag-value
```
--
Java::
+
--
```java
config.getNetworkConfig().getAwsConfig().setEnabled(true)
.setProperty("iam-role", "my-iam-role")
.setProperty("security-group-name", "my-security-group-name")
.setProperty("tag-key", "my-ec2-instance-tag-key")
.setProperty("tag-value", "my-ec2-instance-tag-value");
```
--
====

Hazelcast Client discovery parameters are the same as mentioned above.
* *Client running outside AWS:*

If Hazelcast Client is run *outside AWS*, then you need to always specify the following parameters:
For the Java clients running *outside AWS*, you always need to specify the following parameters:

- `access-key`, `secret-key` - IAM role cannot be used from outside AWS
- `region` - it cannot be detected automatically
- `use-public-ip` - must be set to `true`
- `access-key`, `secret-key` - IAM role cannot be used from outside AWS.
- `region` - it cannot be detected automatically.
- `use-public-ip` - must be set to `true`.

Note also that your EC2 instances must have public IP assigned.
Note also that your EC2 instances must have a public IP assigned.

Following are example declarative and programmatic configuration snippets.
The following is an example configuration snippet.

[tabs]
====
Expand Down Expand Up @@ -158,9 +262,6 @@ clientConfig.getNetworkConfig().getAwsConfig()
--
====

Hazelcast Client on EC2 can discover the members running in ECS cluster. For doing so you need specify the needed <<ecs_properties,ECS related properties>> (`cluster`, `family`, `service-name`) and make sure to set the <<ecs_permissions,required permissions>>.
If none of the ECS or EC2 related properties are specified, the AWS discovery tries to discover the EC2 members. If none is found, then it tries to discover ECS members.

== ECS/Fargate Configuration

The plugin works both for *Hazelcast Member Discovery* (forming Hazelcast cluster) and *Hazelcast Client Discovery*.
Expand Down Expand Up @@ -241,14 +342,102 @@ NOTE: If you don't specify any of the properties, then the plugin discovers all

NOTE: ECS discovery can use `IAM Role` assigned to the ECS Task instead of using `access-key`, `secret-key`.

=== ECS Hazelcast Client Configuration
=== Client Discovery of ECS Hazelcast Cluster

Hazelcast Client can automatically discover the Hazelcast cluster running in ECS environment.

* *Client running in ECS:*

To discover the Hazelcast Cluster running on ECS using the client running on the ECS, use the same discovery parameters as mentioned above.

Hazelcast Client discovery parameters are the same as mentioned above.
Following are example declarative and programmatic configuration snippets.

If Hazelcast Client is run *outside ECS cluster*, then you need to always specify the following parameters:
[tabs]
====
XML::
+
--
```xml
<hazelcast-client>
<network>
<aws enabled="true">
</aws>
</network>
</hazelcast-client>
```
--
YAML::
+
--
```yaml
hazelcast-client:
network:
aws:
enabled: true
```
--
Java::
+
--
```java
clientConfig.getNetworkConfig().getAwsConfig()
.setEnabled(true);
```
--
====

- `access-key`
- `secret-key` - IAM role cannot be used from outside AWS
* *Client running in EC2:*

Hazelcast Client on EC2 instances can discover the members running on ECS. For this you need to specify the needed <<ecs_properties,ECS related properties>> (`cluster`, `family`, `service-name`) and make sure to set the `ec2:DescribeInstances` permission.
If none of the ECS or EC2 related properties are specified, the AWS discovery tries to discover the EC2 members. If none is found, then it tries to discover ECS members.

Following are example declarative and programmatic configuration snippets.

[tabs]
====
XML::
+
--
```xml
<hazelcast-client>
<network>
<aws enabled="true">
<cluster>my-ecs-cluster</cluster>
<service-name>my-ecs-service</service-name>
</aws>
</network>
</hazelcast-client>
```
--
YAML::
+
--
```yaml
hazelcast-client:
network:
aws:
enabled: true
cluster: my-ecs-cluster
service-name: my-ecs-service
```
--
Java::
+
--
```java
clientConfig.getNetworkConfig().getAwsConfig()
.setEnabled(true)
.setProperty("cluster", "my-ecs-cluster")
.setProperty("service-name", "my-ecs-service");
```
--
====

* *Client running outside AWS:*

If Hazelcast Client runs *outside AWS*, then you need to always specify the following parameters:

- `access-key`, `secret-key` - IAM role cannot be used from outside AWS
- `region` - it cannot be detected automatically
- `cluster` - it cannot be detected automatically
- `use-public-ip` - must be set to `true`
Expand Down Expand Up @@ -306,9 +495,6 @@ clientConfig.getNetworkConfig().getAwsConfig()
--
====

Hazelcast Client on ECS can discover the members running on EC2 instances. For this you need to specify the needed <<ec2_properties,EC2 related properties>> (`iam-role`, `security-group-name`) and make sure to set the `ec2:DescribeInstances` permission.
If none of the ECS or EC2 related properties are specified, the AWS discovery tries to discover the ECS members. If none is found, then it tries to discover EC2 members.

== ECS Environment with EC2 Discovery

If you use ECS on EC2 instances (not Fargate), you may also set up your ECS Tasks to use `host` network mode and then use EC2 discovery mode instead of ECS. In that case, your Hazelcast configuration would look as follows.
Expand Down

0 comments on commit 65b8df9

Please sign in to comment.