Payara Server と Payara Micro のクラスタリング

この記事は、Payara Advent Calendar 2016 の 5 日目です。昨日は「Payara Micro の設計と実装」です。

Payara Micro のマニュアルには、Payara Micro は Payara Server とクラスタを組むことができるという記載があります (原文: "Payara Micro can cluster with Payara Server and share web session and JCache data.")。そこで、実際に試してみました。

なお、実施時期の都合により Payara 4.1.1.163 (旧バージョン) での検証となることをはじめにお断りしておきます。

1. 事前準備

Payara Micro は既定で Hazelcast の Auto Clustering が有効になっているため、特別な設定は必要ありません。一方、Payara Server は GlassFish との互換性維持のためか、Hazelcast が既定で無効となっているため、最初に有効化する必要があります。

Payara Server の Hazelcast を有効化するには、大きく管理コンソール (GUI) を使用する方法と、asadmin の起動時オプションで指定する方法の 2 種類があります。

1.1. 管理コンソールから Hazelcast を有効化する

管理コンソールから Hazelcast の設定を変更するには、図1 の Hazelcast 構成画面から操作します。手前味噌ですが、この画面の日本語化を担当したのは私です (Payara 4.1.153 以降)。

Hazelcast Configuration.png
figure 1.1 - Hazelcast 構成画面

スクリーンショットの赤枠で囲った部分、「Hazelcast分散キャッシュ機能を有効にするかどうかを決定します。」のチェックを ON にして、設定を保存してください。再起動は不要です。これだけで Payara Server の Hazelcast は有効になります。

もし、はじめからチェックが ON になっていた場合は、既に Hazelcast が有効になっています。

管理コンソールからだけでは本当に Hazelcast が有効化されたのか確認できないため (メッセージを信じるしかない)、念のため server.log の中身を見てみます。ここではスペースの関係上、Hazelcast に関連する場所だけ抜粋します。

[2016-12-05T23:38:15.075+0900] [Payara 4.1] [INFO] [] [com.hazelcast.instance.DefaultAddressPicker] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095075] [levelValue: 800] [[
  [LOCAL] [development] [3.6.4] Prefer IPv4 stack is true.]]

[2016-12-05T23:38:15.134+0900] [Payara 4.1] [INFO] [] [com.hazelcast.instance.DefaultAddressPicker] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095134] [levelValue: 800] [[
  [LOCAL] [development] [3.6.4] Picked Address[192.168.184.67]:5900, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5900], bind any local is true]]

[2016-12-05T23:38:15.147+0900] [Payara 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095147] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Hazelcast 3.6.4 (20160701 - 5b94d9f) starting at Address[192.168.184.67]:5900]]

[2016-12-05T23:38:15.147+0900] [Payara 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095147] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.]]

[2016-12-05T23:38:15.148+0900] [Payara 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095148] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Configured Hazelcast Serialization version : 1]]

[2016-12-05T23:38:15.281+0900] [Payara 4.1] [INFO] [] [com.hazelcast.spi.OperationService] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095281] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Backpressure is disabled]]

[2016-12-05T23:38:15.300+0900] [Payara 4.1] [INFO] [] [com.hazelcast.spi.impl.operationexecutor.classic.ClassicOperationExecutor] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095300] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Starting with 2 generic operation threads and 4 partition operation threads.]]

[2016-12-05T23:38:15.706+0900] [Payara 4.1] [INFO] [] [com.hazelcast.instance.Node] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095706] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Creating MulticastJoiner]]

[2016-12-05T23:38:15.710+0900] [Payara 4.1] [INFO] [] [com.hazelcast.core.LifecycleService] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095710] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Address[192.168.184.67]:5900 is STARTING]]

[2016-12-05T23:38:15.790+0900] [Payara 4.1] [INFO] [] [com.hazelcast.nio.tcp.nonblocking.NonBlockingIOThreadingModel] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443095790] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads]]

[2016-12-05T23:38:18.600+0900] [Payara 4.1] [INFO] [] [com.hazelcast.cluster.impl.MulticastJoiner] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098600] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] 


Members [1] {
	Member [192.168.184.67]:5900 this
}
]]

[2016-12-05T23:38:18.651+0900] [Payara 4.1] [INFO] [] [com.hazelcast.jmx.ManagementService] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098651] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Hazelcast JMX agent enabled.]]

[2016-12-05T23:38:18.679+0900] [Payara 4.1] [INFO] [] [com.hazelcast.core.LifecycleService] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098679] [levelValue: 800] [[
  [192.168.184.67]:5900 [development] [3.6.4] Address[192.168.184.67]:5900 is STARTED]]

[2016-12-05T23:38:18.691+0900] [Payara 4.1] [INFO] [] [fish.payara.nucleus.hazelcast.HazelcastCore] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098691] [levelValue: 800] [[
  Hazelcast Instance Bound to JNDI at payara/Hazelcast]]

[2016-12-05T23:38:18.692+0900] [Payara 4.1] [INFO] [] [fish.payara.nucleus.hazelcast.HazelcastCore] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098692] [levelValue: 800] [[
  JSR107 Caching Provider Bound to JNDI at payara/CachingProvider]]

[2016-12-05T23:38:18.692+0900] [Payara 4.1] [INFO] [] [fish.payara.nucleus.hazelcast.HazelcastCore] [tid: _ThreadID=136 _ThreadName=admin-thread-pool(6)] [timeMillis: 1478443098692] [levelValue: 800] [[
  JSR107 Default Cache Manager Bound to JNDI at payara/CacheManager]]

Hazelcast クラスタへ参加したことを確定付けているのは、上記ログのうち以下の部分です。Hazelcast でクラスタ・ノードの追加・削除を繰り返しているとその度に現れるメッセージですので、覚えておきましょう。

Members [1] {
	Member [192.168.184.67]:5900 this
}

ちなみに、現時点では 1 つのノードしかクラスタに参加していないため、Member は 1 つしか表示されていません。

1.2. asadmin から Hazelcast を有効化する

管理コンソールが使用できない環境では、asadmin から Hazelcast を有効化します。

asadmin set-hazelcast-configuration --enabled=true [ --dynamic=true ]

asadmin を使用する場合には、サーバーを起動するたびに設定を行う必要があるます。

1.3. domain.xml に設定を記述する

domain を停止した状態で domain.xml に以下の 1 行を追加します。

<hazelcast-runtime-configuration enabled="true"/>

2. Payara Server を起動する

Hazelcast が有効になっている状態で Payara Server を起動します。念のため、1.1. で示したようなログが server.log に出力されていることを確認してください。

3. Payara Micro を起動する

ここまでの手順に引き続き、Payara Micro を起動します。以下に起動時のログを示します。

hasunuma@pasiphae:~$ java -jar greeting\-0.0.1\-SNAPSHOT.jar                    
[2016-12-05T23:40:54.549+0900] [Payara Micro 4.1] [INFO] [NCLS-CORE-00087] [javax.enterprise.system.core] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254549] [levelValue: 800] Grizzly Framework 2.3.25 started in: 26ms - bound to [/0.0.0.0:8080]

[2016-12-05T23:40:54.640+0900] [Payara Micro 4.1] [INFO] [NCLS-CORE-00058] [javax.enterprise.system.core] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254640] [levelValue: 800] Network listener https-listener on port 8443 disabled per domain.xml

[2016-12-05T23:40:54.752+0900] [Payara Micro 4.1] [INFO] [SEC-SVCS-00100] [javax.enterprise.security.services] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254752] [levelValue: 800] Authorization Service has successfully initialized.

[2016-12-05T23:40:54.805+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.instance.DefaultAddressPicker] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254805] [levelValue: 800] [LOCAL] [development] [3.6.4] Prefer IPv4 stack is true.

[2016-12-05T23:40:54.809+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.instance.DefaultAddressPicker] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254809] [levelValue: 800] [LOCAL] [development] [3.6.4] Picked Address[192.168.184.89]:5901, using socket ServerSocket[addr=/0.0.0.0,localport=5901], bind any local is true

[2016-12-05T23:40:54.819+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254819] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Hazelcast 3.6.4 (20160701 - 5b94d9f) starting at Address[192.168.184.89]:5901

[2016-12-05T23:40:54.820+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254820] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.

[2016-12-05T23:40:54.820+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.system] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254820] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Configured Hazelcast Serialization version : 1

[2016-12-05T23:40:54.961+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.spi.OperationService] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254961] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Backpressure is disabled

[2016-12-05T23:40:54.978+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.spi.impl.operationexecutor.classic.ClassicOperationExecutor] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443254978] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Starting with 2 generic operation threads and 2 partition operation threads.

[2016-12-05T23:40:55.338+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.instance.Node] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443255338] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Creating MulticastJoiner

[2016-12-05T23:40:55.341+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.core.LifecycleService] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443255341] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Address[192.168.184.89]:5901 is STARTING

[2016-12-05T23:40:55.413+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.nio.tcp.nonblocking.NonBlockingIOThreadingModel] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443255413] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads

[2016-12-05T23:40:55.562+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.cluster.impl.MulticastJoiner] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443255562] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Trying to join to discovered node: Address[192.168.184.67]:5900

[2016-12-05T23:40:55.634+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.nio.tcp.InitConnectionTask] [tid: _ThreadID=58 _ThreadName=cached2] [timeMillis: 1478443255634] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Connecting to /192.168.184.67:5900, timeout: 0, bind-any: true

[2016-12-05T23:40:55.641+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.nio.tcp.TcpIpConnectionManager] [tid: _ThreadID=58 _ThreadName=cached2] [timeMillis: 1478443255641] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Established socket connection between /192.168.184.89:47714 and /192.168.184.67:5900

[2016-12-05T23:41:02.583+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.cluster.ClusterService] [tid: _ThreadID=38 _ThreadName=hz._hzInstance_1_development.generic-operation.thread-0] [timeMillis: 1478443262583] [levelValue: 800] [[
  [192.168.184.89]:5901 [development] [3.6.4] 

Members [2] {
        Member [192.168.184.67]:5900
        Member [192.168.184.89]:5901 this
}
]]

[2016-12-05T23:41:04.602+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.jmx.ManagementService] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264602] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Hazelcast JMX agent enabled.

[2016-12-05T23:41:04.730+0900] [Payara Micro 4.1] [INFO] [] [com.hazelcast.core.LifecycleService] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264730] [levelValue: 800] [192.168.184.89]:5901 [development] [3.6.4] Address[192.168.184.89]:5901 is STARTED

[2016-12-05T23:41:04.735+0900] [Payara Micro 4.1] [INFO] [] [fish.payara.nucleus.eventbus.EventBus] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264735] [levelValue: 800] Payara Clustered Event Bus Enabled

[2016-12-05T23:41:04.736+0900] [Payara Micro 4.1] [INFO] [] [fish.payara.nucleus.store.ClusteredStore] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264736] [levelValue: 800] Payara Clustered Store Service Enabled

[2016-12-05T23:41:04.738+0900] [Payara Micro 4.1] [INFO] [] [fish.payara.nucleus.exec.ClusterExecutionService] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264738] [levelValue: 800] Payara Clustered Exector Service Enabled

[2016-12-05T23:41:04.738+0900] [Payara Micro 4.1] [INFO] [] [fish.payara.nucleus.store.ClusteredStore] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1478443264738] [levelValue: 800] Payara Cluster Service Enabled

Hazelcast クラスタへ参加したことを確定付けているのは、上記ログのうち以下の部分です。

Members [2] {
        Member [192.168.184.67]:5900
        Member [192.168.184.89]:5901 this
}

ちなみに、このでは 2 つ目のノードがクラスタに参加したため、Member には 2 つ表示されています。

4. 説明しなかったこと

さらなるインスタンスの追加や削除については、スペースの都合で割愛しましたが、ノード間のネットワーク接続が正常で、かつ、ここまでの手順に問題が発生していなければ正しく行われるはずです。結果は必ず Hazelcast のログである Members [n] { (ノード一覧) } で簡単に確認できるため、最低限この部分のチェックだけは怠らないようにしてください。

5. Payara 4.1.1.164 追補

Payara 4.1.1.164 では管理コンソールに Hazelcast のクラスタ・メンバを一覧できる画面が追加されました。合わせて、Hazelcast 構成タブの位置も移動になっています。

hazelcast-cluster-members.png
figure 5.1 - Hazelcast クラスタ・メンバ一覧

hazelcast-configuration.png
figure 5.2 - Hazelcast 構成