Ceph Octopus : Configure ISCSI Initiator For Vmware Esxi 7 With Cephadm

4 min readJan 27, 2021

Traditionally, block-level access to a Ceph storage cluster has been limited to QEMU and librbd, which is a key enabler for adoption within OpenStack environments. Starting with the Ceph Luminous release, block-level access is expanding to offer standard iSCSI support allowing wider platform usage, and potentially opening new use cases.

Prepare the image VMWARE_DISK1

Create rbd pool with PG autoscaling with name myrbd

[root@ceph1 ~]# ceph osd pool create myrbd
[root@ceph1 ~]# ceph osd pool application enable myrbd rbd

Create the image VMWARE_DISK1 in myrbd pool

[root@ceph1 ~]# rbd --pool myrbd create VMWARE_DISK1 --size=100G

Deploy the iSCSI gateways

Deploy the iSCSI gateways in ceph2 & ceph3

[root@ceph1 ~]# ceph orch apply iscsi myrbd admin admin01 --placement ceph2,ceph3 --trusted_ip_list 10.9.0.24,10.9.0.25

Check deployment

[root@ceph1 ~]# ceph orch ls
NAME                       RUNNING  REFRESHED  AGE  PLACEMENT         
alertmanager                   1/1  100s ago   6h   count:1           
crash                          3/3  103s ago   6h   *                 
grafana                        1/1  100s ago   6h   count:1           
iscsi.iscsi                    2/2  103s ago   2m   ceph2;ceph3       
mgr                            2/2  102s ago   6h   count:2           
mon                            3/3  103s ago   4h   ceph1;ceph2;ceph3 
node-exporter                  3/3  103s ago   6h   *                 
osd.all-available-devices      9/9  103s ago   3h   *                 
prometheus                     1/1  100s ago   6h   count:1

Configure iSCSI TARGETS

In ceph2 sereve, find iscsi contener

[root@ceph2 ~]# cephadm ls --no-detail | jq '.[].name'
"crash.ceph2"
"mgr.ceph2.dqggxs"
"node-exporter.ceph2"
"mon.ceph2"
"osd.1"
"osd.4"
"osd.7"
"iscsi.iscsi.ceph2.kovryz"

Connect to the iscsi container with cephadm

[root@ceph2 ~]# cephadm enter -n  iscsi.iscsi.ceph2.kovryz
Inferring fsid 78c597f2-6080-11eb-b0a3-8ee8efbd56e1
[ceph: root@ceph2 /]#

Check iscsi gateway

[ceph: root@ceph2 /]# gwcli ls

Go to iscsi-targets and create a target with the name iqn.2021–01.lan.mandem.iscsi-gw:iscsi-igw

[ceph: root@ceph2 /]# gwcli   
/> cd /iscsi-targets 
/iscsi-targets> create iqn.2021-01.lan.mandem.iscsi-gw:iscsi-igw
ok

Create the iSCSI gateways

/iscsi-targets> cd iqn.2021-01.lan.mandem.iscsi-gw:iscsi-igw/gateways/iscsi-target...-igw/gateways> create ceph2 10.9.0.24
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
ok
/iscsi-target...-igw/gateways> create ceph3 10.9.0.25
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
ok

Add the RBD image with the name VMWARE_DISK1 in the pool myrbd

/iscsi-target...-igw/gateways> cd /disks
/disks> attach myrbd/VMWARE_DISK1

Create a client with the initiator name iqn.2021-01.lan.mandem:rh8-client

/disks> cd /iscsi-targets/iqn.2021-01.lan.mandem.iscsi-gw:iscsi-igw/hosts
/iscsi-target...csi-igw/hosts> create iqn.2021-01.lan.mandem:rh8-client

Set the client’s CHAP

/iscsi-target...em:rh8-client> auth username=iscsiadmin password=iscsiadmin01

Add the disk to the client

/iscsi-target...em:rh8-client> disk add myrbd/VMWARE_DISK1

iSCSI Discovery and Multipath Device Setup

Enable Software iSCSI

Click on “Storage” from “Navigator”, and select the “Adapters” tab. From there right click “Software iSCSI”.

2. Configure ISCSI

iSCSI enabled : Enabled
Alias : iqn.2021–01.lan.mandem:rh8-client
CHAP authentication
* Name : iscsiadmin
* Secret: iscsiadmin01
Dynamic targets
* Address : 10.9.0.24
* Port : 3260

Save Configuration

3. Right click “Configure iSCSI”.

Dynamic targets : added the second gateway 10.9.0.15/3260

Save Configuration

4 . Change initiator name

change initiator name in the “Name & alias” section is not the same name used when creating the client during gwcli setup

[root@esx7:~] esxcli iscsi adapter list
Adapter  Driver     State   UID            Description
-------  ---------  ------  -------------  -----------
vmhba65  iscsi_vmk  online  iscsi.vmhba65  iSCSI Software Adapter[root@esx7:~] esxcli iscsi adapter set -A vmhba65 -n iqn.2021-01.lan.mandem:rh8-client