Ceph. AlmaLinux 8. Ceph 17 Quincy. Ceph iSCSI-target create

Description

Ceph as an iscsi-target.

In this example we will deploy iscsi-target in ceph cluster on 2 nodes, but you can use as many nodes as you want.

Links

https://docs.ceph.com/en/latest/rbd/iscsi-target-cli/

https://github.com/ceph/ceph-iscsi

Configuring

Ceph01 node

Tcmu-runner installation

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/tcmu-runner-1.5.2-89.g245914c.el8.x86_64.rpm

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/libtcmu-1.5.2-89.g245914c.el8.x86_64.rpm

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/libtcmu-devel-1.5.2-89.g245914c.el8.x86_64.rpm

dnf localinstall -y ./*.rpm

Ceph-iscsi installation

cd /etc/yum.repos.d

wget https://download.ceph.com/ceph-iscsi/latest/rpm/el8/ceph-iscsi.repo

dnf install -y ceph-iscsi

Create pool iscsi01

ceph osd pool create iscsi01 32

Set mark at new pool iscsi01 as app "iscsi" (just a name)

ceph osd pool application enable iscsi01 iscsi

Config iscsi-gateway

nano /etc/ceph/iscsi-gateway.cfg

"cluster_name = ceph" - "ceph" is a default cluster name, don't change it

[config]

# Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
# access to the Ceph storage cluster from the gateway node is required, if not
# colocated on an OSD node.
cluster_name = ceph

# new created pool for iscsi -target images
pool = iscsi01

# Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
# directory and reference the filename here
gateway_keyring = ceph.client.admin.keyring

# API settings.
# The API supports a number of options that allow you to tailor it to your
# local environment. If you want to run the API under https, you will need to
# create cert/key files that are compatible for each iSCSI gateway node, that is
# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
# to switch to https mode.

# To support the API, the bare minimum settings are:
api_secure = false

# Additional API configuration options are as follows, defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# trusted_ip_list = 192.168.0.10,192.168.0.11

Start services

systemctl daemon-reload

systemctl enable --now rbd-target-gw

systemctl status rbd-target-gw

systemctl enable --now rbd-target-api

systemctl status rbd-target-api

Ceph02 node

All the same as at ceph01 node, except creating pool

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/tcmu-runner-1.5.2-89.g245914c.el8.x86_64.rpm

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/libtcmu-1.5.2-89.g245914c.el8.x86_64.rpm

wget https://4.chacra.ceph.com/r/tcmu-runner/main/245914c1446dddf07d5b58b0a7b2060b50fde4d7/centos/8/flavors/default/x86_64/libtcmu-devel-1.5.2-89.g245914c.el8.x86_64.rpm

dnf localinstall -y ./*.rpm

cd /etc/yum.repos.d

wget https://download.ceph.com/ceph-iscsi/latest/rpm/el8/ceph-iscsi.repo

dnf install -y ceph-iscsi

nano /etc/ceph/iscsi-gateway.cfg

[config]

# Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
# access to the Ceph storage cluster from the gateway node is required, if not
# colocated on an OSD node.
cluster_name = ceph

# new created pool for iscsi -target images
pool = iscsi01

# Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
# directory and reference the filename here
gateway_keyring = ceph.client.admin.keyring

# API settings.
# The API supports a number of options that allow you to tailor it to your
# local environment. If you want to run the API under https, you will need to
# create cert/key files that are compatible for each iSCSI gateway node, that is
# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
# to switch to https mode.

# To support the API, the bare minimum settings are:
api_secure = false

# Additional API configuration options are as follows, defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# trusted_ip_list = 192.168.0.10,192.168.0.11

systemctl daemon-reload

systemctl enable --now rbd-target-gw

systemctl status rbd-target-gw

systemctl enable --now rbd-target-api

systemctl status rbd-target-api

Ceph01 node

Enter shell for configuring iscsi-target

gwcli

/> cd /iscsi-targets

Create target

/iscsi-targets> create iqn.2023-03.ceph.com:iscsi-igw

    ...
    ok
    ...

Create gw for at least 2 ceph nodes. Or for any number of nodes you want.

ceph01, 192.168.199.121 - ceph cluster FIRST node IP and HOSTNAME

ceph02, 192.168.199.122 - ceph cluster SECOND node IP and HOSTNAME

/iscsi-targets> cd /iscsi-targets/iqn.2023-03.ceph.com:iscsi-igw/gateways

/iscsi-target...-igw/gateways> create ceph01 192.168.199.121 skipchecks=true

    ...
    OS version/package checks have been bypassed
    Adding gateway, sync'ing 0 disk(s) and 0 client(s)
    ok
    ...

/iscsi-target...-igw/gateways> create ceph02 192.168.199.122 skipchecks=true

Configuring disk

/iscsi-target...-igw/gateways> cd /disks

/disks> create pool=iscsi01 image=iscsi01-disk01 size=16G

    ...
    ok
    ...

/disks> cd /

Configuring account for remote iscsi-initiator, which will connect to this ceph iscsi-target

/> cd /iscsi-targets/iqn.2023-03.ceph.com:iscsi-igw/hosts

/iscsi-target...csi-igw/hosts> create iqn.2023-03.ceph.com:ceph-client

    ...
    ok
    ...

Configuring log/pass for remote initiator    

/> cd /iscsi-targets/iqn.2023-03.ceph.com:iscsi-igw/hosts/iqn.2023-03.ceph.com:ceph-client

/iscsi-target...m:ceph-client> auth username=ceph-client password=PassWord#1

    ...
    ok
    ...

Add earlier created disk for initiator

/iscsi-target...m:ceph-client> disk add iscsi01/iscsi01-disk01

    ...
    ok
    ...

Done!

Now we can attach iscsi-target "iqn.2023-03.ceph.com:iscsi-igw" to iscsi-initiator with login "ceph-client"

Log iscsi-target

tail /var/log/rbd-target-api/rbd-target-api.log