将设为首页浏览此站
开启辅助访问 天气与日历 收藏本站联系我们切换到窄版

易陆发现论坛

 找回密码
 开始注册
查看: 276|回复: 2
收起左侧

ceph luminous 提供iscsi服务

[复制链接]
发表于 2021-7-10 17:41:02 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有帐号?开始注册

x
开始安装之前必须要满足以下条件:
& N* ~) X! y8 }0 }正在运行的Ceph Luminous版本集群或更高版本的存储群集;(本文我们采用的是M版本). }' W+ Y2 P7 ~# Z6 E
RHEL/CentOS 7.5或者Linux kernel v4.16或更新;2 a- i# U" ]/ ^, ~/ H
同时拥有以下几个包:
1 U- B' q( Y' w- btargetcli-2.1.fb47 or      newer package% t% M! S9 u+ o5 h9 m
python-rtslib-2.1.fb64 or      newer package
6 q  p. F+ n* t/ [- r" L* ztcmu-runner-1.3.0 or      newer package
4 X0 R; ^1 `7 T% [) d; Uceph-iscsi-config-2.4 or      newer package
3 Z3 H, Y) [3 z3 mceph-iscsi-cli-2.5 or      newer package
4 t0 `9 h' f- k* c6 H注意:我们本次安装ISCSI GW是在OSD 节点上安装
/ y4 @3 c! u0 D( o由于很多童鞋可能获取不到rpm包,所以我这里整理了一下所需要的rpm并且分享了出来,以下是链接" W) [/ d7 u' u! h& \9 ]; B
链接: https://pan.baidu.com/s/1piNymp6oWLNHvXKaxBszLg 密码: 4vag
, h' c8 N5 {; T安装部分就不说了只要把rpm包下载到本地,安装很简单,下面来说下配置部分:6 @# k8 N9 H2 y3 o5 a! t# L7 K
首先创建一个iscsi gw的配置文件
2 G8 r, ~5 \) l7 a! f# touch /etc/ceph/iscsi-gateway.cfg' A0 S  y! Z; f6 b/ f1 x6 a! Z
里面内容如下:: _- b5 s2 ~- Y+ k8 A- D
[config]
) A, Y0 }2 ~# v# Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
! n' S4 M6 R( \, o# access to the Ceph storage cluster from the gateway node is required, if not! Q8 s% z( O% X
# colocated on an OSD node.
& J4 U7 w- {5 G% g0 ?- Qcluster_name = ceph
( O# O$ Y; h5 N! y4 N$ e$ u8 r( Y0 a % U" q) W1 e; b) H" U  Y: t
# Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
2 l/ c1 C: A: M7 e! ~# drectory and reference the filename here
; Z0 s. ~9 S  ~7 ugateway_keyring = ceph.client.admin.keyring. V  Y/ h2 I4 n  L$ H& Y' p% V

: R+ v) g& X. G& Z 7 F+ C6 U$ ^% \( V' O
# API settings.
" |2 X2 d" ]% x- h6 j9 A# The API supports a number of options that allow you to tailor it to your
# N+ N6 ~$ l! {6 t& u# w# local environment. If you want to run the API under https, you will need to. W5 R, R5 I1 R3 `1 ?) _
# create cert/key files that are compatible for each iSCSI gateway node, that is
* a3 a- R6 m& n) B6 n# not locked to a specific node. SSL cert and key files *must* be called, T1 E9 R& M/ |( \! c6 D8 t
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
7 C4 _4 i' H+ h$ W4 h# on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
# f# F$ _2 R7 `3 w/ h# to switch to https mode.
: ?# ~3 N3 z$ \0 z/ g; x- u8 s4 e
2 l8 J4 w. h  Z2 ]4 \/ V# To support the API, the bear minimum settings are:  O- v% |, ?" P; s
api_secure = false
6 {& R: W# ~2 g3 m
+ C+ D: Y$ W4 M. N1 F1 ?4 k# Additional API configuration options are as follows, defaults shown.4 _% t# H" x5 f- K5 V1 p" F
# api_user = admin
7 p. p. M) x& _# api_password = admin
9 @% u. o2 B6 _+ w, a# api_port = 5001
( f2 I6 ?8 A' ]. o( I( l5 y# trusted_ip_list = 192.168.0.10,192.168.0.11
% O  w- m9 N9 L) K3 w4 ]; T注意:需要修改的地方就行把trusted_ip_list替换成自己实际场景的节点ip
' J( H' h+ {( D4 f完事之后重启iscsi服务
8 H3 V/ I- g. m+ `2 U6 D) J# systemctl daemon-reload- M- k% Z: [$ _# ~% w
# systemctl enable rbd-target-api' _9 \6 ~6 r4 k. u% d; o$ H
# systemctl start rbd-target-api2 F6 A9 z( x# X& {' ~
OK到此为止,iscsi gw已经安装配置完成,下面开始创建iscsi target8 l5 w: j3 c. {; g4 t* O% K/ k
! e! M) g% e* m/ n5 X
1.执行gwcli命令进入iscsi
1 e% E0 {, h. }- `6 S& X8 r  y
& b  }' M' o- N* ^3 c2 ]! K2.创建iscsi target
  i2 ?0 a" E7 ]9 f> /> cd /iscsi-target
$ P! b, B  M7 f/ v) W4 I0 i) j> /iscsi-target>  create iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
  k0 k0 [. m' G2 j. M : W' D% p) r# ^+ [, F; g
3.创建iSCSI网关。以下使用的IP是用于iSCSI数据传输的IP,它们可以与trusted_ip_list中列出的用于管理操作的IP相同,也可以不同,看有没有做多网卡分离
3 @' l+ S0 r7 O" _4 u> /iscsi-target> cd iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/gateways
  C9 j( g5 {3 K: ?9 t) X# a/ E> /iscsi-target...-igw/gateways>  create ceph-gw-1 10.172.19.21 skipchecks=true3 B0 z/ @+ b  g1 c. r% W
> /iscsi-target...-igw/gateways>  create ceph-gw-2 10.172.19.22 skipchecks=true
. @  L9 L& R: t& U! m+ v 8 m0 n2 h$ m# q1 H" E3 Z
4.创建一个rbd设备disk_1
  H6 j* ~: |' d: U> /iscsi-target...-igw/gateways> cd /disks8 f6 {% p3 a+ L& f
> /disks> create pool=rbd image=disk_1 size=90G  K3 m, O4 R$ ]3 H

1 w" M* R3 P2 M5.创建iqn, m+ V2 u4 z; H
> /disks> cd /iscsi-target/iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/hosts
# m# I; d9 ^/ M2 f1 N: O! G9 I> /iscsi-target...eph-igw/hosts>  create iqn.1994-05.com.redhat:rh7-client' W$ s0 Z4 g/ D  q# v& {, T
' f: D( Z" P& I  D' O
6.创建chap的用户名密码,并且chap必须设置,否则服务端是禁止连接的
9 d6 k( C: c1 T( U- M7 u> /iscsi-target...at:rh7-client>  auth chap=myiscsiusername/myiscsipassword
* Q- H: j3 f! z2 T0 u
( J' S- D4 ^# M7.增加磁盘到客户端
0 L  V; P1 K4 k* ~" K& O> /iscsi-target...at:rh7-client> disk add rbd.disk_1* r. a- D: _1 D% n
$ U$ v8 m, @  H6 a5 v: W
OK,到这里就配置完成了,我们看下最终是什么样的
' r: U8 q! z1 t  U- riscsi.png2 G3 {- J: L) U! l: M

8 ?) E8 t5 f5 d安装配置完之后就可以挂载使用了,一般也就是windows/Linux,我们先说下Linux下挂载方式
1 @8 K" {$ p/ a7 i0 O4 k# M) u- B7 S1 c$ ~6 P9 d: H
1.安装iscsi工具
6 ^0 I# B; u1 e# yum install iscsi-initiator-utils$ Y( H, I7 _: x( d
# yum install device-mapper-multipath6 V: E. K' r8 w% e) p0 J# \# g

3 p6 a0 }# i5 i* C! q  T; J2.配置多路径( A/ R3 F3 R1 w* L4 W$ ?. _9 I: P
# mpathconf --enable --with_multipathd y
7 ]5 b/ t+ l- tdevices {
' r; Y3 f: A& l% S# F        device {  o8 B# G9 E$ `" I( k' h4 E* ^$ x2 X
                vendor                 "LIO-ORG"
5 w2 }! h1 y/ C$ ~                hardware_handler       "1 alua"
* c8 |. N6 g* y; b  u                path_grouping_policy   "failover"7 V8 y' e5 v& [2 l
                path_selector          "queue-length 0"
: x7 J+ C3 k8 g0 u) f" R                failback               60- j6 V/ O! \, Y) j' F8 p8 n
                path_checker           tur' o3 f- s4 b* U0 W
                prio                   alua; G) |8 r; q9 ]8 Z! A$ t
                prio_args              exclusive_pref_bit
  l! V. _! c4 g9 J, G& G2 T8 d4 Q                fast_io_fail_tmo       25
4 w" k" C2 v2 N$ z# w                no_path_retry          queue' ~- o  [5 f* n% r7 {
        }) Y& ~: W* i6 {, V: i, o8 w
}9 [) |7 d4 H5 }/ U& N- T9 B. e
' l3 ^, @: ]6 G) X
3.重启服务
/ ^+ l2 ?! {* P5 ]3 o4 f) B+ I) E# systemctl reload multipathd
0 p1 p. ~8 F# z 4 \1 W  x4 l9 R) {. P
4.配置chap认证
. M) }+ e: E8 j/ p+ b, X修改配置客户端的名称为之前设置的名称
3 `3 o1 a. {! Y' J7 [) w# vi /etc/iscsi/initiatorname.iscsi
& m. `, i# Y* s InitiatorName= iqn.1994-05.com.redhat:rh7-client( v0 z7 t0 v! X2 D- ]
修改chap认证配置文件
$ m1 k- v1 \1 H5 x8 c) c+ q( b8 y # vi /etc/iscsi/iscsid.conf2 E/ ~- O$ v& ^# u4 w; i
node.session.auth.authmethod = CHAP* @0 x2 ~, B  D5 F
node.session.auth.username = myiscsiusername
5 l& E/ f9 K. I6 F node.session.auth.password = myiscsipassword  m1 `2 ^# Y1 _0 b0 j3 x

( g6 {  A4 k1 b5.查询iscsi target
% F0 h1 D- Q( j7 e) h( u3 R# iscsiadm -m discovery -t -st 10.172.19.21" ~! t" t0 J2 @5 R) n
5 N5 z5 ~2 M7 ]: ]
6.连接target* h. `) s* U0 a, W3 _% P
# iscsiadm -m node -T iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw -l
1 x$ r: E9 v8 @1 [# multipath –ll
0 Z- L8 \; j5 w$ Y/ V  S+ u9 R
) D4 E4 ?/ ]. h3 C
 楼主| 发表于 2021-7-10 21:21:50 | 显示全部楼层

配置yum文件ceph-iscsi

[root@ceph01 ~]# vi /etc/yum.repos.d/CentOS-Base.repo + U( [; K$ d5 d7 S- r
2 s9 m2 q& @% @! B) d! l
[Storage]

2 i! n# J8 `5 z: z! j
name=CentOS-$releasever - Storage$ k/ a2 D# y3 N% D% w
baseurl=https://mirrors.huaweicloud.com/centos/7.7.1908/storage/x86_64
/ceph-nautilus/
# I/ e# t- v; I% e, e
gpgcheck=0

3 V( ^8 o4 i% e: H! a
enabled=1
8 e* }7 `7 d' p3 ?1 {1 A1 `
# T1 W( C; D$ d( d* y- h
[Gluster]
1 n3 l  b7 g) z4 `$ l
name=CentOS-$releasever - Storage
- r7 y& d" g$ P$ w# l9 ]$ R
baseurl=https://mirrors.huaweicloud.com/centos/7.7.1908/storage/x86_64/gluster-5
/. o. u$ b4 v4 P3 [
gpgcheck=0
& m9 D) q# Z  ~/ i7 H% T
enabled=1
) a; \7 G" e) D9 x3 v. m
) p. Y1 D$ R7 F& `2 x
[ceph-iscsi]

4 y" |  k% F3 S0 t$ j8 t; L
name=CentOS-$releasever - Storage
$ K9 v; n; l, U! Z
baseurl=https://mirrors.huaweicloud.com/ceph/ceph-iscsi/3
/rpm/el7/noarch/
: r6 F  L: Z9 w8 r  q
gpgcheck=0

; R9 }4 h% _/ c( m% }+ d$ g
enabled=1
( E2 z/ m* }, b& t, X; c

+ D  f- z' Z, t, i6 t- a[epel]

. i/ ^* V) N& t1 `, o0 X
name=CentOS-$releasever - Storage. o! w, h4 P& n* \6 ]6 u
baseurl=https://mirrors.huaweicloud.com/epel/7Server/x86_64
/! N$ E% b/ a7 ~8 `6 \
gpgcheck=0

6 l/ {# Y1 ^6 U! b
enabled=1
7 Q! o% D4 C- F( R9 L6 i% u0 d5 Z
 楼主| 发表于 2021-7-10 21:37:05 | 显示全部楼层
直接yum install试试可以发现,只有targetcli和python-rtslib能装上,而且版本都比官网说的要低,好吧,麻烦来了。& c" t7 m4 Y) j- V. G7 {( T3 G
& [" A5 j( K" @# ^5 F
经过一段几个小时*的搜索……从下面链接中找到了rpm包:8 e* T% l5 l  E" ~2 y
/ h4 [. I- ]% [  c8 q
Build new RPM for 3.0
; G) t) V8 a' \  LMissing ceph-iscsi-cli package' g8 S+ e- e" [  q. |8 d
新建repo文件:! p/ f' t3 R$ b$ Y5 W* S
  \# d. e2 |7 N# J4 ]9 ^' B
sudo vim /etc/yum.repo.d/iscsi.repo0 P' a/ S/ k  e4 p0 l0 S$ V2 @! U8 n) Z
cat /etc/yum.repo.d/iscsi.repo
2 e/ }' a( @2 i9 f! t1 ^, |0 G, s8 c  P: U3 T# d  Z8 D" B: s
[ceph-iscsi]
* c+ b7 J+ V- o5 i8 r; dname=Ceph-iscsi$ V# A0 X2 F- m, ?! N; u3 e
baseurl=https://4.chacra.ceph.com/r/ceph-iscsi/master/88f3f67981c7da15448f140f711a1a8413d450b0/centos/7/flavors/default/noarch/5 i8 f8 H7 j' M- K* ~3 a* ]+ W
priority=1
' x( h+ H# A& Y: _+ O+ t( ?gpgcheck=0
$ p7 q# _6 X9 w( K# q, x( O! q. O# r0 x# E" e1 d% T2 X1 U; y/ p- ?& g
[tcmu-runner]
) H( K$ j; m. q! V: nname=tcmu-runner% C2 p5 Y1 J. M; S7 V0 R
baseurl=https://3.chacra.ceph.com/r/tcmu-runner/master/eef511565078fb4e2ed52caaff16e6c7e75ed6c3/centos/7/flavors/default/x86_64/
+ F5 ]$ r6 q& a3 }* ~% e+ Rpriority=1
: t3 g# d: Q5 j6 Y* Agpgcheck=0
+ Q) q- g5 n4 \, S% p* M" o. z0 z
$ Q; w' x; o% D$ C! P[python-rtslib]
4 q- q9 ]5 j" g% hname=python-rtslib+ U' a4 Y7 ]0 i4 a5 T$ w
baseurl=https://2.chacra.ceph.com/r/python-rtslib/master/67eb1605c697b6307d8083b2962f5170db13d306/centos/7/flavors/default/noarch/
+ `0 ~' E  `8 b/ R( spriority=13 L( i' o8 t$ X
gpgcheck=0
" m& x) n! J/ x5 [! n1 P% c; m这里我使用的是本地源,将上面的包下载到本地源:$ C3 D. H. q0 n! M

4 |/ K) j5 Z; _8 h# L+ M5 P7 t sudo yum install --downloadonly --downloaddir=yum/ceph-iscsi/ targetcli python-rtslib tcmu-runner ceph-iscsi
1 x2 v) h/ n1 H0 n createrepo -p -d -o yum/ yum/% U: G6 L- |' d
注意到这里没有包含targetcli的repo,因为没有找到,使用yum基础的Base源或者是Ceph源可以安装targetcli-2.1.fb46-7.el7.noarch.rpm, 虽然官网需要的是targetcli-2.1.fb47 or newer package,但在后续使用中发现没有影响,所以这里就不用管targetcli了。  O. S7 v  F, s0 A6 C8 Y

  E/ y+ N6 S! _& ?/ @& j" W& _% g那么这里下载下来的就是:
& n. a' A  o6 O! o& Q0 P
3 y3 m. p6 w+ ztargetcli-2.1.fb46-7.el7.noarch.rpm
) j3 A. f- ?9 l( I# J0 Xpython-rtslib-2.1.fb68-1.noarch.rpm
5 t. k0 a. ?2 d( [tcmu-runner-1.4.0-0.1.51.geef5115.el7.x86_64.rpm
' S0 y: T6 A! L" e) P  ?* n9 yceph-iscsi-3.2-8.g88f3f67.el7.noarch.rpm% S9 n/ M  P' q# P/ K
ceph-iscsi网关初始配置1 \& m9 H0 C  Q- b. x+ n
如果使用的不是集群内的节点作为ceph-iscsi网关,那就需要进行一些初始的配置。
- @7 r0 s, d- S1 W0 u& b! i
0 S% [; f+ i# L* h2 a# m) z安装ceph。" f6 g' }7 U" j0 B& y$ D
从集群中的一台机器上拷贝/etc/ceph/ceph.conf到本机的/etc/ceph/ceph.conf。4 e+ O# ]4 L; C9 b" k- b2 A
从集群中的一台机器上拷贝/etc/ceph/ceph.client.admin.keyring到本机的/etc/ceph/ceph.client.admin.keyring。
5 [6 H' V8 [* e& Z当然第2和3步可以直接在deploy节点使用ceph-deploy admin {node-gateway},{node-gateway}就表示网关节点的名字。8 O* A/ ~4 v9 \% w1 {' }0 G& [1 M

! |. Y$ h  S: E  u% @可见这里就是为了将ceph-iscsi网关节点变成一个admin节点。
+ \7 v9 g1 m9 M1 N$ ~. `* }4 e6 v
8 n4 l+ Z/ Z& C: S( U这时在网关节点上应该可以执行相关命令操作ceph集群,例如sudo ceph -s查询当前集群的状态。
+ a) T1 P4 s- b: \3 ]( m2 j4 B' g7 O: u7 T/ I" _2 b
安装配置iscsi
8 ^  S, v; G7 `) ~( q这里官网建议先切换到root用户,方便一点:2 L5 Y% `. g) d$ q) p
( u5 H( f7 f1 x+ \" K
su root
6 E2 u; \: O' [3 l在两个网关节点上都安装iscsi(注意到上面已经将相关包下载到了本地源,所以可以直接yum安装):: }$ [* N" d: X) D- T
9 n3 D% D. ^# x/ q( ?8 X. f1 \' B
yum install -y ceph-iscsi
' x% O+ f8 i& d. E服务启动:7 ]9 w" z+ S' G! r# p  O' ~

4 v6 c, b# |0 R先创建rbd pool,如果没有的话。3 g  c$ R/ ^+ M3 n

0 u$ O! t- ]$ |  e9 ?; Z( | ceph osd lspools' q3 I, w% j# z+ ~

+ U% W7 P" b0 J1 cephfs_data7 {2 }/ f7 h8 V
2 cephfs_metadata6 Y/ e9 b$ ?" b5 _# N

8 ^' y0 `% w8 h& I1 ]0 C  `: s; I* q+ k1 R ceph osd pool create rbd 128
& X) R' L9 l; T( ?$ i( S+ I  b3 x8 I" N
pool 'rbd' created% Q; X  C* r3 w% S9 [5 B' I% Q1 I9 s# A
创建并修改/etc/ceph/iscsi-gateway.cfg文件:$ f' `6 B+ V- K" d5 e

  I/ j4 T+ [3 o; k; N vim /etc/ceph/iscsi-gateway.cfg
+ l+ `0 ?: t# t3 ~" a# Q/ ^ cat /etc/ceph/iscsi-gateway.cfg+ {* t" `' ]; U; I

& i7 r; i) R1 O. L8 ~[config]' f, ~! Q0 f6 A. J( c# Q
Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
( Y  C0 N1 G7 s4 O access to the Ceph storage cluster from the gateway node is required, if not, T) p$ v, ^) Z* T4 P6 l
colocated on an OSD node.
7 j; y8 V; e4 @! b! F! G5 _cluster_name = ceph3 I: X, E0 X' j3 K6 l2 D
; p2 m1 ~0 e) `; s$ Z4 H
Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
3 \4 J4 u( Z: J* O& j" u- Q drectory and reference the filename here$ K% {0 p$ C, D  F, U; U2 {# X" V
gateway_keyring = ceph.client.admin.keyring
5 b/ n) M8 ?; j( y0 }
) s# I0 y$ [9 X' m5 E# y* m: ^: x0 I6 d- l% I# }- ]
API settings.! h* k: R7 q+ D: J% Q4 ?
The API supports a number of options that allow you to tailor it to your
+ X  D5 V! ^0 f% c" L! O7 ], [( Q local environment. If you want to run the API under https, you will need to
( X% q4 G! f: u+ m create cert/key files that are compatible for each iSCSI gateway node, that is
) H8 Y6 _2 r2 [4 r- Z not locked to a specific node. SSL cert and key files *must* be called1 N! U' x$ [6 G8 ?# d: r* V
'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
! a& q# l  A; [ on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
* A3 j5 e! b& b0 A9 t4 T to switch to https mode.
; w9 D. `; ]6 e$ B% x, P3 v1 U1 G
8 S: V' `# a1 u+ u4 |) L6 p2 ^ To support the API, the bear minimum settings are:
) a+ m5 |' n- iapi_secure = false
9 {! a# j( I, h0 U0 C2 r. V' A0 G( g- |; I
Additional API configuration options are as follows, defaults shown.
* x& R* B% |# z; _+ L+ v api_user = admin# X; G( |( d  ~8 ~; R8 A. H
api_password = admin! ^- p/ b  ]- o& h, L
api_port = 5001; f  h! N* q; J# x" _: b3 G4 l
trusted_ip_list = 192.168.90.234,192.168.90.2354 X. l. {+ f8 S2 B- p  e
上面trusted_ip_list填写的就是两台网关的ip(这里不讨论多网卡的情况)。2 T$ F! L; Q5 c0 D3 i

! Z1 o& R' d* C4 @, b9 @( y6 g在另外一台网关上复制这个文件:* l' Z" @  x1 ~+ I- Q2 ~/ t
! }: I0 J3 ~! `3 C+ `7 ^
sudo scp cluster@node2:/etc/ceph/iscsi-gateway.cfg /etc/ceph/iscsi-gateway.cfg2 `) l+ G; J( |! E* H# U7 D8 v4 n+ t  A
在两台网关上都开启rbd-target-api服务:
, {( s( I3 F9 P* u
& N9 O: \$ @& b" Z systemctl daemon-reload
7 P9 L, K* l2 P5 o2 e+ G5 M systemctl enable rbd-target-api, u% R) Q& A+ a. f
systemctl start rbd-target-api
& H+ H" t1 T9 Z: ?( a配置:(在其中一台网关配置就行)+ {, U( w) _! B, k
. l: J; T" V) V  S; c
gwcli% R5 \! n" [3 A' {
cd /iscsi-targets0 f# G! h$ a2 w# D9 S4 c
3 f* |5 V( V  }5 s6 `# j! l
create iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
* ^& |! X( @* l( \+ i: Kok# y2 U! d8 c! v' I. o/ i) s
$ M7 x0 r. e- x, c
cd iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/gateways  Y- Q4 s( J, _7 y
- I4 m5 s* E* @( ]$ C
/iscsi-target...-igw/gateways> create node2 192.168.90.234
6 v! i# v' y! s3 nAdding gateway, sync'ing 0 disk(s) and 0 client(s)
1 M4 H+ X9 C0 q# Hok; b' u$ e# v1 C# R
- V8 D2 P9 C8 L( |
/iscsi-target...-igw/gateways> create node3 192.168.90.2359 C0 l& B% K) d! ^
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
5 s" b. U, y6 Vok; |  n9 s  U+ t7 P0 W. P

1 Z; d  N1 {( u: T- e/iscsi-target...-igw/gateways> cd /disks8 K4 Z* Z3 D7 m5 d

9 S4 \( F, w. d4 Y* _1 ?5 o  { create pool=rbd image=disk_1 size=200G2 b- v8 I* H8 ~' S
ok/ Z( F- T! w6 e& M0 D

- Z# Q; l. f$ V' v9 a; w3 ` cd /iscsi-targets/iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/hosts  t! d0 B& ^2 |9 A3 {3 A. S
6 S0 |+ E1 P6 W5 P2 r" J. U: p, G
/iscsi-target...csi-igw/hosts> create iqn.1994-05.com.redhat:rh7-client
1 G  x" h; S) dok
: R7 g, G4 J/ y1 I$ F8 A5 ?
' m% M8 i  \0 O! s1 P/iscsi-target...at:rh7-client> auth username=myiscsiusername password=myiscsipassword
" a9 D/ a5 E& T: D( H3 N. E( uok
, k: d$ N9 i2 n. j7 \9 {) D5 J1 Z$ F3 Q, Q5 C8 A( u* m4 e9 j
/iscsi-target...at:rh7-client> disk add rbd/disk_1
$ Z) _2 K* G# uok
) @/ A' K2 l! ]$ u( g" I配置完成,可以看到我当前的目录结构:
" L# D3 b) {& d6 H$ P% p7 N$ t' O, ]8 o. }
ls3 H% [; C, d: i5 a. m! b& C
o- / ......................................................................... [...]
) x1 @: _6 r* A; I8 d# v& a  G0 g% }  o- cluster ......................................................... [Clusters: 1]; X" A& H# }1 ~3 p
  | o- ceph .......................................................... [HEALTH_WARN]6 k8 Z: W8 {$ T" W
  |   o- pools .......................................................... [Pools: 3]
# j7 \; ~1 i% X1 O5 e7 V  |   | o- cephfs_data ... [(x3), Commit: 0.00Y/2028052096K (0%), Used: 2029431878b]
8 J1 Q& V+ Z* T  |   | o- cephfs_metadata .... [(x3), Commit: 0.00Y/2028052096K (0%), Used: 77834b]. ^' I+ U- I# S* r  t; G
  |   | o- rbd ................ [(x3), Commit: 200G/2028052096K (10%), Used: 15352b]
  q# `$ E0 Q. Q, p4 Q! c' }+ J: W  W; l  |   o- topology ................................................ [OSDs: 6,MONs: 3]: _* @# f0 S4 }7 W
  o- disks ........................................................ [200G, Disks: 1]
# X: V/ l3 w; b  | o- rbd ............................................................ [rbd (200G)]4 ]/ s, ^, W! \2 e
  |   o- disk_1 ................................................ [rbd/disk_1 (200G)]: S  v! k" R9 s8 c4 S8 p+ h% ?& T
  o- iscsi-targets ............................... [DiscoveryAuth: None, Targets: 1]
7 Q" @& G' \/ x- C    o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw ..................... [Gateways: 2]0 X3 b9 S( z" W; v; ^
      o- disks .......................................................... [Disks: 1]$ S  R/ h) t4 s" r# J1 j" Z4 M
      | o- rbd/disk_1 ............................................... [Owner: node3]0 S! T) m) Y. P) x( r
      o- gateways ............................................ [Up: 2/2, Portals: 2]2 J5 {* i+ N# H* z, ~9 d1 }
      | o- node2 ............................................. [192.168.90.234 (UP)]
: y# n9 x+ X3 s  S$ M8 s6 h      | o- node3 ............................................. [192.168.90.235 (UP)]8 K) i1 _; e2 }0 U" a
      o- host-groups .................................................. [Groups : 0]
) N0 ?' c/ N0 {$ K* a% H1 p      o- hosts .............................................. [Hosts: 1: Auth: CHAP]
  W) K7 c1 X5 W2 L1 x        o- iqn.1994-05.com.redhat:rh7-client .......... [Auth: CHAP, Disks: 1(200G)]
! t! w# y9 h( c" X0 p          o- lun 0 ................................ [rbd/disk_1(200G), Owner: node3]
; {1 _: t: I% W( ^: x% D/ nlinux客户端配置4 p7 Y& l( `4 H4 `
在作为客户端的linux主机上。
5 }6 v) b0 Z4 _* I+ Z) D4 C& I; i* f& _
安装相关组件:
4 l% w9 N0 I5 B
  v. g$ H0 g: N9 K sudo yum install -y iscsi-initiator-utils
6 X- C# m8 a+ Z/ W5 b sudo yum install -y device-mapper-multipath7 D. c6 _0 N" v. a3 ?& \
开启multipathd服务并进行配置:+ y& n  C, Y/ v; k% C# C
" D5 v7 q' [2 d! L  C7 R
sudo mpathconf --enable --with_multipathd y7 }5 R& x5 ?# [1 l& d6 G) C

" y( |1 a& \, m5 \  T sudo vim /etc/multipath.conf5 Z3 Y8 v4 p; [/ T& e
sudo cat /etc/multipath.conf% w  a$ L% V; b5 B5 X

6 x" {7 p( [' I0 Q* udevices {3 X, ?! X( S/ I* a
        device {
/ d# D. g4 @2 n4 X0 d) z                vendor                 "LIO-ORG"
) k, i( b4 [! |+ K" v2 l                hardware_handler       "1 alua"
7 S0 @' Y" q& v# [5 k                path_grouping_policy   "failover"
5 t8 |, B+ Q, S  Y8 ~/ j) ^                path_selector          "queue-length 0"
) O5 p: j; S* J! V$ n1 {                failback               604 N/ Y4 j( C* z+ S6 a2 U( f! w, D
                path_checker           tur
9 T2 y3 z0 Y3 R. t                prio                   alua) g* D; ?$ F! _: W7 [* X, W, h
                prio_args              exclusive_pref_bit
* A: X- o9 S$ p                fast_io_fail_tmo       25
; @: M4 B) Q( g5 _7 O; r                no_path_retry          queue1 s2 I0 A$ P2 Q5 I/ f
        }
+ I9 B+ H' f: ?$ p3 I0 d4 V! f! X" B( Q}1 X) L6 a) A6 y) L
修改客户端名称:
$ i' w. v% x0 P# _5 o- V& A! K: I, z8 ?( @5 {8 w
sudo vim /etc/iscsi/initiatorname.iscsi/ O5 G. ~) N3 _
sudo cat /etc/iscsi/initiatorname.iscsi
0 m5 P8 [) R8 B3 L' {# W- u4 D) k! O2 L7 G  Z$ O3 @- J1 y, u* a' L- v
InitiatorName=iqn.1994-05.com.redhat:rh7-client
% ~/ C, [4 r" V" x4 _修改chap认证配置文件:2 n9 p" s0 z, y& Y* c% g0 z$ S5 Z
4 ?( n" x  X0 `+ X
sudo vim /etc/iscsi/iscsid.conf. ^) |# ^( Q1 m' q, k- ^1 T
sudo cat /etc/iscsi/iscsid.conf5 {7 d' {7 D8 l5 S" Y+ v

% [+ W; h3 d  F. W...
% Z2 D/ ~% l* C. e$ L *************8 c  R- k1 U- Y
CHAP Settings1 T' P7 ]1 q# W2 k+ Z, H& l+ {
*************
5 m6 Q& \' G# s% D" R. ^5 R5 i. X/ @/ U; e5 E1 d
To enable CHAP authentication set node.session.auth.authmethod
' |9 e8 B$ a/ Q* P. l to CHAP. The default is None.
- A# L6 U% m% S2 m- vnode.session.auth.authmethod = CHAP( a& r8 e/ b0 b" f( d/ F" _

) A7 k' s! j# k  v To set a CHAP username and password for initiator( m8 J! J- _7 N  o4 G* M1 N6 R
authentication by the target(s), uncomment the following lines:! C4 D4 J2 T0 h' O; L
node.session.auth.username = myiscsiusername( u5 F2 s2 x3 u- r9 G+ S
node.session.auth.password = myiscsipassword
- `" a  A5 S+ C' S0 ^...+ A" F: H: x/ \: m

( ]5 f) X* G  {* F) i发现target:7 x+ A8 V8 N" X. L+ g7 F# l
4 y0 F* z7 Z: K  T
sudo iscsiadm -m discovery -t st -p 192.168.90.234
2 d2 D0 g& Z) q. X- z( t# O9 K3 W2 d; U; o& f. G# X
192.168.90.234:3260,1 iqn.2003-01.org.linux-iscsi.rheln1
. h; p7 R6 g; Q1 I$ h192.168.90.235:3260,2 iqn.2003-01.org.linux-iscsi.rheln1
: g: f; [1 y( R: o: u登入target:! K2 U( ^: G; W1 s. U

5 |3 G1 g) L5 A% ? sudo iscsiadm -m node -T iqn.2003-01.org.linux-iscsi.rheln1 -l3 A- S: k7 F1 ^/ v, y5 T
查看是否成功:  V( x$ L  m; F  W

/ s1 B' F; o, S1 [  v1 ~) a sudo multipath -ll8 B8 K/ L2 g& ]1 l: Y7 P
mpatha (360014050fedd563975249adb2e84e978) dm-2 LIO-ORG ,TCMU device     
, W' J( t* N, x' I- }- b2 ?7 Usize=200G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
+ l$ R7 G, g0 r) o0 {9 N6 ]5 B4 f4 b|-+- policy='queue-length 0' prio=50 status=active
# {' P2 x, t7 }& ~| `- 3:0:0:0 sdc 8:32 active ready running8 J/ ~2 U* h0 O. ^/ F
`-+- policy='queue-length 0' prio=10 status=enabled3 |3 ], S* L3 t( Y
  `- 2:0:0:0 sdb 8:16 active ready running
9 _9 n+ r; ?# Q在fdisk中就可以直接看到这个“硬盘”:
5 D+ C' d( J/ m# b3 m
; i- {- ^. ?/ n sudo fdisk -l
0 H& G6 o; ]. L0 m+ G+ q( T
1 T: V. L# {5 r5 v...
6 w/ b% Y, A% Y  E4 MDisk /dev/mapper/mpatha: 10.7 GB, 10737418240 bytes, 20971520 sectors
/ [, R' a/ n0 E  SUnits = sectors of 1 * 512 = 512 bytes
6 O8 S, p* J# }6 oSector size (logical/physical): 512 bytes / 512 bytes
4 V2 [% q* u& }; p+ U8 VI/O size (minimum/optimal): 512 bytes / 524288 bytes
您需要登录后才可以回帖 登录 | 开始注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

如有购买积分卡请联系497906712

QQ|返回首页|Archiver|手机版|小黑屋|易陆发现 点击这里给我发消息

GMT+8, 2021-9-27 03:32 , Processed in 0.047092 second(s), 22 queries .

Powered by LR.LINUX.cloud bbs168x X3.2 Licensed

© 2012-2022 Comsenz Inc.

快速回复 返回顶部 返回列表