CEPH环境搭建06 – Neohope的网络笔记

本节主要是使用ganesha将cephfs和rgw，映射为nfs，从而提供给其他系统使用。

0、原本计划使用容器继续处理，但没能成功，如果你成功了麻烦告诉我一下如何做的。

#这条路没测试成功，原理上应该是一样的
ceph orch apply nfs mynfs nfspool --placement="1 ceph-0003"

后面是通过编译源码的方式来完成的，操作系统是Ubuntu18.0.4，由于没有高版本的librgw的开发库（Ubuntu18.0.4只有1.1.6，而ganesha却要1.1.7以上）,所以最后没能支持rgw，只能支持cephfs：

1、安装所需包

#ceph-0003
apt-get install gcc git cmake autoconf libtool bison flex
apt-get install libssl-dev  libnfs-dev doxygen libgssglue-dev libkrb5-dev liburcu-dev libntirpc-dev
apt-get install libnfsidmap-dev uuid-dev libblkid-dev
apt-get install librados-dev librgw-dev libcephfs-dev nfs-ganesha-ceph
snap install cmake  --classic
#缺少libzardfs_client,webclient，暂时没有影响

2、下载源码

#ceph-0003
git clone https://github.com/nfs-ganesha/nfs-ganesha.git
cd nfs-ganesha
git checkout V3.2
git submodule update --init --recursive

3、编译

#ceph-0003
mkdir build
cd build

#这里要把rgw的自持去掉，否则无法通过
ccmake ../src
>config
>generate

make
make install

4、创建ganesha配置文件

# ceph-0003
# 配置文件是支持cephfs和rgw的
vi /etc/ganesha/ganesha.conf
NFS_CORE_PARAM {
        Enable_NLM = false;
        Enable_RQUOTA = false;
        Protocols = 4;
        NFS_Port = 2049;
}

NFSv4 {
        Delegations = false;
        Minor_Versions = 1, 2;
}

CACHEINODE {
        Dir_Chunk = 0;
        NParts = 1;
        Cache_Size = 1;
}

EXPORT_DEFAULTS {
        Attr_Expiration_Time = 0;
}

EXPORT {
    Export_ID = 1001;
    Protocols = 3,4;
    Path = /;
    Pseudo = /cephfs;
    Access_Type = RW;
    #Transport_Protocols = TCP;
    Squash = Root_squash;
    FSAL {
        Name = CEPH;
    }
}

EXPORT
{
    Export_ID=1002;
    NFS_Protocols = 3,4;
    Path = "/";
    Pseudo = /rgw;
    Access_Type = RW;
    #Transport_Protocols = TCP;
    Squash = Root_squash;

    FSAL {
        Name = RGW;
        User_Id = "s3user";
        Access_Key_Id ="J5IWQUN50UJHVH48461Q";
        Secret_Access_Key = "5fgLtqd07EI6LS3PamPVNBpqwPPfPYJNxULBlgpj";
    }
}

LOG {
    Facility {
        name = FILE;
        destination = "/var/log/ganesha/ganesha.log";
        enable = active;
    }
}

CEPH {
    ceph_conf = '/etc/ceph/ceph.conf';
    cluster = "ceph";
    name = "rgw.myrealm.myzone";
}

RGW {
    ceph_conf = "/etc/ceph/ceph.conf";
}

5、运行ganesha

ganesha.nfsd -f /etc/ganesha/ganesha.conf -L /var/log/ganesha/ganesha.log -N NIV_INFO

6、挂载

#ceph-0004
mount -t nfs4 ceph-0003:/cephfs  /mnt/nfs/

ls /mnt/nfs
fuse.txt  volumes

cat /mnt/nfs/fuse.txt
fuse

ls /mnt/nfs/volumes/
subvg01  subvg02

7、取消挂载

#ceph-0004
umount /mnt/nfs

8、补充：
前面傻了，只用了ubuntu的源，忘记使用ceph的源了。其实用ceph的源后，就可以解决问题了，操作和上面相同，编译方式如下：

#添加ceph源
wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
sudo apt-add-repository 'deb https://download.ceph.com/debian-octopus/ bionic main'
sudo apt-get update

#安装需要的软件
apt-get install gcc git cmake autoconf libtool bison flex
apt-get install libssl-dev  libnfs-dev doxygen libgssglue-dev libkrb5-dev liburcu-dev libntirpc-dev
apt-get install libnfsidmap-dev uuid-dev libblkid-dev
apt-get install librados-dev librgw-dev libcephfs-dev nfs-ganesha-ceph
snap install cmake  --classic
#缺少libzardfs_client,webclient，暂时没有影响

#下载源码并编译
git clone https://github.com/nfs-ganesha/nfs-ganesha.git
git checkout V3.2
git submodule update --init --recursive

mkdir build
cd build
ccmake ../src

make
make install

Leave a Reply Cancel reply