在 Ubuntu/Debian 安裝 NFS Ganesha 取代 NFS-kernel service

因為詢問 ChatGPT 是否有提升 NFS 服務效能的輔助軟體, 結果給了這答案:
使用 NFS 緩存 : 某些軟體和硬體方案提供 NFS 緩存，可減少對伺服器的存取，提升效能。這些緩存可以是軟體層面的，例如 nfs-ganesha 或硬體層面的存儲緩存。
也看到預設 Ubuntu/Debian 可透過 apt 直接安裝, 門檻很低就直接安裝測試

安裝程序

sudo apt update
sudo apt install nfs-ganesha nfs-ganesha-vfs

設定 nfs 分享目錄 Exp. /nfsshare 可讀寫, 只允許 10.20.0.0/22 這網段的用戶存取

vi /etc/ganesha/ganesha.conf

EXPORT
{
    Export_ID = 1;
    Path = /nfsshare;
    Pseudo = /nfsshare;
    Access_Type = RW;
    Squash = No_Root_Squash;
    FSAL
    {
        Name = VFS;
    }
    Clients = 10.20.0.0/22;
}

設定啟動 nfs-ganesha 服務

sudo systemctl restart nfs-ganesha
sudo systemctl enable nfs-ganesha

查詢服務狀態
```
sudo systemctl status nfs-ganesha
```

如果服務狀態不是完全正常

確認是否原本的 nfs-kernel service 還沒關閉, 關閉方式如下:

sudo systemctl stop nfs-kernel-server
sudo systemctl disable nfs-kernel-server

查詢 log 了解問題發生的原因: Exp.

tail -100 /var/log/ganesha/ganesha.log

看範例紀錄

# tail -100 /var/log/ganesha/ganesha.log
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:6): Unexpected character (▒)
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:6): Unexpected character (▒)
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:6): Unexpected character (▒)
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:6): Unexpected character (▒)
:
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:10): Unexpected character (▒)
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:3): Syntax error in block
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:0): Configuration syntax errors found
03/01/2024 05:24:53 : epoch 6594efa5 : test-jonathan-45 : ganesha.nfsd-4407[main] main :NFS STARTUP :FATAL :Fatal errors.  Server exiting...
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4431[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 3.0.3
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] load_rados_config :CONFIG :CRIT :Unknown urls backend
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] nfs_set_param_from_conf :NFS STARTUP :EVENT :Configuration file successfully parsed
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] init_server_pkgs :NFS STARTUP :EVENT :Initializing ID Mapper.
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] init_server_pkgs :NFS STARTUP :EVENT :ID Mapper successfully initialized.
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] nfs_start_grace :STATE :EVENT :NFS Server Now IN GRACE, duration 90
03/01/2024 05:26:18 : epoch 6594effa : test-jonathan-45 : ganesha.nfsd-4432[main] load_fsal :NFS STARTUP :FATAL :Could not dlopen module: /usr/lib/x86_64-linux-gnu/ganesha/libfsalvfs.so Error: /usr/lib/x86_64-linux-gnu/ganesha/libfsalvfs.so: cannot open shared object file: No such file or directory. You might want to install the nfs-ganesha-VFS package
03/01/2024 05:27:38 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4589[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 3.0.3
03/01/2024 05:27:38 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] load_rados_config :CONFIG :CRIT :Unknown urls backend
03/01/2024 05:27:38 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_set_param_from_conf :NFS STARTUP :EVENT :Configuration file successfully parsed
:
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_Start_threads :THREAD :EVENT :admin thread was started successfully
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[_9p_disp] _9p_dispatcher_thread :9P DISP :EVENT :9P dispatcher started
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_Start_threads :THREAD :EVENT :reaper thread was started successfully
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_Start_threads :THREAD :EVENT :General fridge was started successfully
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_start :NFS STARTUP :EVENT :-------------------------------------------------
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_start :NFS STARTUP :EVENT :             NFS SERVER INITIALIZED
03/01/2024 05:27:39 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[main] nfs_start :NFS STARTUP :EVENT :-------------------------------------------------
03/01/2024 05:29:09 : epoch 6594f04a : test-jonathan-45 : ganesha.nfsd-4591[reaper] nfs_lift_grace_locked :STATE :EVENT :NFS Server Now NOT IN GRACE

調整自訂記憶體當 Cache
- Mem_Max 預設可能是 64 MB 調高至 2048 MB → Mem_Max = 2147483648
- Mem_Cache_Size 預設可能是 32 MB 調高至 512 MB → Mem_Cache_Size = 536870912
- Dir_MaxCacheSize 預設可能是 512 KB 調高至 4096 KB → Dir_MaxCacheSize = 4194304
- 將設定加入 ganesha.conf 內
```
vi /etc/ganesha/ganesha.conf
```
```
NFS_Core_Param
{
    Mem_Max = 2147483648;
    Mem_Cache_Size = 536870912;
    Dir_MaxCacheSize = 4194304;
}

EXPORT
:
```
重啟 nfs-ganesha 服務讓設定生效
```
systemctl restart nfs-ganesha
```

使用 fio 來驗證調教後效能提升大概 18%

調教前看完整測試資料

Run status group 0 (all jobs):
   READ: bw=1637KiB/s (1676kB/s), 1637KiB/s-1637KiB/s (1676kB/s-1676kB/s), io=192MiB (202MB), run=120272-120272msec
  WRITE: bw=1637KiB/s (1677kB/s), 1637KiB/s-1637KiB/s (1677kB/s-1677kB/s), io=192MiB (202MB), run=120360-120360msec

調教後看完整測試資料

Run status group 0 (all jobs):
   READ: bw=1935KiB/s (1981kB/s), 1935KiB/s-1935KiB/s (1981kB/s-1981kB/s), io=228MiB (239MB), run=120434-120434msec
  WRITE: bw=1953KiB/s (2000kB/s), 1953KiB/s-1953KiB/s (2000kB/s-2000kB/s), io=230MiB (241MB), run=120500-120500msec

nfs

Num	Page	Accesses
1	tech:fortigate_tips	963
2	tech:win10_hash	800
3	tech:ubuntu_xrdp	762
4	start	721
5	tech:sqlsrv	446
6	tech:pvetips	318
7	tech:win10snmpd	299
8	tech:nginx_proxy_manager	286
9	tech:squid	266
10	tech:openvpn-client_ubunt...	265

在 Ubuntu/Debian 安裝 NFS Ganesha 取代 NFS-kernel service

安裝與設定方式

進階調教