Sharing Storage

Introduction

Why and How sharing storage ?

Sharing storage to an LXC, VMs or a Docker container is not that easy in proxmox. The traditionnal way is just by allocating sufficiant storage to the VM or the LXC. But in my case, I want VMs and container to be running on my SSDs for maximum performance and when they need to store data (user docs, films, tv shows, ...), I want it to be stored on my HDDs.

Here you can see the different ways to share data depending of the technology :

Technology
Mountpoint
Disk Pass-through
Network share

LXC

✅

✅

⚠️ (Only privileged)

VM

❌

✅

✅

Docker container

✅ (With Host)

❌

✅

To increase security and have a better backup stategy, I want to do as little as possible directly on the Proxmox host :

  • Docker containers will be deployed on a separate VM.

  • If I need to do network shares using SMB or NFS, I don't want to install those tools directly on the host.

That let me with two choices :

  • Install a network sharing server on a LXC to share storage that is mounted using mountpoints.

  • Pass-through my 2 data disks to a VMs that will handle those disks and all the sharing

My first idea was to use the second option, with an architecture simillar to what the excellent channel IBRACORP presented through their IBRANAS series : Using a Truenas Scale VM (I used Truenas scale in the past so I know well their GUI), pass-through data disks, and Truenas handled the ZFS Pool and the sharing to other VMs and even back to the host.

The problem is that in their config, they don't use ZFS for their system pool. So I can't use Truenas, because Truenas works also with ZFS, installing a VM with truenas on my system would layer ZFS-over-ZFS (not for data, but on my system SSDs), therefore drastically decrease performance and my SSDs lifetime by doubling I/O.

Selected Solution

The best way to do it in my scenario is to let Proxmox handle ZFS on my drives, then create a specific dataset that will be mounted on the container. The dataset creation has been done in Configure Storage.

The container will then handle all the network shares, managing users etc..

This way, I don't install anything on my proxmox host, and the container is backed up periodically.

Installation

LXC Creation

First, we have to download a CT Template, here we're chosing Debian 11 Bullseye.

Then we just have to create a LXC Container, click on "Create CT" on the top of the proxmox GUI.

And use this configuration :

Choose you name in the hostname tab, but no domain name, just "nas" for example.

Let CT ID as it is

Password = default root password of the container

Let the container unprivileged, we don't need it for sharing, and keep nesting as Debian 11 requires it to be performant.

First boot

Start the CT a first time, log in on the console using the root password you set earlier.

Execute the traditionnal command to update packages :

Then create a folder where your data will be mounted, for it will be /mnt/nas/

And then, if you are like me and you don't need ssh (you always have console through proxmox interface), then disable it with :

Then stop the container before next step.

Creating mountpoint

Open your proxmox console and execute this command to add the mountpoint between the storage you want to add to the container and the path to mount it on the container

then there is two options :

  • This dataset in your proxmox will be used for something apart from this LXC, and your LXC is unprivileged, then you have to map UID as refered in the doc of proxmox :

LogoUnprivileged LXC containers - Proxmox VEpve.proxmox.com
How to map UID - Proxmox Documentation

  • This dataset is only used in this container, and then you can just change the owner of the folder on the host to 100000:100000 with this command on the host :

That is because unprivileged container are using offset ID, UID 0 on the CT (root) = UID 100000 on the host.

Therefore, if the folder is owned by the Proxmox Host root UID 0, for the container it's like this folder is owned by some strange -100000 UID, all files in the mountpoint appears with a nobody:nogroup owner, restricting access to read-only

Install Samba

For this install, I chose to use samba for 3 reasons :

  • Restrict shares with authentification by user (only Kerberos in NFS, no time to configure that at the moment)

  • Possibility to mount those shares easier on Windows

  • I couldn't get NFS Server to work in a LXC Container

First install samba with apt :

Then backup the old config file and create a new one :

Configure the file with your share, here for example my config file with one share :

You can add all the shares we have and save this file, now we have to create the folder we specified and the users that goes along.

I chose to have a specific users for each share to increase security, if one password get stolen, this account cannot be used to access other shares which could contains sensitive information, and even if they could see the share, they at least could not write to it due to the mask and the forced user.

For each share and users, execute those commands :

Now reboot the container and you should be able to access your share at the address with the appropriate user and password :

PreviousConfigure BackupsNextDocker VM

Last updated