diff --git a/README.md b/README.md index e7c2aea..fe0a857 100644 --- a/README.md +++ b/README.md @@ -1,153 +1,63 @@ -| :zap: February 2023: This is a mirrored repository of a private repository. The project is currently in a testing stage, please contact the project developer at pietro.marini@nextcloud.com, if you want to start collaborating.| +| :zap: April 2023: This is a mirrored repository of a private repository. The project is currently in a testing stage, please contact the project developer at pietro.marini@nextcloud.com, if you want to start collaborating.| |----------------------------------------------------------------------------------------------------| +### Introduction -### Introduction to nc-env - -nc-env is a project that enables you to provision isolated Nextcloud test environments in your machine. It is built on vagrant and LXD. +nc-env is a tool that enables you to provision isolated Nextcloud test environments in your machine. It is built on [vagrant](https://www.vagrantup.com/) and [LXD](https://linuxcontainers.org/). ### Features -You can leverage this project to: -* Quickly setup and run an isolated Nextcloud instance in your machine -* Try features or troubleshoot issues without any manual installation -* Learn and explore the integration of Nextcloud with common enterprise systems, such as LDAP, Single Sign On or an Office Editing Service + +You can leverage this tool to: + +* Quickly setup and run an isolated Nextcloud instance in your machine. +* Try features or troubleshoot issues without any manual installation and minimal configuration. +* Learn and explore the integration of Nextcloud with common enterprise systems, such as LDAP, ElasticSearch, Single Sign-On or an Office Editing Service. ### Available Templates +nc-env is based on templates. A template is a set of instructions and artifacts sufficient to have a service up and running in a container. + The following table summarizes the list of available templates: -|Template | Description| -| --- | --- | -| `template00-clean-server` | Clean server | -| `template01-nextcloud-standalone` | Nextcloud standalone server | -| `template02-collabora-online` | Collabora Online server - CODE or Enterprise | -| `template03-keycloak` | Keycloak standalone server | -| `template04-ldap` | OpenLDAP server | -| `template05-elasticsearch` | Elastic Search server | -| `template06-nextcloud-db-standalone` | Database node (to be used in a cluster) | -| `template07-glusterfs-server` | GlusterFS node (to be used in a cluster) | -| `template08-haproxy-server` | Haproxy (to be used in a cluster) | -| `template09-web-server-node` | Web Server Node (to be used in a cluster) | -| `template10-redis-server` | Redis server(to be used in a cluster) | -| `template11-minio-storage-server` | MinIO Storage Server | -| `template13-talk-hpb` | Talk High Performance Backend | -| `template14-self-hosted-appstore` | Nextcloud Self-Hosted Appstore | -| `template16-zabbix-server` | Zabbix Monitoring Server | - -### Setup the environment - -#### Install and configure LXD - -The initial configuration guide is [here](https://linuxcontainers.org/lxd/getting-started-cli/) and applies to many Linux distributions. - -If you are interested in understanding the difference between application and system containers, [this](https://ubuntu.com//blog/what-are-linux-containers) is a good short article that explains it. - -##### Configure the host to resolve container hostnames - -While container hostnames can be defined in `/etc/hosts`, you can avoid this additional manual step by defining a systemd service that instructs the resolver to query the built-in DNS service for domains ending with `dns.domain`. - -To do that, follow these instructions (reference documentation [here](https://linuxcontainers.org/lxd/docs/master/networks/#integration-with-systemd-resolved)) - -Get the network interface used by LXD (by default `lxdbr0`, and assumed as such in the following) and the IP of the built-in DNS (in this output `10.23.46.1`): - -``` -$ lxc network list - -+--------+----------+---------+---------------+-------------------------+-------------+---------+ -| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | -+--------+----------+---------+---------------+-------------------------+-------------+---------+ -| lxdbr0 | bridge | YES | 10.23.46.1/24 | fd42:fa6c:5eee:12::1/64 | | 25 | -+--------+----------+---------+---------------+-------------------------+-------------+---------+ -``` - -Set the configuration variables `dns.domain` (this is optional, the default is `lxd`) and `dns.mode`: - -``` -$ lxc network set lxdbr0 dns.domain 'localenv.com' - -$ lxc network set lxdbr0 dns.mode managed -``` - -Create a systemd unit file called `lxd-dns-lxdbr0.service` that allows this setup to be persistent across machine reboots. - -``` -$ sudo cat /etc/systemd/system/lxd-dns-lxdbr0.service -[Unit] -Description=LXD per-link DNS configuration for lxdbr0 -BindsTo=sys-subsystem-net-devices-lxdbr0.device -After=sys-subsystem-net-devices-lxdbr0.device - -[Service] -Type=oneshot -ExecStart=/usr/bin/resolvectl dns lxdbr0 10.23.46.1 -ExecStart=/usr/bin/resolvectl domain lxdbr0 '~localenv.com' - -[Install] -WantedBy=sys-subsystem-net-devices-lxdbr0.device - -``` - -Enable it and start it: - -``` -$ sudo systemctl daemon-reload - -$ sudo systemctl enable lxd-dns-lxdbr0.service - -$ sudo systemctl start lxd-dns-lxdbr0.service - -$ sudo systemctl status lxd-dns-lxdbr0.service -? lxd-dns-lxdbr0.service - LXD per-link DNS configuration for lxdbr0 - Loaded: loaded (/etc/systemd/system/lxd-dns-lxdbr0.service; enabled; vendor preset: enabled) - Active: inactive (dead) since Tue 2022-01-04 12:32:09 CET; 1h 26min ago - Process: 1916 ExecStart=/usr/bin/resolvectl dns lxdbr0 10.23.46.1 (code=exited, status=0/SUCCESS) - Process: 1917 ExecStart=/usr/bin/resolvectl domain lxdbr0 ~localenv.com (code=exited, status=0/SUCCESS) - Main PID: 1917 (code=exited, status=0/SUCCESS) - CPU: 8ms - -ene 04 12:32:09 my-host systemd[1]: Starting LXD per-link DNS configuration for lxdbr0... -ene 04 12:32:09 my-host systemd[1]: lxd-dns-lxdbr0.service: Deactivated successfully. -ene 04 12:32:09 my-host systemd[1]: Finished LXD per-link DNS configuration for lxdbr0. - -$ sudo resolvectl status lxdbr0 -Link 4 (lxdbr0) - Current Scopes: DNS - Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported -Current DNS Server: 10.23.46.1 - DNS Servers: 10.23.46.1 - DNS Domain: ~localenv.com - -``` +|Template | Description| Maintained | +| --- | --- | --- | +| `template00-clean-server` | Clean server | YES | +| `template01-nextcloud-standalone` | Nextcloud standalone server | YES | +| `template02-collabora-online` | Collabora Online server - CODE or Enterprise | YES | +| `template03-keycloak` | Keycloak standalone server | YES | +| `template04-ldap` | OpenLDAP server + phpLDAPadmin | YES | +| `template05-elasticsearch` | ElasticSearch server | YES | +| `template06-nextcloud-db-standalone` | Database node (to be used in a cluster) | YES | +| `template07-glusterfs-server` | GlusterFS node (to be used in a cluster) | YES | +| `template08-haproxy-server` | HAproxy (to be used in a cluster) | YES | +| `template09-web-server-node` | Web Server Node (to be used in a cluster) | YES | +| `template10-redis-server` | Redis server(to be used in a cluster) | YES | +| `template11-minio-storage-server` | MinIO Storage Server | YES | +| `template13-talk-hpb` | Talk High Performance Backend | YES | +| `template14-self-hosted-appstore` | Nextcloud Self-Hosted Appstore | NO | +| `template16-zabbix-server` | Zabbix Monitoring Server | YES | -#### Install and configure vagrant -You can learn about installation [here](https://www.vagrantup.com/docs/installation). For Ubuntu-based systems, it is as simple as `sudo apt install vagrant` +### Setup -#### Install LXD plugin vagrant-lxd -This plugin adds the ability to manage containers with LXD. See [here](https://gitlab.com/catalyst-it/devtools/vagrant-lxd) for installation instructions. +This section aims at giving an high-level procedure to setup your environment. -#### Install mkcert +Please head to the `how-to` folder to find step-by-step guides for main Linux Desktop distributions ([Fedora](./how-to/How-To-Setup-Nc-Env-In-Fedora-Workstation.md), [Ubuntu](./how-to/How-To-Setup-Nc-Env-In-Ubuntu-Desktop.md), [Debian](./how-to/ +How-To-Setup-Nc-Env-in-Debian-Bullseye.md)). -Download it [here](https://github.com/FiloSottile/mkcert/releases)) and run `mkcert --install`. This command installs a local certificate for the Certificate Authority along with its key. This is handy as it avoids many warnings and errors when -dealing with self-trusted certificates. +1. Install and configure LXD +2. Configure the host to resolve container hostnames +3. Install and configure vagrant +4. Install LXD plugin vagrant-lxd +5. Install mkcert +6. Get the latest release of nc-env +7. Start provisioning -#### Get the latest release +### Resources -Download the latest release of nc-env from the repository Releases section. - -### Provision a container - -Once the environment is ready, an instance of the container template that you want to work with can be created and started. - -Copy it in the target folder and make sure that the `artifacts` folder contains all the artifacts documented in the `Readme.md` file, located in the template root folder. - -If some additional configuration step is needed, it will be also documented there. - -After running the provisioning step, make sure your host system is able to resolve the domain name that you specified for the container, for example by adding an entry in `/etc/hosts`. - -Afterwards, the environment is ready to be used. - +* Article: [nc-env: A tool to deploy Nextcloud Environments in your laptop](https://propuestas.eslib.re/2022/miscelanea/nc-env-creacion-entornos-nextcloud-local). +* Video: [This workshop](https://commons.wikimedia.org/wiki/File:EsLibre_2022_P24_-_Pietro_Marini_-_C%C3%B3mo_montar_un_cl%C3%BAster_de_Nextcloud.webm), recorded at esLibre 2022, is focused on how to build a Nextcloud cluster using nc-env. In Spanish. diff --git a/how-to/How-To-Setup-Nc-Env-In-Ubuntu-Desktop.md b/how-to/How-To-Setup-Nc-Env-In-Ubuntu-Desktop.md index e3dcc26..0db76b6 100644 --- a/how-to/How-To-Setup-Nc-Env-In-Ubuntu-Desktop.md +++ b/how-to/How-To-Setup-Nc-Env-In-Ubuntu-Desktop.md @@ -17,12 +17,18 @@ | Component |Version | |----------------|----------------------------------| -|Operating System|Ubuntu 22.04 LTS (Jammy Jellyfish)| -|LXD |5.10 | +|Operating System|Ubuntu 22.04.2 LTS (Jammy Jellyfish)| +|LXD |5.12 | |vagrant |2.3.4 | |vagrant-lxd |0.6.0 | +## Pre-requisites + +* An Ubuntu Desktop environment, physical or virtual machine +* At least 20 GB of disk space to dedicate to nc-env containers. This could be in the OS partition, another partition or on a dedicated block device + + ## Procedure Install all the software needed for the project, starting by updating the package repositories: @@ -43,7 +49,7 @@ ``` $ sudo snap install lxd -lxd 5.0.0-b0287c1 from Canonical✓ installed +lxd 5.12-c63881f from Canonical\u2713 installed ``` Verify that LXD has been correctly installed: @@ -51,19 +57,32 @@ ``` $ snap list lxd Name Version Rev Tracking Publisher Notes -lxd 5.10-b392610 24323 latest/stable canonical✓ - +lxd 5.12-c63881f 24643 latest/stable canonical\u2713 - + ``` +### Deciding about the storage backend + +LXD supports several storage backends, you can get more information in the [documentation](https://linuxcontainers.org/lxd/docs/latest/explanation/storage/). + +These are the recommended options, in descending order: + +1. A block device or a partition on a block device dedicated to the project, to be ZFS-formatted +2. A loop device to be ZFS-formatted +3. The filesystem in the OS partition or another one + +In any case, it is recommended to assign nc-env at least 20 GB. + ### ZFS Utilities Installation --- -:memo: While it is recommeded to use ZFS as backend for the LXD storage pools for this project, you can skip this section if you plan to use directory-based LXD storage pools. +:memo: Go thorugh this section only if you choose option 1 above. --- -If you plan to use ZFS as backend for the LXD storage pools, you should have a disk or unformatted partition, of 20GB or more, for that. +If you plan to use ZFS as backend for the LXD storage pools, you should have a disk or unformatted partition, of 20GB or more, for that. Install the ZFS package: @@ -75,8 +94,8 @@ ``` $ sudo zpool --version -zfs-2.1.4-0ubuntu0.1 -zfs-kmod-2.1.4-0ubuntu0.1 +zfs-2.1.5-1ubuntu6~22.04.1 +zfs-kmod-2.1.5-1ubuntu6~22.04.1 ``` ### Vagrant Installation @@ -85,7 +104,7 @@ ``` -$ wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg +$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg $ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list $ sudo apt update @@ -111,19 +130,6 @@ ``` $ vagrant plugin install vagrant-lxd -Installing the 'vagrant-lxd' plugin. This can take a few minutes... -Fetching thread_safe-0.3.6.gem -Fetching tzinfo-1.2.9.gem -Fetching minitest-5.15.0.gem -Fetching activesupport-5.2.7.gem -Fetching multipart-post-2.1.1.gem -Fetching faraday-0.17.5.gem -Fetching public_suffix-4.0.7.gem -Fetching addressable-2.8.0.gem -Fetching sawyer-0.9.1.gem -Fetching hyperkit-1.3.0.gem -Fetching vagrant-lxd-0.5.6.gem -Installed the plugin 'vagrant-lxd (0.5.6)'! ``` Verify that it has been correctly installed: @@ -164,13 +170,20 @@ It should be under `${HOME}/.local/share/mkcert` +--- + +:memo: mkcert has browser detection logic, so if one is detected (Chrome, Firefox) the install command will let you know. If the browser is not detected for some reason (ex. Firefox installed via snap) the browser is not detected, you will need to manually import +the certificate in the browser. + +--- + ### Nc-env Installation -Pick the latest release of the project from [here](https://codeberg.org/pmarini/nc-env/releases), in this example `v20220421`: +Pick the latest release of the project from [here](https://codeberg.org/pmarini/nc-env/releases), in this example `v20230421`: ``` -$ wget https://codeberg.org/pmarini/nc-env/archive/v20220421.tar.gz -$ tar -xzf v20220421.tar.gz +$ wget https://codeberg.org/pmarini/nc-env/archive/v20230421.tar.gz +$ tar -xzf v20230421.tar.gz $ cd nc-env/ ``` @@ -180,7 +193,7 @@ --- -:memo: The process below assumes you have a free local block device that we can use as a storage pool. If you don’t have one, you can use the `dir` backend +:memo: The process below assumes you have a free local block device that we can use as a storage pool. If you don’t have one, you can use a ZFS-formatted loop device or nolxthe `dir` backend --- @@ -217,9 +230,11 @@ Create a LXD profile for the containers created in nc-env: ``` -$ lxc profile create nc-env-profile -$ lxc profile device add nc-env-profile root disk path=/ pool=pool01 size=5GB -$ lxc profile device add nc-env-profile eth0 nic nictype=bridged name=eth0 parent=lxdbr0 +$ lxc profile create prf-nc-env +$ lxc profile device add prf-nc-env root disk path=/ pool=pool01 size=5GB +$ lxc profile device add prf-nc-env eth0 nic nictype=bridged name=eth0 parent=lxdbr0 +$ lxc profile set prf-nc-env limits.memory 1GiB +$ lxc profile set prf-nc-env limits.cpu 1 $ lxc profile list +----------------+---------------------+---------+ | NAME | DESCRIPTION | USED BY | @@ -314,7 +329,7 @@ Let's start a sample container: ``` -$ lxc launch ubuntu:20.04 c1 +$ lxc launch ubuntu:22.04 c1 Creating c1 Starting c1 ``` @@ -362,9 +377,11 @@ $ lxc delete -f c1 ``` -## Containers Creation with nc-env +## Your first container with nc-env -Create the first container with nc-env. As a first container let’s choose the Nextcloud Standalone container, that is `template01-nextcloud-standalone`: +Now the environment is ready to be used and you can create the first container with nc-env. + +For example, let’s pick the Nextcloud Standalone container, that is `template01-nextcloud-standalone`: ``` $ pwd @@ -375,7 +392,7 @@ $ cd nc01 ``` -Follow the instructions that you find in the Readme.md file. +Follow the instructions that you find in the `Readme.md` file. At the end of the configuration the local folder layout should be similar to the following: @@ -383,18 +400,16 @@ ``` ├── artifacts -│   ├── mkcert │   ├── nc-fulltext-live-indexer.service │   ├── nc-redirect.conf -│   ├── nextcloud-23.0.3.tar.bz2 -│   ├── nextcloud-23.0.3.tar.bz2.md5 +│   ├── nextcloud-26.0.0.tar.bz2 +│   ├── nextcloud-26.0.0.tar.bz2.md5 │   ├── nextcloud.conf │   ├── occ │   ├── redis.conf │   ├── rootCA-key.pem │   └── rootCA.pem ├── log -│   └── provision.log ├── provision.sh ├── Readme.md └── Vagrantfile