Browse Source

Try to fix Snap.

Thanks to xet7 !
Lauri Ojansivu 3 days ago
parent
commit
9d9f77a731
2 changed files with 176 additions and 35 deletions
  1. 103 0
      docs/Platforms/FOSS/Snap/Snap-build.md
  2. 73 35
      snapcraft.yaml

+ 103 - 0
docs/Platforms/FOSS/Snap/Snap-build.md

@@ -0,0 +1,103 @@
+# Building the Wekan snap without timeouts
+
+This guide focuses on macOS hosts (Multipass VM) and common timeout fixes. It also applies to Linux hosts with LXD.
+
+## Quick options
+
+- Fastest: use Canonical builders (no local VM)
+
+  ```zsh
+  # One-time: login to the store (required for remote-build)
+  snapcraft login
+  
+  # Build for amd64 on Canonical builders
+  snapcraft remote-build --build-for=amd64
+  ```
+
+- Local VM (macOS + Multipass): increase resources and build verbosely
+
+  ```zsh
+  # Give the builder more CPU/RAM/disk to avoid sluggish downloads
+  export SNAPCRAFT_BUILD_ENVIRONMENT=hosted-multipass
+  export SNAPCRAFT_BUILD_ENVIRONMENT_CPU=4
+  export SNAPCRAFT_BUILD_ENVIRONMENT_MEMORY=8G
+  export SNAPCRAFT_BUILD_ENVIRONMENT_DISK=40G
+  
+  # Clean previous state and build
+  snapcraft clean
+  snapcraft --verbose --debug
+  ```
+
+## What changed to reduce timeouts
+
+- Downloads in `wekan` part now retry with exponential backoff.
+- `caddy` part now attempts APT with retries and falls back to a static binary from the official GitHub release if APT is slow or unreachable.
+
+These changes make the build resilient to transient network issues and slow mirrors.
+
+## Diagnosing where it stalls
+
+- Run a single step for a part to reproduce:
+  ```zsh
+  snapcraft pull wekan -v
+  snapcraft build wekan -v
+  ```
+- Drop into the build environment when it fails:
+  ```zsh
+  snapcraft --debug
+  # Then run the failing commands manually
+  ```
+
+## Tips for macOS + Multipass
+
+- Check networking:
+  ```zsh
+  multipass list
+  multipass exec snapcraft-*-wekan -- ping -c2 github.com
+  ```
+- If the instance looks wedged, recreate it:
+  ```zsh
+  snapcraft clean --use-lxd || true   # harmless on macOS
+  snapcraft clean --step pull
+  multipass delete --purge $(multipass list | awk '/snapcraft-/{print $1}')
+  snapcraft --verbose
+  ```
+
+## Linux hosts (optional)
+
+On Linux, using LXD is often faster and more reliable than Multipass:
+
+```bash
+sudo snap install lxd --channel=5.21/stable
+newgrp lxd
+snapcraft --use-lxd -v
+```
+
+## Common environment knobs
+
+- Proxy/mirror environments inside the build VM if needed:
+  ```zsh
+  export http_proxy=http://proxy.example:3128
+  export https_proxy=$http_proxy
+  export SNAPCRAFT_PROXY_HTTP=$http_proxy
+  export SNAPCRAFT_PROXY_HTTPS=$https_proxy
+  ```
+
+- Speed up apt by pinning retries (already set in the recipe) or switching to a closer mirror by customizing sources in an override if needed.
+
+## Cleaning up caches
+
+If repeated attempts keep hitting corrupt downloads, clean Snapcraft caches:
+
+```zsh
+snapcraft clean --destructive-mode || true
+rm -rf ~/.cache/snapcraft/*
+```
+
+## Reporting
+
+If you still hit timeouts, capture and share:
+- The exact step (pull/build/stage/prime) and part name
+- Output of `snapcraft --verbose --debug`
+- Host OS and Snapcraft version: `snapcraft --version`
+- Multipass resources: `multipass list`

+ 73 - 35
snapcraft.yaml

@@ -131,9 +131,32 @@ parts:
         stage-packages:
             - libfontconfig1
         override-build: |
-            echo "Cleaning environment first"
+                        set -euo pipefail
+                        echo "Cleaning environment first"
             #rm -rf ~/.meteor ~/.npm /usr/local/lib/node_modules
             rm -rf .build
+                        # Helper: resilient downloader (tries curl, then wget) with retries/backoff
+                        download() {
+                            url="$1"; out="$2"; attempts="${3:-5}"; sleepsec=5
+                            for i in $(seq 1 "$attempts"); do
+                                echo "[download] ($i/$attempts) $url -> $out"
+                                if command -v curl >/dev/null 2>&1; then
+                                    if curl -fL --retry 5 --retry-all-errors --connect-timeout 20 --max-time 0 -o "$out" "$url"; then
+                                        return 0
+                                    fi
+                                fi
+                                if command -v wget >/dev/null 2>&1; then
+                                    if wget --tries=5 --waitretry=5 --retry-connrefused -O "$out" "$url"; then
+                                        return 0
+                                    fi
+                                fi
+                                echo "[download] attempt $i failed, sleeping ${sleepsec}s before retry..."
+                                sleep "$sleepsec" || true
+                                sleepsec=$(( sleepsec * 2 ))
+                            done
+                            echo "[download] ERROR: Unable to download $url after $attempts attempts" >&2
+                            return 1
+                        }
             #echo "Using http npm packages so speedup install process https://stackoverflow.com/questions/39760113/callback-called-more-than-once-while-running-npm-install"
             #echo "registry=http://registry.npmjs.org/" > ~/.npmrc
             #echo "Installing npm, node-gyp, node-pre-gyp, fibers"
@@ -166,9 +189,9 @@ parts:
             # Cleanup
             mkdir .build
             cd .build
-            wget https://github.com/wekan/wekan/releases/download/v8.15/wekan-8.15-amd64.zip
-            unzip wekan-8.15-amd64.zip
-            rm wekan-8.15-amd64.zip
+            download https://github.com/wekan/wekan/releases/download/v8.15/wekan-8.15-amd64.zip wekan-8.15-amd64.zip 6
+            unzip -q wekan-8.15-amd64.zip
+            rm -f wekan-8.15-amd64.zip
             cd ..
             ##cd .build/bundle
             ##find . -type d -name '*-garbage*' | xargs rm -rf
@@ -183,9 +206,9 @@ parts:
             #rm fibers-multi.7z
             #cd ../../../../../../..
             # Copy to Snap
-            wget https://github.com/wekan/node-v14-esm/releases/download/v14.21.4/node-v14.21.4-linux-x64.tar.xz
+            download https://github.com/wekan/node-v14-esm/releases/download/v14.21.4/node-v14.21.4-linux-x64.tar.xz node-v14.21.4-linux-x64.tar.xz 6
             tar -xf node-v14.21.4-linux-x64.tar.xz node-v14.21.4-linux-x64/bin/node
-            rm node-v14.21.4-linux-x64.tar.xz
+            rm -f node-v14.21.4-linux-x64.tar.xz
             mkdir $SNAPCRAFT_PART_INSTALL/bin
             cp -p node-v14.21.4-linux-x64/bin/node $SNAPCRAFT_PART_INSTALL/bin/
             rm -rf node-v14.21.4-linux-x64
@@ -226,40 +249,55 @@ parts:
             - gnupg
             - curl
         override-build: |
-            # Add Caddy repository
-            echo "Installing Caddy 2 from the official repository..."
-            curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /tmp/caddy-stable-archive-keyring.gpg
-            mkdir -p /etc/apt/keyrings
-            cp /tmp/caddy-stable-archive-keyring.gpg /etc/apt/keyrings/
-            echo "deb [signed-by=/etc/apt/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main" > /etc/apt/sources.list.d/caddy-stable.list
-            apt update
-            apt -y install caddy
+                        set -euo pipefail
+                        # Resilient install of Caddy: try APT with retries, fallback to static binary
+                        echo "Installing Caddy 2..."
+                        try_apt_install() {
+                            echo "[caddy] Adding repository and installing via APT"
+                            curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /tmp/caddy-stable-archive-keyring.gpg
+                            mkdir -p /etc/apt/keyrings
+                            cp /tmp/caddy-stable-archive-keyring.gpg /etc/apt/keyrings/
+                            echo "deb [signed-by=/etc/apt/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main" > /etc/apt/sources.list.d/caddy-stable.list
+                            apt-get -o Acquire::Retries=5 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 update
+                            DEBIAN_FRONTEND=noninteractive apt-get -o Acquire::Retries=5 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 -y install caddy
+                        }
+                        download_caddy_static() {
+                            echo "[caddy] Falling back to static binary download"
+                            CADDY_URL="https://github.com/caddyserver/caddy/releases/download/v2.8.4/caddy_2.8.4_linux_amd64.tar.gz"
+                            TMPDIR=$(mktemp -d)
+                            curl -fL --retry 5 --retry-all-errors --connect-timeout 20 --max-time 0 "$CADDY_URL" -o "$TMPDIR/caddy.tgz" || wget --tries=5 --waitretry=5 --retry-connrefused -O "$TMPDIR/caddy.tgz" "$CADDY_URL"
+                            tar -C "$TMPDIR" -xzf "$TMPDIR/caddy.tgz" caddy
+                            install -m 0755 "$TMPDIR/caddy" /usr/bin/caddy
+                            rm -rf "$TMPDIR"
+                        }
+                        if ! try_apt_install; then
+                            echo "[caddy] APT path failed; using static binary"
+                            download_caddy_static
+                        fi
 
-            # Display installed Caddy version for confirmation
-            echo "Installed Caddy version:"
-            /usr/bin/caddy version
+                        echo "Installed Caddy version:"
+                        /usr/bin/caddy version || true
 
-            # Create directory structure in the snap
-            mkdir -p $SNAPCRAFT_PART_INSTALL/bin
+                        # Create directory structure in the snap
+                        mkdir -p $SNAPCRAFT_PART_INSTALL/bin
+                        # Copy Caddy binary
+                        cp /usr/bin/caddy $SNAPCRAFT_PART_INSTALL/bin/
+                        chmod +x $SNAPCRAFT_PART_INSTALL/bin/caddy
 
-            # Copy Caddy binary
-            cp /usr/bin/caddy $SNAPCRAFT_PART_INSTALL/bin/
-            chmod +x $SNAPCRAFT_PART_INSTALL/bin/caddy
+                        # Create license files manually since they don't exist in the package
+                        mkdir -p $SNAPCRAFT_PART_INSTALL/license
+                        echo "Caddy is licensed under the Apache License 2.0. See https://github.com/caddyserver/caddy/blob/master/LICENSE" > $SNAPCRAFT_PART_INSTALL/license/CADDY_LICENSE
 
-            # Create license files manually since they don't exist in the package
-            mkdir -p $SNAPCRAFT_PART_INSTALL/license
-            echo "Caddy is licensed under the Apache License 2.0. See https://github.com/caddyserver/caddy/blob/master/LICENSE" > $SNAPCRAFT_PART_INSTALL/license/CADDY_LICENSE
+                        # Create a basic default Caddyfile for the snap
+                        mkdir -p $SNAPCRAFT_PART_INSTALL/etc
+                        cat > $SNAPCRAFT_PART_INSTALL/etc/Caddyfile << 'EOF'
+                        # Default Caddyfile for Wekan
+                        # This is loaded by caddy-control script if no other config is provided
 
-            # Create a basic default Caddyfile for the snap
-            mkdir -p $SNAPCRAFT_PART_INSTALL/etc
-            cat > $SNAPCRAFT_PART_INSTALL/etc/Caddyfile << 'EOF'
-            # Default Caddyfile for Wekan
-            # This is loaded by caddy-control script if no other config is provided
-
-            :8080 {
-                reverse_proxy localhost:3000
-            }
-            EOF
+                        :8080 {
+                                reverse_proxy localhost:3000
+                        }
+                        EOF
         stage:
             - bin/caddy
             - license/CADDY_LICENSE