Installer-rework

split installer structure into systems, updated non-functional Debian12 installer with some minor fixes to Ubuntu24 installer.
Updated docs.

docs/HW_INSTALL.md

@@ -7,49 +7,69 @@ To download and install NetAlertX on the hardware/server directly use the `curl`
 >
 > 🙏 Looking for maintainers for this installation method 🙂 Current community volunteers: 
 >   - [slammingprogramming](https://github.com/slammingprogramming)
+>   - [ingoratsdorf](https://github.com/ingoratsdorf)
 >
 > There is no guarantee that the install script or any other script will gracefully handle other installed software.
 > Data loss is a possibility, **it is recommended to install NetAlertX using the supplied Docker image**.
 
-A warning to the installation method below: Piping to bash is [controversial](https://pi-hole.net/2016/07/25/curling-and-piping-to-bash) and may
+> [!WARNING]
+> A warning to the installation method below: Piping to bash is [controversial](https://pi-hole.net/2016/07/25/curling-and-piping-to-bash) and may
 be dangerous, as you cannot see the code that's about to be executed on your system.
 
-Alternatively you can download the installation script `install/install.debian.sh` from the repository and check the code yourself (beware other scripts are
+If you trust this repo, you can download the install script via one of the methods (curl/wget) below and it will fo its best to install NetAlertX on your system.
-downloaded too - only from this repo).
+
+Alternatively you can download the installation script from the repository and check the code yourself.
 
 NetAlertX will be installed in `/app` and run on port number `20211`.
 
 Some facts about what and where something will be changed/installed by the HW install setup (may not contain everything!):
 
+- dependencies will be installed from the respective system repos
+- required python modules will be installed
 - `/app` directory will be deleted and newly created
-- `/app` will contain the whole repository (downloaded by `install/install.debian.sh`)
+- `/app` will contain the whole repository (downloaded by the install script)
 - The default NGINX site `/etc/nginx/sites-enabled/default` will be disabled (sym-link deleted or backed up to `sites-available`)
 - `/var/www/html/netalertx` directory will be deleted and newly created
-- `/etc/nginx/conf.d/netalertx.conf` will be sym-linked to `/app/install/netalertx.debian.conf`
+- `/etc/nginx/conf.d/netalertx.conf` will be sym-linked to the appropriate installer location (depending on your system installer script)
 - Some files (IEEE device vendors info, ...) will be created in the directory where the installation script is executed
 
 ## Limitations
 
-- No system service is provided. NetAlertX must be started using `/app/install/start.debian.sh`.
+- No system service is provided. NetAlertX must be started using `/app/install/<system>/start.<system>.sh`.
 - No checks for other running software is done.
-- Only tested to work on Debian Bookworm (Debian 12).
+- Only tested to work on the system listed in the install directory.
 - **EXPERIMENTAL** and not recommended way to install NetAlertX.
 
-## 📥 Installation via CURL
-
 > [!TIP]  
 > If the below fails try grabbing and installing one of the [previous releases](https://github.com/jokob-sk/NetAlertX/releases) and run the installation from the zip package.
 
-```bash
+These commands will download the `install.debian12.sh` script from the GitHub repository, make it executable with `chmod`, and then run it using `./install.debian12.sh`.
-curl -o install.debian.sh https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/install.debian.sh && sudo chmod +x install.debian.sh && sudo ./install.debian.sh
-```
-
-## 📥 Installation via WGET
-
-```bash
-wget https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/install.debian.sh -O install.debian.sh && sudo chmod +x install.debian.sh && sudo ./install.debian.sh
-```
-
-These commands will download the `install.debian.sh` script from the GitHub repository, make it executable with `chmod`, and then run it using `./install.debian.sh`.
 
 Make sure you have the necessary permissions to execute the script.
+
+
+## 📥 Debian 12 (Bookworm)
+
+### Installation via curl
+```bash
+curl -o install.debian12.sh https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/debian12/install.debian12.sh && sudo chmod +x install.debian12.sh && sudo ./install.debian12.sh
+```
+
+### Installation via wget
+
+```bash
+wget https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/debian12/install.debian12.sh -O install.debian12.sh && sudo chmod +x install.debian12.sh && sudo ./install.debian12.sh
+```
+
+## 📥 Ubuntu 24 (Noble Numbat)
+
+### Installation via curl
+```bash
+curl -o install.ubuntu24.sh https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/ubuntu24/install.ubuntu24.sh && sudo chmod +x install.ubuntu24.sh && sudo ./install.ubuntu24.sh
+```
+
+### Installation via wget
+
+```bash
+wget https://raw.githubusercontent.com/jokob-sk/NetAlertX/main/install/ubuntu24/install.ubuntu24.sh -O install.ubuntu24.sh && sudo chmod +x install.ubuntu24.sh && sudo ./install.ubuntu24.sh
+```

install/debian12/install.debian12.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+
+# 🛑 Important: This is only used for the bare-metal install 🛑 
+# Update /install/start.debian12.sh in most cases is preferred 
+
+echo "---------------------------------------------------------"
+echo "[INSTALL]                         Run install.debian12.sh"
+echo "---------------------------------------------------------"
+
+# Set environment variables
+INSTALL_DIR=/app  # Specify the installation directory here
+
+# Check if script is run as root
+if [[ $EUID -ne 0 ]]; then
+    echo "This script must be run as root. Please use 'sudo'." 
+    exit 1
+fi
+
+# Prepare the environment
+apt-get update
+apt-get install sudo -y
+
+# Install Git
+apt-get install -y git
+
+# Clean the directory
+rm -R $INSTALL_DIR/
+
+# Clone the application repository
+git clone https://github.com/jokob-sk/NetAlertX "$INSTALL_DIR/"
+
+# Check for buildtimestamp.txt existence, otherwise create it
+if [ ! -f $INSTALL_DIR/front/buildtimestamp.txt ]; then
+  date +%s > $INSTALL_DIR/front/buildtimestamp.txt
+fi
+
+# Start NetAlertX
+chmod +x "$INSTALL_DIR/install/debian12/start.debian12.sh"
+"$INSTALL_DIR/install/debian12/start.debian12.sh"

install/debian12/install_dependencies.debian12.sh

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+
+echo "---------------------------------------------------------"
+echo "[INSTALL]            Run install_dependencies.debian12.sh"
+echo "---------------------------------------------------------"
+
+# ❗ IMPORTANT - if you modify this file modify the root Dockerfile as well ❗
+
+# Check if script is run as root
+if [[ $EUID -ne 0 ]]; then
+    echo "This script must be run as root. Please use 'sudo'." 
+    exit 1
+fi
+
+# Install dependencies
+apt-get install -y \
+    tini snmp ca-certificates curl libwww-perl arp-scan perl apt-utils cron sudo \
+    nginx-light php php-cgi php-fpm php-sqlite3 php-curl sqlite3 dnsutils net-tools \
+    python3 python3-dev iproute2 nmap python3-pip zip usbutils traceroute nbtscan avahi-daemon avahi-utils openrc build-essential git
+
+# alternate dependencies
+sudo apt-get install nginx nginx-core mtr php-fpm php8.2-fpm php-cli php8.2 php8.2-sqlite3 -y
+sudo phpenmod -v 8.2 sqlite3 
+
+# setup virtual python environment so we can use pip3 to install packages
+apt-get install python3-venv -y
+python3 -m venv myenv
+source myenv/bin/activate
+
+update-alternatives --install /usr/bin/python python /usr/bin/python3 10
+
+#  install packages thru pip3
+pip3 install openwrt-luci-rpc asusrouter asyncio aiohttp graphene flask flask-cors unifi-sm-api tplink-omada-client wakeonlan pycryptodome requests paho-mqtt scapy cron-converter pytz json2table dhcp-leases pyunifi speedtest-cli chardet python-nmap dnspython librouteros yattag git+https://github.com/foreign-sub/aiofreepybox.git 

install/debian12/netalertx.conf

@@ -0,0 +1,20 @@
+server {
+    listen 20211 default_server;
+    root /var/www/html/netalertx;
+    index index.php;
+    #rewrite /app/(.*) / permanent;
+    add_header X-Forwarded-Prefix "/netalertx" always;
+    proxy_set_header X-Forwarded-Prefix "/netalertx";
+
+  location ~* \.php$ {
+    # Set Cache-Control header to prevent caching on the first load
+    add_header Cache-Control "no-store";
+    fastcgi_pass unix:/run/php/php8.2-fpm.sock;
+    include         fastcgi_params;
+    fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
+    fastcgi_param   SCRIPT_NAME        $fastcgi_script_name;
+    fastcgi_connect_timeout 75;
+    fastcgi_send_timeout 600;
+    fastcgi_read_timeout 600;
+}
+}

install/debian12/start.debian12.sh

@@ -0,0 +1,183 @@
+#!/usr/bin/env bash
+
+echo "---------------------------------------------------------"
+echo "[INSTALL]                           Run start.debian12.sh"
+echo "---------------------------------------------------------"
+echo
+echo "This script will set up and start NetAlertX on your Debian12 system."
+
+INSTALL_DIR=/app  # Specify the installation directory here
+
+# DO NOT CHANGE ANYTHING BELOW THIS LINE!
+INSTALLER_DIR=$INSTALL_DIR/install/ubuntu24
+CONF_FILE=app.conf
+DB_FILE=app.db
+NGINX_CONF_FILE=netalertx.conf
+WEB_UI_DIR=/var/www/html/netalertx
+NGINX_CONFIG_FILE=/etc/nginx/conf.d/$NGINX_CONF_FILE
+OUI_FILE="/usr/share/arp-scan/ieee-oui.txt" # Define the path to ieee-oui.txt and ieee-iab.txt
+INSTALL_PATH=$INSTALL_DIR/
+FILEDB=$INSTALL_PATH/db/$DB_FILE
+# DO NOT CHANGE ANYTHING ABOVE THIS LINE!
+
+# if custom variables not set we do not need to do anything
+if [ -n "${TZ}" ]; then    
+  FILECONF=$INSTALL_PATH/config/$CONF_FILE 
+  if [ -f "$FILECONF" ]; then
+    sed -ie "s|Europe/Berlin|${TZ}|g" $INSTALL_PATH/config/$CONF_FILE 
+  else 
+    sed -ie "s|Europe/Berlin|${TZ}|g" $INSTALL_PATH/back/$CONF_FILE.bak 
+  fi
+fi
+
+# Check if script is run as root
+if [[ $EUID -ne 0 ]]; then
+    echo "This script must be run as root. Please use 'sudo'." 
+    exit 1
+fi
+
+
+
+echo "---------------------------------------------------------"
+echo "[INSTALL] Installing dependencies"
+echo "---------------------------------------------------------"
+echo
+
+
+"${INSTALL_PATH}/install/debian12/install_dependencies.debian12.sh" # if modifying this file transfer the changes into the root Dockerfile.debian as well!
+
+
+echo "---------------------------------------------------------"
+echo "[INSTALL] Installing NGINX and setting up the web server"
+echo "---------------------------------------------------------"
+echo
+echo "[INSTALL] Stopping any NGINX web server"
+
+service nginx stop 2>/dev/null
+pkill -f "python ${INSTALL_DIR}/server" 2>/dev/null
+echo "[INSTALL] Updating the existing installation..."
+
+# Remove default NGINX site if it is symlinked, or backup it otherwise
+if [ -L /etc/nginx/sites-enabled/default ] ; then
+  echo "Disabling default NGINX site, removing sym-link in /etc/nginx/sites-enabled"
+  sudo rm /etc/nginx/sites-enabled/default
+elif [ -f /etc/nginx/sites-enabled/default ]; then
+  echo "Disabling default NGINX site, moving config to /etc/nginx/sites-available"
+  sudo mv /etc/nginx/sites-enabled/default /etc/nginx/sites-available/default.bkp_netalertx
+fi
+
+# Clear existing directories and files
+if [ -d $WEB_UI_DIR ]; then
+  echo "[INSTALL] Removing existing NetAlertX web-UI"
+  rm -R $WEB_UI_DIR
+fi
+
+echo "[INSTALL] Removing existing NetAlertX NGINX config"
+rm "$NGINX_CONFIG_FILE" 2>/dev/null || true
+
+# create symbolic link to the  install directory
+ln -s $INSTALL_PATH/front $WEB_UI_DIR
+# create symbolic link to NGINX configuration coming with NetAlertX
+sudo ln -s "${INSTALL_PATH}/install/debian12/netalertx.conf" /etc/nginx/conf.d/$NGINX_CONF_FILE
+
+# Use user-supplied port if set
+if [ -n "${PORT}" ]; then
+  echo "Setting webserver to user-supplied port ($PORT)"
+  sudo sed -i 's/listen 20211/listen '"$PORT"'/g' /etc/nginx/conf.d/$NGINX_CONF_FILE
+fi
+
+# Change web interface address if set
+if [ -n "${LISTEN_ADDR}" ]; then
+  echo "Setting webserver to user-supplied address (${LISTEN_ADDR})"
+  sed -ie 's/listen /listen '"${LISTEN_ADDR}":'/g' /etc/nginx/conf.d/$NGINX_CONF_FILE
+fi
+
+# Run the hardware vendors update at least once
+echo "[INSTALL] Run the hardware vendors update"
+
+# Check if ieee-oui.txt or ieee-iab.txt exist
+if [ -f "$OUI_FILE" ]; then
+  echo "The file ieee-oui.txt exists. Skipping update_vendors.sh..."
+else
+  echo "The file ieee-oui.txt does not exist. Running update_vendors..."
+
+  # Run the update_vendors.sh script
+  if [ -f "${INSTALL_PATH}/back/update_vendors.sh" ]; then
+    "${INSTALL_PATH}/back/update_vendors.sh"
+  else
+    echo "update_vendors.sh script not found in $INSTALL_DIR."    
+  fi
+fi
+
+# Create an empty log files
+
+# Create the execution_queue.log file if it doesn't exist
+touch "${INSTALL_DIR}"/log/{app.log,execution_queue.log,app_front.log,app.php_errors.log,stderr.log,stdout.log,db_is_locked.log}
+touch "${INSTALL_DIR}"/api/user_notifications.json
+# Create plugins sub-directory if it doesn't exist in case a custom log folder is used
+mkdir -p "${INSTALL_DIR}"/log/plugins
+
+# Fixing file permissions
+echo "[INSTALL] Fixing file permissions"
+chown root:www-data "${INSTALL_DIR}"/api/user_notifications.json
+
+echo "[INSTALL] Fixing WEB_UI_DIR: ${WEB_UI_DIR}"
+chmod -R a+rwx $WEB_UI_DIR
+
+echo "[INSTALL] Fixing INSTALL_DIR: ${INSTALL_DIR}"
+
+chmod -R a+rw $INSTALL_PATH/log
+chmod -R a+rwx $INSTALL_DIR
+
+echo "[INSTALL] Copy starter $DB_FILE and $CONF_FILE if they don't exist"
+
+# DANGER ZONE: ALWAYS_FRESH_INSTALL 
+if [ "$ALWAYS_FRESH_INSTALL" = true ]; then
+  echo "[INSTALL] ❗ ALERT /db and /config folders are cleared because the ALWAYS_FRESH_INSTALL is set to: ${ALWAYS_FRESH_INSTALL}❗"
+  # Delete content of "/config/"
+  rm -rf "${INSTALL_PATH}/config/"*
+  
+  # Delete content of "/db/"
+  rm -rf "${INSTALL_PATH}/db/"*
+fi
+
+
+# Copy starter $DB_FILE and $CONF_FILE if they don't exist
+cp -n "${INSTALL_PATH}/back/$CONF_FILE" "${INSTALL_PATH}/config/$CONF_FILE" 
+cp -n "${INSTALL_PATH}/back/$DB_FILE"  "$FILEDB"
+
+echo "[INSTALL] Fixing permissions after copied starter config & DB"
+
+if [ -f "$FILEDB" ]; then
+    chown -R www-data:www-data $FILEDB
+fi
+
+chmod -R a+rwx $INSTALL_DIR # second time after we copied the files
+chmod -R a+rw $INSTALL_PATH/config
+sudo chgrp -R www-data  $INSTALL_PATH
+
+# Check if buildtimestamp.txt doesn't exist
+if [ ! -f "${INSTALL_PATH}/front/buildtimestamp.txt" ]; then
+    # Create buildtimestamp.txt
+    date +%s > "${INSTALL_PATH}/front/buildtimestamp.txt"
+fi
+
+# start PHP
+/etc/init.d/php8.2-fpm start
+nginx -t || { echo "[INSTALL] nginx config test failed"; exit 1; }
+/etc/init.d/nginx start
+
+# Start Nginx and your application to start at boot (if needed)
+# systemctl start nginx
+# systemctl enable nginx
+
+# # systemctl enable pi-alert
+# sudo systemctl restart nginx
+
+#  Activate the virtual python environment
+source myenv/bin/activate
+
+echo "[INSTALL] 🚀 Starting app - navigate to your <server IP>:${PORT}"
+
+# Start the NetAlertX python script
+python $INSTALL_PATH/server/

install/ubuntu24/install.ubuntu24.sh

@@ -14,6 +14,7 @@ echo "---------------------------------------------------------"
 
 # Set environment variables
 INSTALL_DIR=/app  # Specify the installation directory here
+INSTALLER_DIR=$INSTALL_DIR/install/ubuntu24
 
 # Check if script is run as root
 if [[ $EUID -ne 0 ]]; then
@@ -36,10 +37,12 @@ apt-get install -y git
 if [ -d "$INSTALL_DIR" ]; then
   echo "The installation directory exists. Removing it to ensure a clean install."
   echo "Are you sure you want to continue? This will delete all existing files in $INSTALL_DIR."
+  echo "This will include ALL YOUR SETTINGS AND DATABASE! (if there are any)"
+  echo
   echo "Type:"
-  echo " - 'install' to continue"
+  echo " - 'install' to continue and DELETE ALL!"
-  echo " - 'update' to just update from GIT"
+  echo " - 'update' to just update from GIT (keeps your db and settings)"
-  echo " - 'start' to do nothing, leave install as-is"
+  echo " - 'start' to do nothing, leave install as-is (just run the start script)"
   if [ "$1" == "install" ] || [ "$1" == "update" ] || [ "$1" == "start" ]; then
     confirmation=$1
   else
@@ -52,9 +55,9 @@ if [ -d "$INSTALL_DIR" ]; then
 
       # Stop nginx if running
       if command -v systemctl >/dev/null 2>&1 && systemctl list-units --type=service | grep -q nginx; then
-      systemctl stop nginx 2>/dev/null
+        systemctl stop nginx 2>/dev/null
       elif command -v service >/dev/null 2>&1; then
-      service nginx stop 2>/dev/null
+        service nginx stop 2>/dev/null
       fi
 
       # Kill running NetAlertX server processes in this INSTALL_DIR
@@ -73,14 +76,10 @@ if [ -d "$INSTALL_DIR" ]; then
       echo "INSTALL_DIR is not set, is root, or is invalid. Aborting for safety."
       exit 1
     fi
-    else
-      echo "INSTALL_DIR is not set or is root. Aborting for safety."
-      exit 1
-    fi
   elif [ "$confirmation" == "update" ]; then
     echo "Updating the existing installation..."
     service nginx stop 2>/dev/null
-   pkill -f "python ${INSTALL_DIR}/server" 2>/dev/null
+    pkill -f "python ${INSTALL_DIR}/server" 2>/dev/null
     cd "$INSTALL_DIR" || { echo "Failed to change directory to $INSTALL_DIR"; exit 1; }
     git pull
   elif [ "$confirmation" == "start" ]; then
@@ -101,5 +100,6 @@ fi
 # Start NetAlertX
 
 # This is where we setup the virtual environment and install dependencies
-cd "$INSTALL_DIR/install/ubuntu" || { echo "Failed to change directory to $INSTALL_DIR/install/ubuntu"; exit 1; }
+cd "$INSTALLER_DIR" || { echo "Failed to change directory to $INSTALLER_DIR"; exit 1; }
-"$INSTALL_DIR/install/ubuntu/start.ubuntu.sh"
+chmod +x "$INSTALLER_DIR/start.ubuntu24.sh"
+"$INSTALLER_DIR/start.ubuntu24.sh"

install/ubuntu24/start.ubuntu24.sh

@@ -4,15 +4,16 @@ echo "---------------------------------------------------------"
 echo "[INSTALL]"
 echo "---------------------------------------------------------"
 echo
-echo "This script will set up and start NetAlertX on your Ubuntu system."
+echo "This script will set up and start NetAlertX on your Ubuntu24 system."
 
 # Specify the installation directory here
 INSTALL_DIR=/app
 
 # DO NOT CHANGE ANYTHING BELOW THIS LINE!
+INSTALLER_DIR=$INSTALL_DIR/install/ubuntu24
 CONF_FILE=app.conf
 DB_FILE=app.db
-NGINX_CONF_FILE=netalertx.ubuntu.conf
+NGINX_CONF_FILE=netalertx.conf
 WEB_UI_DIR=/var/www/html/netalertx
 NGINX_CONFIG_FILE=/etc/nginx/conf.d/$NGINX_CONF_FILE
 OUI_FILE="/usr/share/arp-scan/ieee-oui.txt" # Define the path to ieee-oui.txt and ieee-iab.txt
@@ -58,7 +59,7 @@ phpenmod -v ${PHPVERSION} sqlite3
 
 update-alternatives --install /usr/bin/python python /usr/bin/python3 10
 
-cd $INSTALL_DIR/install/ubuntu || { echo "Failed to change directory to $INSTALL_DIR/install/ubuntu"; exit 1; }
+cd $INSTALLER_DIR || { echo "Failed to change directory to $INSTALLER_DIR"; exit 1; }
 
 # setup virtual python environment so we can use pip3 to install packages
 apt-get install python3-venv -y
@@ -102,7 +103,7 @@ rm "$NGINX_CONFIG_FILE" 2>/dev/null || true
 # create symbolic link to the  install directory
 ln -s $INSTALL_PATH/front $WEB_UI_DIR
 # create symbolic link to NGINX configuration coming with NetAlertX
-ln -s "${INSTALL_PATH}/install/ubuntu/$NGINX_CONF_FILE" $NGINX_CONFIG_FILE
+ln -s "${INSTALLER_DIR}/$NGINX_CONF_FILE" $NGINX_CONFIG_FILE
 
 # Use user-supplied port if set
 if [ -n "${PORT}" ]; then