dig utility name discovery work #534

FIX unable to send mail with publisher mail plugin - thanks to @LouisOb 🙏

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,20 +0,0 @@
----
-name: Feature request
-about: Suggest an idea for this project
-title: ''
-labels: ''
-assignees: ''
-
----
-
-**Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
-
-**Describe the solution you'd like**
-A clear and concise description of what you want to happen.
-
-**Describe alternatives you've considered**
-A clear and concise description of any alternative solutions or features you've considered.
-
-**Additional context**
-Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,38 @@
+name: Feature Request
+description: 'Suggest an idea for PiAlert'
+labels: ['Feature request➕']
+body:
+- type: checkboxes
+  attributes:
+    label: Is there an existing issue for this?
+    description: Please search to see if an open or closed issue already exists for the feature you are requesting. 
+    options:
+    - label: I have searched the existing open and closed issues
+      required: true
+- type: textarea
+  attributes:
+    label: Is your feature request related to a problem? Please describe
+    description: A clear and concise description of what the problem is.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Describe the solution you'd like
+    description: A clear and concise description of what you want to happen.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Describe alternatives you've considered
+    description: A clear and concise description of any alternative solutions or features you've considered.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Anything else?
+    description: |
+      Links? References? Mockups? Anything that will give us more context about the feature you are encountering!
+      
+      Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
+  validations:
+    required: true

.github/ISSUE_TEMPLATE/i-have-an-issue.md

@@ -1,46 +0,0 @@
----
-name: I have an issue
-about: Describe this issue template's purpose here.
-title: ''
-labels: ''
-assignees: ''
-
----
-
-**Describe the issue**
-
-
-**Paste last few lines from `pialert.log`**
-
-```
-
-paste here
-
-```
-
-**Paste your `pialert.conf` (remove personal info)**
-
-```
-
-paste here
-
-```
-
-**Paste your `docker-compose.yml` and `.env` (remove personal info)**
-
-`docker-compose.yml` 
-```
-
-paste here
-
-```
-
-
-`.env` 
-```
-
-paste here
-
-```
-**Screenshots**
-If applicable, add screenshots to help explain your problem.

.github/ISSUE_TEMPLATE/i-have-an-issue.yml

@@ -0,0 +1,76 @@
+name: Bug Report
+description: 'When submitting an issue enable debug and have a look at the docs.'
+labels: ['bug 🐛']
+body:
+- type: checkboxes
+  attributes:
+    label: Is there an existing issue for this?
+    description: Please search to see if an open or closed issue already exists for the bug you encountered.
+    options:
+    - label: I have searched the existing open and closed issues and I checked the docs https://github.com/jokob-sk/Pi.Alert/tree/main/docs
+      required: true
+- type: textarea
+  attributes:
+    label: Current Behavior
+    description: A concise description of what you're experiencing.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Expected Behavior
+    description: A concise description of what you expected to happen.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Steps To Reproduce
+    description: Steps to reproduce the behavior.
+    placeholder: |
+      1. With these settings...
+      2. With this config...
+      3. Run '...'
+      4. See error...
+  validations:
+    required: false
+- type: textarea
+  attributes:
+    label: pialert.conf
+    description: |
+      Paste your `pialert.conf` (remove personal info)    
+    render: python
+  validations:
+    required: false
+- type: textarea
+  attributes:
+    label: docker-compose.yml
+    description: |
+      Paste your `docker-compose.yml`    
+    render: python
+  validations:
+    required: false
+- type: dropdown
+  attributes:
+    label: What branch are you running?
+    options:
+      - Production
+      - Dev
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: pialert.log
+    description: |
+      Logs with debug enabled (https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md) ⚠
+      ***Generally speaking, all bug reports should have logs provided.***
+      Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
+      Additionally, any additional info? Screenshots? References? Anything that will give us more context about the issue you are encountering!
+      You can use `tail -100 /home/pi/pialert/front/log/pialert.log` in teh container if you have troubles getting to the log files. 
+  validations:
+    required: false
+- type: checkboxes
+  attributes:
+    label: Debug enabled
+    description: I confirm I enabled `debug`
+    options:
+    - label: I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.
+      required: true

.github/workflows/docker.yml

@@ -1,114 +0,0 @@
----
-name: docker
-
-on:
-  push:
-    branches:
-      - '**'
-    tags:
-      - '*.*.*'
-  pull_request:
-    branches:
-      - master
-
-jobs:
-  docker:
-    runs-on: ubuntu-latest
-    if: contains(github.event.head_commit.message, 'PUSHPROD')
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-
-      - name: Set up dynamic build ARGs
-        id: getargs        
-        run: echo "version=$(cat ./stable/VERSION)" >> $GITHUB_OUTPUT
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v3
-        with:
-          # list of Docker images to use as base name for tags
-          images: |
-            jokobsk/pi.alert
-          # generate Docker tags based on the following events/attributes
-          tags: |
-            type=raw,value=latest
-            type=schedule
-            type=ref,event=branch
-            type=ref,event=pr
-            type=semver,pattern={{version}}
-            type=semver,pattern={{major}}.{{minor}}
-            type=semver,pattern={{major}}
-            type=sha
-
-      - name: Login to DockerHub
-        if: github.event_name != 'pull_request'
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Build and push
-        uses: docker/build-push-action@v3
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64,linux/arm/v7
-          push: ${{ github.event_name != 'pull_request' }}
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-  docker_dev:
-    runs-on: ubuntu-latest
-    if: contains(github.event.head_commit.message, 'PUSHPROD') != 'True'
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-
-      - name: Set up dynamic build ARGs
-        id: getargs       
-        run: echo "version=$(cat ./stable/VERSION)" >> $GITHUB_OUTPUT
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v3
-        with:
-          # list of Docker images to use as base name for tags
-          images: |
-            jokobsk/pi.alert_dev
-          # generate Docker tags based on the following events/attributes
-          tags: |
-            type=raw,value=latest
-            type=schedule
-            type=ref,event=branch
-            type=ref,event=pr
-            type=semver,pattern={{version}}
-            type=semver,pattern={{major}}.{{minor}}
-            type=semver,pattern={{major}}
-            type=sha
-
-      - name: Login to DockerHub
-        if: github.event_name != 'pull_request'
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Build and push
-        uses: docker/build-push-action@v3
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64,linux/arm/v7
-          push: ${{ github.event_name != 'pull_request' }}
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}

.github/workflows/docker_cache-cleaner.yml

@@ -0,0 +1,25 @@
+name: ci-package-cleaner
+
+on:
+
+  workflow_dispatch: # manual option
+
+  schedule:
+    - cron: '15 22 * * 1' # every Monday 10.15pm UTC (~11.15am Tuesday NZT)
+
+jobs:
+
+  package-cleaner:
+    name: package-cleaner
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      packages: write
+    steps:
+
+      - uses: actions/delete-package-versions@v4
+        with: 
+          package-name: pi.alert
+          package-type: container
+          min-versions-to-keep: 0
+          delete-only-untagged-versions: true

.github/workflows/docker_dev.yml

@@ -0,0 +1,91 @@
+---
+name: docker
+
+on:
+  push:
+    branches:
+      - '**'
+    tags:
+      - '*.*.*'
+  pull_request:
+    branches:
+      - master
+
+jobs: 
+  docker_dev:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+      packages: write
+    if: >
+      contains(github.event.head_commit.message, 'PUSHPROD') != 'True' &&
+      github.repository == 'jokob-sk/Pi.Alert'  
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Set up dynamic build ARGs
+        id: getargs        
+        run: echo "version=$(cat ./stable/VERSION)" >> $GITHUB_OUTPUT
+
+      - name: Get release version
+        id: get_version
+        run: echo "::set-output name=version::${{ 'Dev' }}"
+
+      - name: Create .VERSION file
+        run: echo "${{ steps.get_version.outputs.version }}" >> .VERSION
+
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          # list of Docker images to use as base name for tags
+          images: |
+            jokobsk/pi.alert_dev
+          # generate Docker tags based on the following events/attributes
+          tags: |
+            type=raw,value=latest
+            type=schedule
+            type=ref,event=branch
+            type=ref,event=pr
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=semver,pattern={{major}}
+            type=sha
+
+      - name: Log in to Github Container registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: jokob-sk
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Login to DockerHub
+        if: github.event_name != 'pull_request'
+        uses: docker/login-action@v2
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      # # Disable this after use
+      # - name: Prune Docker Builder
+      #   run: docker builder prune --force
+
+      - name: Build and push
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          # # ⚠ disable cache if build is failing to download debian packages
+          # cache-from: type=registry,ref=ghcr.io/jokob-sk/pi.alert:buildcache
+          # cache-to: type=registry,ref=ghcr.io/jokob-sk/pi.alert:buildcache,mode=max

.github/workflows/docker_prod.yml

@@ -0,0 +1,85 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# GitHub recommends pinning actions to a commit SHA.
+# To get a newer version, you will need to update the SHA.
+# You can also reference a tag or branch, but the action may change without warning.
+
+name: Publish Docker image
+
+on:
+  release:
+    types: [published]
+    tags:
+      - '*.*.*'
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Set up dynamic build ARGs
+        id: getargs        
+        run: echo "version=$(cat ./stable/VERSION)" >> $GITHUB_OUTPUT
+
+      - name: Get release version
+        id: get_version
+        run: echo "::set-output name=version::${GITHUB_REF#refs/tags/}"
+
+      - name: Create .VERSION file
+        run: echo "${{ steps.get_version.outputs.version }}" >> .VERSION
+
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          # list of Docker images to use as base name for tags
+          images: |
+            jokobsk/pi.alert
+          # generate Docker tags based on the following events/attributes
+          tags: |
+            type=semver,pattern={{version}},value=${{ inputs.version }}
+            type=semver,pattern={{major}}.{{minor}},value=${{ inputs.version }}
+            type=semver,pattern={{major}},value=${{ inputs.version }}
+            type=ref,event=branch,suffix=-{{ sha }}
+            type=ref,event=pr
+            type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/') }}
+
+      - name: Log in to Github Container registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: jokob-sk
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Login to DockerHub
+        if: github.event_name != 'pull_request'
+        uses: docker/login-action@v2
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Build and push
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          # # ⚠ disable cache if build is failing to download debian packages
+          # cache-from: type=registry,ref=ghcr.io/jokob-sk/pi.alert:buildcache
+          # cache-to: type=registry,ref=ghcr.io/jokob-sk/pi.alert:buildcache,mode=max

.gitignore

@@ -1,4 +1,20 @@
+.vscode
 .DS_Store
+config/*
 config/pialert.conf
 db/*
+db/pialert.db
 front/log/*
+front/api/*
+**/plugins/**/*.log
+**/%40eaDir/
+**/@eaDir/
+
+__pycache__/
+*.py[cod]
+*$py.class
+
+**/last_result.log
+**/script.log
+**/pialert.conf_bak
+**/pialert.db_bak

CONTRIBUTING

@@ -6,7 +6,7 @@ The issue tracker is the preferred channel for bug reports, features requests an
 
 Before submitting a new issue please spend a couple of minutes on research:
 
-* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/tree/main/dockerfiles#-common-issues) 
+* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#common-issues) 
 * Check [💡 Closed issues](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed) if a similar issue was solved in the past.
 
 ## Pull-requests (PRs)

Dockerfile

@@ -1,21 +1,15 @@
-FROM debian:bullseye-slim
+FROM debian:bookworm-slim
 
 # default UID and GID
-ENV USER=pi USER_ID=1000 USER_GID=1000 TZ=Europe/London PORT=20211
+ENV USER=pi USER_ID=1000 USER_GID=1000  PORT=20211 
+#TZ=Europe/London
 
 # Todo, figure out why using a workdir instead of full paths don't work
 # Todo, do we still need all these packages? I can already see sudo which isn't needed
 
-RUN apt-get update \
-    && apt-get install --no-install-recommends tini ca-certificates curl libwww-perl arp-scan perl apt-utils cron sudo nginx-light php php-cgi php-fpm php-sqlite3 php-curl sqlite3 dnsutils net-tools python3 iproute2 nmap python3-pip zip -y \
-    && pip3 install requests paho-mqtt scapy cron-converter pytz \
-    # && pip3 install requests paho-mqtt ssdpy upnpclient \
-    && update-alternatives --install /usr/bin/python python /usr/bin/python3 10 \
-    && apt-get clean autoclean \
-    && apt-get autoremove \
-    && rm -rf /var/lib/apt/lists/* \
-    && rm -rf /var/www/html \
-    && ln -s /home/pi/pialert/front /var/www/html 
+RUN apt-get update 
+RUN apt-get install sudo -y 
+
 
 # create pi user and group
 # add root and www-data to pi group so they can r/w files and db
@@ -31,17 +25,26 @@ RUN groupadd --gid "${USER_GID}" "${USER}" && \
 
 COPY --chmod=775 --chown=${USER_ID}:${USER_GID} . /home/pi/pialert/
 
-# Pi.Alert
-RUN rm /etc/nginx/sites-available/default \
-    && ln -s /home/pi/pialert/install/default /etc/nginx/sites-available/default \
-    && sed -ie 's/listen 80/listen '${PORT}'/g' /etc/nginx/sites-available/default \
-    # run the hardware vendors update
-    && /home/pi/pialert/back/update_vendors.sh \
-    # Create a backup of the pialert.conf to be used if the user didn't supply a configuration file
-    && cp /home/pi/pialert/config/pialert.conf /home/pi/pialert/back/pialert.conf_bak \
-    # Create a backup of the pialert.db to be used if the user didn't supply a database
-    && cp /home/pi/pialert/db/pialert.db /home/pi/pialert/back/pialert.db_bak 
 
-ENTRYPOINT ["tini", "--"]
+# ❗ IMPORTANT - if you modify this file modify the /install/install_dependecies.sh file as well ❗ 
+
+RUN apt-get install -y \
+    tini snmp ca-certificates curl libwww-perl arp-scan perl apt-utils cron sudo \
+    nginx-light php php-cgi php-fpm php-sqlite3 php-curl sqlite3 dnsutils net-tools \
+    python3 iproute2 nmap python3-pip zip systemctl usbutils traceroute
+
+# Alternate dependencies
+RUN apt-get install nginx nginx-core mtr php-fpm php8.2-fpm php-cli php8.2 php8.2-sqlite3 -y
+RUN phpenmod -v 8.2 sqlite3 
+
+# Setup virtual python environment and use pip3 to install packages
+RUN apt-get install -y python3-venv
+RUN python3 -m venv myenv
+RUN /bin/bash -c "source myenv/bin/activate && update-alternatives --install /usr/bin/python python /usr/bin/python3 10 && pip3 install requests paho-mqtt scapy cron-converter pytz json2table dhcp-leases pyunifi speedtest-cli chardet"
+
+# Create a buildtimestamp.txt to later check if a new version was released
+RUN date +%s > /home/pi/pialert/front/buildtimestamp.txt
 
 CMD ["/home/pi/pialert/dockerfiles/start.sh"]
+
+

README.md

@@ -1,108 +1,91 @@
-# Pi.Alert
-<!--- --------------------------------------------------------------------- --->
+# 💻🔍 Network security scanner & notification framework
 
-💻🔍 WIFI / LAN intruder detector.
+Get visibility of what's going on on your WIFI/LAN network. Scan for devices, port changes and get alerts if unknown devices or changes are found. Write your own [Plugins](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins#readme) with auto-generated UI and in-build notification system. 
 
-Scans for devices connected to your WIFI / LAN and alerts you if new and unknown devices are found.
-
-![Main screen][main]
-
-
-# 🐳 Docker image 
-[![Docker](https://img.shields.io/github/actions/workflow/status/jokob-sk/Pi.Alert/docker.yml?branch=main&label=Build&logo=GitHub)](https://github.com/jokob-sk/Pi.Alert/actions/workflows/docker.yml)
+[![Docker](https://img.shields.io/github/actions/workflow/status/jokob-sk/Pi.Alert/docker_prod.yml?label=Build&logo=GitHub)](https://github.com/jokob-sk/Pi.Alert/actions/workflows/docker_prod.yml)
 [![GitHub Committed](https://img.shields.io/github/last-commit/jokob-sk/Pi.Alert?color=40ba12&label=Committed&logo=GitHub&logoColor=fff)](https://github.com/jokob-sk/Pi.Alert)
 [![Docker Size](https://img.shields.io/docker/image-size/jokobsk/pi.alert?label=Size&logo=Docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
 [![Docker Pulls](https://img.shields.io/docker/pulls/jokobsk/pi.alert?label=Pulls&logo=docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
 [![Docker Pushed](https://img.shields.io/badge/dynamic/json?color=0aa8d2&logoColor=fff&label=Pushed&query=last_updated&url=https%3A%2F%2Fhub.docker.com%2Fv2%2Frepositories%2Fjokobsk%2Fpi.alert%2F&logo=docker&link=http://left&link=https://hub.docker.com/repository/docker/jokobsk/pi.alert)](https://hub.docker.com/r/jokobsk/pi.alert)
 
-🐳 [Docker hub](https://registry.hub.docker.com/r/jokobsk/pi.alert) | 📄 [Dockerfile](https://github.com/jokob-sk/Pi.Alert/blob/main/Dockerfile) | 📚 [Docker instructions](https://github.com/jokob-sk/Pi.Alert/blob/main//dockerfiles/README.md) | 🆕 [Release notes](https://github.com/jokob-sk/Pi.Alert/issues/138)
+  | 🐳 [Docker hub](https://registry.hub.docker.com/r/jokobsk/pi.alert) |  📑 [Docker guide](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md) |🆕 [Release notes](https://github.com/jokob-sk/Pi.Alert/releases) | 📚 [All Docs](https://github.com/jokob-sk/Pi.Alert/tree/main/docs) |
+  |----------------------|----------------------| ----------------------|  ----------------------| 
 
-## 🔍 Scan Methods
-The system continuously scans the network for, **New devices**, **New connections** (re-connections), **Disconnections**, **"Always Connected" devices down**, Devices **IP changes** and **Internet IP address changes**. Scanning methods are:
-  - **Method 1: arp-scan**. The arp-scan system utility is used to search
-        for devices on the network using arp frames.
-  - **Method 2: Pi-hole**. This method is optional and complementary to
-        method 1. If the Pi-hole DNS server is active, Pi.Alert examines its
-        activity looking for active devices using DNS that have not been
-        detected by method 1.
-  - **Method 3. dnsmasq**. This method is optional and complementary to the
-        previous methods. If the DHCP server dnsmasq is active, Pi.Alert
-        examines the DHCP leases (addresses assigned) to find active devices
-        that were not discovered by the other methods.
+## Why PiAlert❓ 
+
+Most of us don't know what's going on on our home network, but we want our family and data to be safe.  _Command-line tools_ are great, but the output can be _hard to understand_ and action if you are not a network specialist.
+
+PiAlert gives you peace of mind. _Visualize and immediately report 📬_ what is going on in your network - this is the first step to enhance your _network security 🔐_. 
+
+_PiAlert combines several network and other scanning tools 🔍 with notifications 📧 into one user-friendly package 📦_. You get an overview of network device Sessions, Connected devices, Events, Presence, Down alerts, and IPs. You can schedule Nmap scans to detect changes in device ports and visualize your Network topology (even with undetectable, dummy devices).
+
+Setup a _kill switch ☠_ for your network via a smart plug with the available [Home Assistant](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HOME_ASSISTANT.md) integration. Implement custom automations with the [CSV device Exports 📤](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins/csv_backup), [Webhooks](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/WEBHOOK_N8N.md), or [API endpoints](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md) features. 
+
+Extend the app if you want to create your own scanner [Plugin](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins#readme) and handle the results and notifications in PiAlert. 
+
+Looking forward to your contributions if you decide to share your work with the community ❤.
+
+  | ![Main screen][main] | ![Screen 1][screen1]  | ![Screen 5][screen5] |
+  |----------------------|----------------------| ----------------------| 
+  | ![Screen 3][screen3] | ![Screen 4][screen4] | ![Screen 6][screen6]  |  
+  | ![Screen 8][screen8] | ![Report 2][report2] | ![Screen 9][screen9]  | 
+
+## Scan Methods, Notifications, Integration, Extension system
+
+| Features    | Details    | 
+|-------------|-------------|
+|      🔍     |   The app scans your network for, **New devices**, **New connections** (re-connections), **Disconnections**, **"Always Connected" devices down**, Devices **IP changes** and **Internet IP address changes**. Discovery & scan methods include: **arp-scan**.  **Pi-hole - DB import**,  **Pi-hole - DHCP leases import**, **Generic DHCP leases import**. **UNIFI controller import**, **SNMP-enabled router import**. Check the [Plugins](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins#readme) docs for more info on individual scans. |
+|📧           | Send notifications to more than 80+ services, including Telegram via [Apprise](https://hub.docker.com/r/caronc/apprise), or use [Pushsafer](https://www.pushsafer.com/), or [NTFY](https://ntfy.sh/). |
+|🧩           | Feed your data and device changes into [Home Assistant](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HOME_ASSISTANT.md), read [API endpoints](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md), or use [Webhooks](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/WEBHOOK_N8N.md) to setup custom automation flows.  |
+|➕           | Build your own scanners with the [Plugin system](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins#readme) |
 
 
-## 🧩 Notification Integrations 
-   - [Apprise](https://hub.docker.com/r/caronc/apprise), [Pushsafer](https://www.pushsafer.com/), [NTFY](https://ntfy.sh/)
-   - Webhooks ([sample JSON](docs/webhook_json_sample.json))
-   - Home Assistant via [MQTT](https://www.home-assistant.io/integrations/mqtt/) 
-     - discovery ~10s per device, deleting not supported, use [MQTT Explorer](https://mqtt-explorer.com/) for now
-
-
-## 🔐 Security
-
-- Configurable login to prevent unauthorized use. 
-
-## 📑 Features   
-  - Display:
-    - Sessions, Connected devices, Favorites, Events, Presence, Concurrent devices, Down alerts, IP's
-    - Manual Nmap scans, Optional speedtest for Device "Internet"
-    - Simple Network relationship display
-  - Maintenance tasks and Settings like:
-    - Status Infos (active scans, database size, backup counter)
-    - Theme Selection (blue, red, green, yellow, black, purple) and Light/Dark-Mode Switch
-    - Language Selection (English, German, Spanish)    
-    - Pause arp-scan
-    - DB maintenance, Backup, Restore tools and CSV Export / Import
-  - Help/FAQ Section 
-
-  | ![Screen 1][screen1] | ![Screen 2][screen2] | 
-  |----------------------|----------------------| 
-  | ![Screen 3][screen3] | ![Screen 4][screen4] |
-  | ![Screen 5][screen5] | ![Screen 6][screen6] |
-  | ![Report 1][report1] | ![Report 2][report2] |
- 
-
-# 📥 Installation
+## Installation & Documentation
 <!--- --------------------------------------------------------------------- --->
 
- ⚠ This [fork (jokob-sk)](https://github.com/jokob-sk/Pi.Alert) is only tested as a [docker container](dockerfiles/README.md). Check out [leiweibau's fork](https://github.com/leiweibau/Pi.Alert/) if you want to install Pi.Alert on the server directly.
+| Docs        | Link    | 
+|-------------|-------------|
+| 📥🐳  | [Docker instructions](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md) 
+| 📥💻  | [HW install (experimental 🧪)](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HW_INSTALL.md) |
+| 📚     | [All Documentation](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/README.md) (App Usage and Configuration) |
  
-Instructions for [pucherot's original code can be found here](https://github.com/pucherot/Pi.Alert/)
+> Other Alternatives
+>
+> - Check out [leiweibau's on HW installed fork](https://github.com/leiweibau/Pi.Alert/) (maintained)
+> - Check instructions for [pucherot's original code](https://github.com/pucherot/Pi.Alert/) (unmaintained)
+> - [WatchYourLAN](https://github.com/aceberg/WatchYourLAN) - Lightweight network IP scanner with web GUI (Open source)
+> - [Fing](https://www.fing.com/) - Network scanner app for your Internet security (Commercial, Phone App, Proprietary hardware)
 
+## ❤ Support me
 
-## 🔗 Other
+Get:
+- Regular updates to keep your data and family safe 🔄 
+- Better and more functionality➕
+- I don't get burned out and the app survives longer🔥🤯
+- Quicker and better support with issues 🆘
+- Less grumpy me 😄
 
+| [![GitHub](https://i.imgur.com/emsRCPh.png)](https://github.com/sponsors/jokob-sk) | [![Buy Me A Coffee](https://i.imgur.com/pIM6YXL.png)](https://www.buymeacoffee.com/jokobsk) | [![Patreon](https://i.imgur.com/MuYsrq1.png)](https://www.patreon.com/user?u=84385063) | 
+| --- | --- | --- | 
 
+- Bitcoin: `1N8tupjeCK12qRVU2XrV17WvKK7LCawyZM`
+- Ethereum: `0x6e2749Cb42F4411bc98501406BdcD82244e3f9C7`
+
+> 📧 Email me at [jokob@duck.com](mailto:jokob@duck.com?subject=PiAlert) if you want to get in touch or if I should add other sponsorship platforms.
+
+## Everything else
 <!--- --------------------------------------------------------------------- --->
 
-<!--- --------------------------------------------------------------------- --->
-### Alternatives
-
-  - [WatchYourLAN](https://github.com/aceberg/WatchYourLAN) - Lightweight network IP scanner with web GUI (Open source)
-  - [Fing](https://www.fing.com/) - Network scanner app for your Internet security (Commercial, Phone App, Proprietary hardware)
-
-### Old docs
-
-  - [Device Management instructions](docs/DEVICE_MANAGEMENT.md)
-  - [Versions History](docs/VERSIONS_HISTORY.md)
-
 ### License
-  GPL 3.0
-  - [Read more here](LICENSE.txt)
-  - Source of the [animated GIF (Loading Animation)](https://commons.wikimedia.org/wiki/File:Loading_Animation.gif)  
-  - Source of the [selfhosted Fonts](https://github.com/adobe-fonts/source-sans)
+>  GPL 3.0 | [Read more here](LICENSE.txt) | Source of the [animated GIF (Loading Animation)](https://commons.wikimedia.org/wiki/File:Loading_Animation.gif) | Source of the [selfhosted Fonts](https://github.com/adobe-fonts/source-sans)
   
-### 🥇 Special thanks 
+### Special thanks 
 
-  This code is a collaborative body of work, with special thanks to: 
+This code is a collaborative body of work, with special thanks to: 
 
-   - 🏆 [pucherot/Pi.Alert](https://github.com/pucherot/Pi.Alert) is the original creator od PiAlert
-      - [leiweibau](https://github.com/leiweibau/Pi.Alert): Dark mode (and much more)
-      - [Macleykun](https://github.com/Macleykun): Help with Dockerfile clean-up
-      - [Final-Hawk](https://github.com/Final-Hawk): Help with NTFY, styling and other fixes
-      - [TeroRERO](https://github.com/terorero): Spanish translation
-      - [jokob-sk](https://github.com/jokob-sk/Pi.Alert): DB Maintenance tools
-      - Please see the [Git commit history](https://github.com/jokob-sk/Pi.Alert/commits/main) for a full list of people and their contributions to the project
+> [pucherot/Pi.Alert](https://github.com/pucherot/Pi.Alert) (the original creator of PiAlert), [leiweibau](https://github.com/leiweibau/Pi.Alert): Dark mode (and much more), [Macleykun](https://github.com/Macleykun) (Help with Dockerfile clean-up) [Final-Hawk](https://github.com/Final-Hawk) (Help with NTFY, styling and other fixes), [TeroRERO](https://github.com/terorero) (Spanish translations), [Data-Monkey](https://github.com/Data-Monkey), (Split-up of the python.py file and more), [cvc90](https://github.com/cvc90) (Spanish translation and various UI work) to name a few...
+>
+> Please see the [Git contributors](https://github.com/jokob-sk/Pi.Alert/graphs/contributors) for a full list of people and their contributions to the project
 
 <!--- --------------------------------------------------------------------- --->
 [main]:    ./docs/img/devices_split.png       "Main screen"
@@ -112,7 +95,9 @@ Instructions for [pucherot's original code can be found here](https://github.com
 [screen4]: ./docs/img/maintenance.png         "Screen 4"
 [screen5]: ./docs/img/network.png             "Screen 5"
 [screen6]: ./docs/img/settings.png            "Screen 6"
-[screen7]: ./docs/img/help_faq.png            "Screen 6"
+[screen7]: ./docs/img/help_faq.png            "Screen 7"
+[screen8]: ./docs/img/plugins_rogue_dhcp.png  "Screen 8"
+[screen9]: ./docs/img/device_nmap.png         "Screen 9"
 [report1]: ./docs/img/4_report_1.jpg          "Report sample 1"
 [report2]: ./docs/img/4_report_2.jpg          "Report sample 2"
 [main_dark]: /docs/img/1_devices_dark.jpg     "Main screen dark"

back/pialert.conf

@@ -15,15 +15,15 @@
 #
 # Scan multiple interfaces (eth1 and eth0):
 # SCAN_SUBNETS    = [ '192.168.1.0/24 --interface=eth1', '192.168.1.0/24 --interface=eth0' ]
+
 SCAN_SUBNETS=['192.168.1.0/24 --interface=eth1']
-PRINT_LOG=False
+
 TIMEZONE='Europe/Berlin'
 PIALERT_WEB_PROTECTION=False
 PIALERT_WEB_PASSWORD='8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92'
-INCLUDED_SECTIONS=['internet','new_devices','down_devices','events']
-SCAN_CYCLE_MINUTES=5
 DAYS_TO_KEEP_EVENTS=90
-REPORT_DASHBOARD_URL='http://pi.alert/'
+# Used for generating links in emails. Make sure not to add a trailing slash!
+REPORT_DASHBOARD_URL='http://pi.alert'
 
 
 # Email
@@ -92,25 +92,6 @@ DDNS_PASSWORD='A0000000B0000000C0000000D0000000'
 DDNS_UPDATE_URL='https://api.dynu.com/nic/update?'
 
 
-# PiHole
-#---------------------------
-# if enabled you need to map '/etc/pihole/pihole-FTL.db' in docker-compose.yml
-PIHOLE_ACTIVE=False
-# if enabled you need to map '/etc/pihole/dhcp.leases' in docker-compose.yml
-DHCP_ACTIVE=False
-
-
-# Pholus
-#---------------------------
-PHOLUS_ACTIVE=False
-PHOLUS_TIMEOUT=20
-PHOLUS_FORCE=False
-PHOLUS_DAYS_DATA=7
-PHOLUS_RUN='once'
-PHOLUS_RUN_TIMEOUT=300
-PHOLUS_RUN_SCHD='0 4 * * *'
-
-
 #-------------------IMPORTANT INFO-------------------#
 #   This file is ingested by a python script, so if  #
 #        modified it needs to use python syntax      #

back/report_sample.html

@@ -0,0 +1,177 @@
+<!-- ---------------------------------------------------------------------------
+   #  Pi.Alert
+   #  Open Source Network Guard / WIFI & LAN intrusion detector 
+   #
+   #  repot_template.html - Back module. Template to email reporting in HTML format
+   #-------------------------------------------------------------------------------
+   #  Puche 2021                GNU GPLv3
+   #--------------------------------------------------------------------------- -->
+   <html>
+    <head></head>
+    <body>
+       <font face=sans-serif>
+          <table align=center width=100% cellpadding=0 cellspacing=0 style="border-radius: 5px;">
+             <tr>
+                <td bgcolor=#3c8dbc  align=center style="padding: 20px 10px 10px 10px; font-size: 30px; font-weight: bold; color:#ffffff; border-top-right-radius: 5px; border-top-left-radius: 5px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
+                   Pi.Alert Report
+                </td>
+             </tr>
+             <tr>
+                <td bgcolor=#2656f1 width=100% align=center style="padding: 20px 10px 10px 10px; font-size: 20px; font-weight: bold; color:#ffffff; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
+                   <a style="color:#ffffff;cursor:pointer;" href="https://github.com/jokob-sk/Pi.Alert/releases">🆕 New version available 🆕</a>
+                </td>
+             </tr>
+             <tr>
+                <td>
+                   <table width=100% border=0 bgcolor=#00c0ef cellpadding=5px cellspacing=0 style="border-collapse: collapse; font-size: 15px; text-align:center; color:#404040">
+                      <tr>
+                         <td width=100%> Report Date: <b>2023-01-30 22:17</b> </td>
+                      </tr>
+                   </table>
+                </td>
+             </tr>
+             <tr>
+                <td bgcolor=#F5F5F5 height=200 valign=top style="padding: 10px">
+                   <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                      <tr>
+                         <th width='120px' style='color:blue; font-size: 12px;' bgcolor='#909090'  >New devices</th>
+                      </tr>
+                      <tr>
+                         <td>
+                            <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                               <tr>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >MAC</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Datetime</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >IP</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Event Type</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Device name</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Comments</th>
+                                  <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Device Vendor</th>
+                               </tr>
+                               <tr>
+                                  <td><a href="http://192.168.1.1:20211/deviceDetails.php?mac=00:00:00:ef:a5:6c">00:00:00:ef:a5:6c</a></td>
+                                  <td>2023-01-30 22:15:09</td>
+                                  <td>192.168.1.1</td>
+                                  <td>New Device</td>
+                                  <td>(name not found)</td>
+                                  <td></td>
+                                  <td></td>
+                               </tr>
+                               <tr>
+                                  <td><a href="http://192.168.1.1:20211/deviceDetails.php?mac=00:00:00:ef:a5:6c">00:00:00:ef:a5:6c</a></td>
+                                  <td>2023-01-30 22:17:59</td>
+                                  <td>192.168.1.82</td>
+                                  <td>New Device</td>
+                                  <td>(name not found)</td>
+                                  <td></td>
+                                  <td></td>
+                               </tr>
+                            </table>
+                         </td>
+                      </tr>
+                   </table>
+                   <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                      <tr>
+                         <th width='120px' style='color:blue; font-size: 12px;' bgcolor='#909090'  >Events</th>
+                      </tr>
+                      <tr>
+                         <td>
+                            <ul>
+                               <li>
+                                  <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                                     <tr>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >MAC</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Datetime</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >IP</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Event Type</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Device name</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Comments</th>
+                                        <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Device Vendor</th>
+                                     </tr>
+                                     <tr>
+                                        <td><a href="http://192.168.1.1:20211/deviceDetails.php?mac=00:00:00:ef:a5:6c">00:00:00:ef:a5:6c</a></td>
+                                        <td>2023-01-30 22:15:09</td>
+                                        <td>192.168.1.92</td>
+                                        <td>Disconnected</td>
+                                        <td>(name not found)</td>
+                                        <td></td>
+                                     </tr>
+                                  </table>
+                               </li>
+                            </ul>
+                         </td>
+                      </tr>
+                   </table>
+                   <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                      <tr>
+                         <th width='120px' style='color:blue; font-size: 12px;' bgcolor='#909090'  >Changed or new ports</th>
+                      </tr>
+                      <tr>
+                         <td>
+                            <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                               <tr>
+                                  <th>new</th>
+                               </tr>                              
+                               <tr>
+                                  <td>
+                                     <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                                        <tr>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Name</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >MAC</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Port</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >State</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Service</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Extra</th>
+                                        </tr>
+                                        <tr>
+                                           <td>New device</td>
+                                           <td><a href="http://192.168.1.1:20211/deviceDetails.php?mac=00:00:00:ef:a5:6c">00:00:00:ef:a5:6c</a></td>
+                                           <td>3263/tcp</td>
+                                           <td>open</td>
+                                           <td>ecolor-imager</td>
+                                           <td></td>
+                                        </tr>
+                                     </table>
+                                  </td>
+                               </tr>
+                               <tr>
+                                  <td>
+                                     <table style="border-collapse: collapse; font-size: 12px; color:#70707" width="100%" cellspacing="0" cellpadding="3px" bordercolor="#C0C0C0" border="1">
+                                        <tr>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Name</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >MAC</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Port</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >State</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Service</th>
+                                           <th width='120px' style='color:#F0F0F0' bgcolor='#909090'  >Extra</th>
+                                        </tr>
+                                        <tr>
+                                           <td>New device</td>
+                                           <td><a href="http://192.168.1.1:20211/deviceDetails.php?mac=00:00:00:ef:a5:6c">00:00:00:ef:a5:6c</a></td>
+                                           <td>3264/tcp</td>
+                                           <td>open</td>
+                                           <td>ccmail</td>
+                                           <td></td>
+                                        </tr>
+                                     </table>
+                                  </td>
+                               </tr>
+                               
+                            </table>
+                         </td>
+                      </tr>
+                   </table>
+             <tr>
+                <td>
+                   <table width=100% bgcolor=#46802e cellpadding=5px cellspacing=0 style="font-size: 13px; font-weight: bold; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px;">
+                      <tr>
+                         <td width=50% style="text-align:center"> Pi.Alert - Synology-NAS</td>
+                      </tr>
+                   </table>
+                </td>
+             </tr>
+          </table>
+       </font>
+    </body>
+ </html>
+ 

back/report_sample.txt

@@ -0,0 +1,50 @@
+Report Date: 2021-12-08 12:30
+Server:      Synology-NAS
+
+New Devices
+----------------------
+Name: 	(name not found)
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.121
+	Time: 	2021-12-08 12:30
+	More Info: 	Micro-Star INTL CO., LTD.
+
+Name: 	(name not found)
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.151
+	Time: 	2021-12-08 12:30
+	More Info: 	Espressif Inc.
+
+
+
+Events
+----------------------
+Name: 	Samsung
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.27
+	Time: 	2021-12-08 12:30
+	Event: 	Connected
+	More Info: 	
+
+Name: 	(name not found)
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.6
+	Time: 	2021-12-08 12:30
+	Event: 	Disconnected
+	More Info: 	
+
+Name: 	Google-Home-Mini
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.16
+	Time: 	2021-12-08 12:30
+	Event: 	Disconnected
+	More Info: 	
+
+Name: 	(name not found)
+	MAC: 	2c:2c:2c:2c:2c:2c
+	IP: 	192.168.1.119
+	Time: 	2021-12-08 12:30
+	Event: 	Disconnected
+	More Info: 	
+
+

back/report_template.html

@@ -1,11 +1,16 @@
-<!-- ---------------------------------------------------------------------------
-#  Pi.Alert
-#  Open Source Network Guard / WIFI & LAN intrusion detector 
-#
-#  repot_template.html - Back module. Template to email reporting in HTML format
-#-------------------------------------------------------------------------------
-#  Puche 2021        pi.alert.application@gmail.com        GNU GPLv3
-#--------------------------------------------------------------------------- -->
+<!--
+#---------------------------------------------------------------------------------#
+#  Pi.Alert                                                                       #
+#  Open Source Network Guard / WIFI & LAN intrusion detector                      #  
+#                                                                                 #
+#  report_template.html - Back module. Template to email reporting in HTML format #
+#---------------------------------------------------------------------------------#
+#    Puche      2021        pi.alert.application@gmail.com   GNU GPLv3            #
+#    jokob-sk   2022        jokob.sk@gmail.com               GNU GPLv3            #
+#    leiweibau  2022        https://github.com/leiweibau     GNU GPLv3            #
+#    cvc90      2023        https://github.com/cvc90         GNU GPLv3            #
+#---------------------------------------------------------------------------------# 
+-->
 
 <html>
 
@@ -16,98 +21,45 @@
     <font face=sans-serif>
     <table align=center width=100% cellpadding=0 cellspacing=0 style="border-radius: 5px;">
         <tr>
-        <td bgcolor=#EFB956  align=center style="padding: 20px 10px 10px 10px; font-size: 30px; font-weight: bold; color:#000000; border-top-right-radius: 5px; border-top-left-radius: 5px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
+        <td bgcolor=#3c8dbc  align=center style="padding: 20px 10px 10px 10px; font-size: 30px; font-weight: bold; color:#ffffff; border-top-right-radius: 5px; border-top-left-radius: 5px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
             Pi.Alert Report
         </td>
         </tr>
         
         <tr>
         <td>
-            <table width=100% border=0 bgcolor=#FFD966 cellpadding=5px cellspacing=0 style="border-collapse: collapse; font-size: 15px; text-align:center; color:#404040">            <tr>
-                <td width=100%> Report Date: <b><REPORT_DATE></b> </td>
+            <table width=100% border=0 bgcolor=#4b99d3 cellpadding=5px cellspacing=0 style="border-collapse: collapse; font-size: 15px; text-align:center; color:#ffffff">            <tr>
+                <td width=100% bgcolor="#3c8dbc"> Report Date: <b><REPORT_DATE></b> </td>
             </tr>
             </table>
         </td>
         </tr>
 
         <tr>
-        <td bgcolor=#F5F5F5 height=200 valign=top style="padding: 10px">
-            <SECTION_INTERNET>
-            <p style="font-size: 24px; font-weight: bold; color:#C04040"> Internet: </p> 
+        <td  height=200 valign=top style="padding: 10px">
             
-            <table width=100% border=1 bordercolor=#C0C0C0 cellpadding=3px cellspacing=0 style="border-collapse: collapse; font-size: 12px; color:#707070">
-                <tr bgcolor=#909090 style="color:#F0F0F0">
-                    <th width=140> Event Type </th>
-                    <th width=130> Datetime </th>
-                    <th width=100> IP </th>
-                    <th> Additional Info </th>
-                </tr>
                 
-                <TABLE_INTERNET>
-            </table>
+                <NEW_DEVICES_TABLE>            
+                <DOWN_DEVICES_TABLE>            
+                <EVENTS_TABLE>                            
+                <PLUGINS_TABLE>
 				
-            <br>
-            </SECTION_INTERNET>
-
-            <SECTION_NEW_DEVICES>
-            <p style="font-size: 14px; font-weight: bold; color:#C04040"> New Devices: </p> 
-
-            <table width=100% border=1 bordercolor=#C0C0C0 cellpadding=3px cellspacing=0 style="border-collapse: collapse; font-size: 12px; color:#707070">
-                <tr bgcolor=#909090 style="color:#F0F0F0">
-                    <th width=150> MAC </th>
-                    <th width=130> Datetime </th>
-                    <th width=100> IP </th>
-                    <th width=140> Device Name </th>
-                    <th> Vendor </th>
-                </tr>
-
-                <TABLE_NEW_DEVICES>
-            </table>
-
-            <br>
-            </SECTION_NEW_DEVICES>
-
-            <SECTION_DEVICES_DOWN>
-            <p style="font-size: 14px; font-weight: bold; color:#C04040"> Devices Down: </p> 
-
-            <table width=100% border=1 bordercolor=#C0C0C0 cellpadding=3px cellspacing=0 style="border-collapse: collapse; font-size: 12px; color:#707070">
-                <tr bgcolor=#909090 style="color:#F0F0F0">
-                    <th width=140> MAC </th>
-                    <th width=130> Datetime </th>
-                    <th width=100> IP </th>
-                    <th> Device Name </th>
-                </tr>
-
-                <TABLE_DEVICES_DOWN>
-            </table>
-
-            <br>
-            </SECTION_DEVICES_DOWN>
-
-            <SECTION_EVENTS>
-            <p style="font-size: 14px; font-weight: bold; color:#409040"> Events: </p> 
-
-            <table width=100% border=1 bordercolor=#C0C0C0 cellpadding=3px cellspacing=0 style="border-collapse: collapse; font-size: 12px; color:#707070">
-                <tr bgcolor=#909090 style="color:#F0F0F0">
-                    <th width=140> MAC </th>
-                    <th width=130> Datetime </th>
-                    <th width=100> IP </th>
-                    <th width=100> Event Type </th>
-                    <th width=140> Device Name </th>
-                    <th> Additional Info </th>
-                </tr>
-
-                <TABLE_EVENTS>
-            </table>
-            </SECTION_EVENTS>
         </td>
         </tr>
         
         <tr>
         <td>
-            <table width=100% bgcolor=#46802e cellpadding=5px cellspacing=0 style="font-size: 13px; font-weight: bold; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px;">
+            <table width=100% bgcolor=#3c8dbc cellpadding=5px cellspacing=0 style="font-size: 13px; font-weight: bold; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px;">
                <tr>
-                    <td width=50% style="text-align:center"> Pi.Alert - <SERVER_NAME></td>                    
+                    <td width=50% style="text-align:center;color: white;" bgcolor="#3c8dbc">
+						<a href="https://github.com/jokob-sk/Pi.Alert" target="_blank" style="color: white">Pi.Alert</a>
+						<a href=".." target="_blank" style="color: white"> (<SERVER_NAME>)</a>
+						<br><span style="display:inline-block;color: white; transform: rotate(180deg)">&copy;</span>2020 Puche (2022+ 
+						<a style="color: white" href="mailto:jokob@duck.com?subject=PiAlert">jokob-sk</a>) | <b>Built on: <BUILD_PIALERT> </b> | <b> Version: <VERSION_PIALERT> </b> |
+					    <a href="https://github.com/jokob-sk/Pi.Alert/tree/main/docs" target="_blank" style="color: white"> 
+						<span>Docs <i class="fa fa-circle-question"></i>
+					    </a><span>
+					</td>                    
                 </tr>
             </table>
         </td>

back/report_template.txt

@@ -1,19 +1,7 @@
 Report Date: <REPORT_DATE>
 Server:      <SERVER_NAME>
-<SECTION_NEW_DEVICES>
-New Devices
-----------------------
-<TABLE_NEW_DEVICES>
-</SECTION_NEW_DEVICES><SECTION_DEVICES_DOWN>
-Devices Down
-----------------------
-<TABLE_DEVICES_DOWN>
-</SECTION_DEVICES_DOWN><SECTION_EVENTS>
-Events
-----------------------
-<TABLE_EVENTS>
-</SECTION_EVENTS><SECTION_INTERNET>
-Internet
-----------------------
-<TABLE_INTERNET>
-</SECTION_INTERNET>
+
+<NEW_DEVICES_TABLE>
+<DOWN_DEVICES_TABLE>
+<EVENTS_TABLE>
+<PLUGINS_TABLE>

back/report_template_new_version.html

@@ -0,0 +1,74 @@
+<!--
+#---------------------------------------------------------------------------------#
+#  Pi.Alert                                                                       #
+#  Open Source Network Guard / WIFI & LAN intrusion detector                      #  
+#                                                                                 #
+# report_template_new_version - Back module. Template to email reporting in text  #
+#---------------------------------------------------------------------------------#
+#    Puche      2021        pi.alert.application@gmail.com   GNU GPLv3            #
+#    jokob-sk   2022        jokob.sk@gmail.com               GNU GPLv3            #
+#    leiweibau  2022        https://github.com/leiweibau     GNU GPLv3            #
+#    cvc90      2023        https://github.com/cvc90         GNU GPLv3            #
+#---------------------------------------------------------------------------------#
+-->
+
+<html>
+
+<head>
+</head>
+
+<body>
+    <font face=sans-serif>
+    <table align=center width=100% cellpadding=0 cellspacing=0 style="border-radius: 5px;">
+        <tr>
+        <td bgcolor=#3c8dbc  align=center style="padding: 20px 10px 10px 10px; font-size: 30px; font-weight: bold; color:#ffffff; border-top-right-radius: 5px; border-top-left-radius: 5px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
+            Pi.Alert Report
+        </td>
+        </tr>
+        <tr> 
+            <td bgcolor=#2656f1 width=100% align=center style="padding: 20px 10px 10px 10px; font-size: 20px; font-weight: bold; color:#ffffff; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2)">
+                <a style="color:#ffffff;cursor:pointer;" href="https://github.com/jokob-sk/Pi.Alert/releases">🆕 New version available 🆕</a>
+            </td>             
+        </tr>
+        
+        <tr>
+        <td>
+            <table width=100% border=0 bgcolor=#4b99d3 cellpadding=5px cellspacing=0 style="border-collapse: collapse; font-size: 15px; text-align:center; color:#ffffff">            <tr>
+                <td width=100% bgcolor="#3c8dbc"> Report Date: <b><REPORT_DATE></b> </td>
+            </tr>
+            </table>
+        </td>
+        </tr>
+
+        <tr>
+        <td  height=200 valign=top style="padding: 10px">            
+                
+                <NEW_DEVICES_TABLE>            
+                <DOWN_DEVICES_TABLE>            
+                <EVENTS_TABLE>                            
+                <PLUGINS_TABLE>
+				
+        </td>
+        </tr>
+        
+        <tr>
+        <td>
+            <table width=100% bgcolor=#3c8dbc cellpadding=5px cellspacing=0 style="font-size: 13px; font-weight: bold; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px;">
+               <tr>
+                    <td width=50% style="text-align:center;color: white;" bgcolor="#3c8dbc">
+						<a href="https://github.com/jokob-sk/Pi.Alert" target="_blank" style="color: white">Pi.Alert</a>
+						<a href=".." target="_blank" style="color: white"> (<SERVER_NAME>)</a>
+						<br><span style="display:inline-block;color: white; transform: rotate(180deg)">&copy;</span>2020 Puche (2022+ 
+						<a style="color: white" href="mailto:jokob@duck.com?subject=PiAlert">jokob-sk</a>) | <b>Built on: <BUILD_PIALERT> </b> | <b> Version: <VERSION_PIALERT> </b> |
+					    <a href="https://github.com/jokob-sk/Pi.Alert/tree/main/docs" target="_blank" style="color: white"> 
+						<span>Docs <i class="fa fa-circle-question"></i>
+					    </a><span>
+					</td>                    
+                </tr>
+            </table>
+        </td>
+        </tr>
+    </table>
+    </font>
+</body>
+</html>

back/update_vendors.sh

@@ -1,4 +1,5 @@
-#!/bin/sh
+#!/usr/bin/env bash
+
 # ------------------------------------------------------------------------------
 #  Pi.Alert
 #  Open Source Network Guard / WIFI & LAN intrusion detector 
@@ -14,43 +15,45 @@
 #    /usr/share/ieee-data
 #    /var/lib/ieee-data
 # ----------------------------------------------------------------------
-
+echo "---------------------------------------------------------"
+echo "[INSTALL]                           Run update_vendors.sh"
+echo "---------------------------------------------------------"
 
 # ----------------------------------------------------------------------
 echo Updating... /usr/share/ieee-data/
-cd /usr/share/ieee-data/
+cd /usr/share/ieee-data/ || { echo "could not enter /usr/share/ieee-data directory"; exit 1; }
 
 sudo mkdir -p 2_backup
-sudo cp *.txt 2_backup
-sudo cp *.csv 2_backup
+sudo cp -- *.txt 2_backup
+sudo cp -- *.csv 2_backup
 echo ""
 echo Download Start
 echo ""
-sudo curl $1  -O https://standards-oui.ieee.org/iab/iab.csv \
-              -O https://standards-oui.ieee.org/iab/iab.txt \
-              -O https://standards-oui.ieee.org/oui28/mam.csv \
-              -O https://standards-oui.ieee.org/iab/iab.txt \
-              -O https://standards-oui.ieee.org/oui28/mam.csv \
-              -O https://standards-oui.ieee.org/oui28/mam.txt \
-              -O https://standards-oui.ieee.org/oui36/oui36.csv \
-              -O https://standards-oui.ieee.org/oui36/oui36.txt \
-              -O https://standards-oui.ieee.org/oui/oui.csv \
-              -O https://standards-oui.ieee.org/oui/oui.txt
+sudo curl "$1" -LO https://standards-oui.ieee.org/oui28/mam.csv \              
+              -LO https://standards-oui.ieee.org/oui28/mam.csv \
+              -LO https://standards-oui.ieee.org/oui28/mam.txt \
+              -LO https://standards-oui.ieee.org/oui36/oui36.csv \
+              -LO https://standards-oui.ieee.org/oui36/oui36.txt \
+              -LO https://standards-oui.ieee.org/oui/oui.csv \
+              -LO https://standards-oui.ieee.org/oui/oui.txt
 echo ""
 echo Download Finished
 
 # ----------------------------------------------------------------------
 echo ""
 echo Updating... /usr/share/arp-scan/
-cd /usr/share/arp-scan
+cd /usr/share/arp-scan || { echo "could not enter /usr/share/arp-scan directory"; exit 1; }
 
 sudo mkdir -p 2_backup
-sudo cp *.txt 2_backup
+sudo cp -- *.txt 2_backup
 
 # Update from /usb/lib/ieee-data
 sudo get-iab -v
 sudo get-oui -v
 
+# make files readable
+sudo chmod +r /usr/share/arp-scan/ieee-oui.txt
+
 # Update from ieee website
 # sudo get-iab -v -u http://standards-oui.ieee.org/iab/iab.txt
 # sudo get-oui -v -u http://standards-oui.ieee.org/oui/oui.txt

back/webhook_json_sample.json

@@ -0,0 +1,122 @@
+[
+  {
+    "headers": {
+      "host": "192.168.1.82:5678",
+      "user-agent": "curl/7.74.0",
+      "accept": "*/*",
+      "content-type": "application/json",
+      "content-length": "872"
+    },
+    "params": {},
+    "query": {},
+    "body": {
+      "username": "Pi.Alert",
+      "text": "There are new notifications",
+      "attachments": [
+        {
+          "title": "Pi.Alert Notifications",
+          "title_link": "",
+          "text": {
+            "new_devices_meta": {
+              "title": "New devices",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
+            "new_devices": [
+              {
+                "MAC": "74:ac:74:ac:74:ac",
+                "Datetime": "2023-01-30 22:15:09",
+                "IP": "192.168.1.1",
+                "Event Type": "New Device",
+                "Device name": "(name not found)",
+                "Comments": null,
+                "Device Vendor": null
+              }
+            ],
+            "down_devices_meta": {
+              "title": "Down devices",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
+            "down_devices": [],
+            "events_meta": {
+              "title": "Events",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
+            "events": [
+              {
+                "MAC": "74:ac:74:ac:74:ac",
+                "Datetime": "2023-01-30 22:15:09",
+                "IP": "192.168.1.92",
+                "Event Type": "Disconnected",
+                "Device name": "(name not found)",
+                "Comments": null,
+                "Device Vendor": null
+              },
+              {
+                "MAC": "74:ac:74:ac:74:ac",
+                "Datetime": "2023-01-30 22:15:09",
+                "IP": "192.168.1.150",
+                "Event Type": "Disconnected",
+                "Device name": "(name not found)",
+                "Comments": null,
+                "Device Vendor": null
+              }
+            ],
+            "plugins_meta": {
+              "title": "Plugins",
+              "columnNames": [
+                "Plugin",
+                "Object_PrimaryID",
+                "Object_SecondaryID",
+                "DateTimeChanged",
+                "Watched_Value1",
+                "Watched_Value2",
+                "Watched_Value3",
+                "Watched_Value4",
+                "Status"
+              ]
+            },
+            "plugins": [
+              {
+                "Index": 138,
+                "Plugin": "INTRSPD",
+                "Object_PrimaryID": "Speedtest",
+                "Object_SecondaryID": "2023-10-08 02:01:16+02:00",
+                "DateTimeCreated": "2023-10-08 02:01:16",
+                "DateTimeChanged": "2023-10-08 02:32:15",
+                "Watched_Value1": "-1",
+                "Watched_Value2": "-1",
+                "Watched_Value3": "null",
+                "Watched_Value4": "null",
+                "Status": "missing-in-last-scan",
+                "Extra": "null",
+                "UserData": "null",
+                "ForeignKey": "null"
+              }
+            ]
+          }
+        }
+      ]
+    }
+  }
+]

config/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore

db/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore

docker-compose.yml

@@ -1,37 +1,62 @@
 version: "3"
 services:
   pialert:
-    build: .
+    privileged: true
+    build:
+      dockerfile: Dockerfile
+      context: .
+      cache_from:
+        - type=registry,ref=docker.io/jokob-sk/pi.alert:buildcache
     container_name: pialert
-    network_mode: "host"
-    restart: unless-stopped
+    network_mode: host
+    # restart: unless-stopped
     volumes:
-      - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
-      # - ${APP_DATA_LOCATION}/pialert/db/pialert.db:/home/pi/pialert/db/pialert.db
-      - ${APP_DATA_LOCATION}/pialert/db:/home/pi/pialert/db
-      # (optional) map an empty file with the name 'setting_darkmode' if you want to force the dark mode on container rebuilt
-      - ${APP_DATA_LOCATION}/pialert/db/setting_darkmode:/home/pi/pialert/db/setting_darkmode
+      - ${APP_DATA_LOCATION}/pialert_dev/config:/home/pi/pialert/config
+      # - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
+      - ${APP_DATA_LOCATION}/pialert_dev/db:/home/pi/pialert/db      
+      # - ${APP_DATA_LOCATION}/pialert/db:/home/pi/pialert/db           
       # (optional) useful for debugging if you have issues setting up the container
       - ${LOGS_LOCATION}:/home/pi/pialert/front/log      
+      # ---------------------------------------------------------------------------
       # DELETE START anyone trying to use this file: comment out / delete BELOW lines, they are only for development purposes
-      - ${DEV_LOCATION}/back/pialert.py:/home/pi/pialert/back/pialert.py
-      - ${DEV_LOCATION}/back/update_vendors.sh:/home/pi/pialert/back/update_vendors.sh
-      - ${DEV_LOCATION}/pholus:/home/pi/pialert/pholus
+      - ${APP_DATA_LOCATION}/pialert/dhcp_samples/dhcp1.leases:/mnt/dhcp1.leases
+      - ${APP_DATA_LOCATION}/pialert/dhcp_samples/dhcp2.leases:/mnt/dhcp2.leases      
+      - ${APP_DATA_LOCATION}/pialert/dhcp_samples/pihole_dhcp_full.leases:/etc/pihole/dhcp.leases      
+      - ${APP_DATA_LOCATION}/pialert/dhcp_samples/pihole_dhcp_2.leases:/etc/pihole/dhcp2.leases      
+      - ${APP_DATA_LOCATION}/pihole/etc-pihole/pihole-FTL.db:/etc/pihole/pihole-FTL.db    
+      - ${DEV_LOCATION}/pialert:/home/pi/pialert/pialert            
       - ${DEV_LOCATION}/dockerfiles:/home/pi/pialert/dockerfiles
-      - ${APP_DATA_LOCATION}/pialert/php.ini:/etc/php/7.4/fpm/php.ini      
+      - ${APP_DATA_LOCATION}/pialert/php.ini:/etc/php/8.2/fpm/php.ini      
+      - ${DEV_LOCATION}/install:/home/pi/pialert/install
       - ${DEV_LOCATION}/front/css:/home/pi/pialert/front/css
+      - ${DEV_LOCATION}/back/update_vendors.sh:/home/pi/pialert/back/update_vendors.sh
+      - ${DEV_LOCATION}/front/lib/AdminLTE:/home/pi/pialert/front/lib/AdminLTE
       - ${DEV_LOCATION}/front/js:/home/pi/pialert/front/js
+      - ${DEV_LOCATION}/dockerfiles/start.sh:/home/pi/pialert/dockerfiles/start.sh
+      - ${DEV_LOCATION}/dockerfiles/user-mapping.sh:/home/pi/pialert/dockerfiles/user-mapping.sh
+      - ${DEV_LOCATION}/install/install.sh:/home/pi/pialert/install/install.sh      
+      - ${DEV_LOCATION}/install/install_dependencies.sh:/home/pi/pialert/install/install_dependencies.sh      
+      - ${DEV_LOCATION}/front/api:/home/pi/pialert/front/api
       - ${DEV_LOCATION}/front/php:/home/pi/pialert/front/php      
       - ${DEV_LOCATION}/front/deviceDetails.php:/home/pi/pialert/front/deviceDetails.php
+      - ${DEV_LOCATION}/front/deviceDetailsTools.php:/home/pi/pialert/front/deviceDetailsTools.php
       - ${DEV_LOCATION}/front/devices.php:/home/pi/pialert/front/devices.php
       - ${DEV_LOCATION}/front/events.php:/home/pi/pialert/front/events.php
+      - ${DEV_LOCATION}/front/plugins.php:/home/pi/pialert/front/plugins.php
+      - ${DEV_LOCATION}/front/pluginsCore.php:/home/pi/pialert/front/pluginsCore.php
       - ${DEV_LOCATION}/front/help_faq.php:/home/pi/pialert/front/help_faq.php
       - ${DEV_LOCATION}/front/index.php:/home/pi/pialert/front/index.php
       - ${DEV_LOCATION}/front/maintenance.php:/home/pi/pialert/front/maintenance.php
       - ${DEV_LOCATION}/front/network.php:/home/pi/pialert/front/network.php
       - ${DEV_LOCATION}/front/presence.php:/home/pi/pialert/front/presence.php
       - ${DEV_LOCATION}/front/settings.php:/home/pi/pialert/front/settings.php      
+      - ${DEV_LOCATION}/front/systeminfo.php:/home/pi/pialert/front/systeminfo.php      
+      - ${DEV_LOCATION}/front/report.php:/home/pi/pialert/front/report.php      
+      - ${DEV_LOCATION}/front/flows.php:/home/pi/pialert/front/flows.php      
+      - ${DEV_LOCATION}/front/donations.php:/home/pi/pialert/front/donations.php      
+      - ${DEV_LOCATION}/front/plugins:/home/pi/pialert/front/plugins      
       # DELETE END anyone trying to use this file: comment out / delete ABOVE lines, they are only for development purposes
+      # ---------------------------------------------------------------------------
     environment:
       - TZ=${TZ}
       - PORT=${PORT}

dockerfiles/README.md

@@ -1,12 +1,13 @@
-[![Docker](https://img.shields.io/github/actions/workflow/status/jokob-sk/Pi.Alert/docker.yml?branch=main&label=Build&logo=GitHub)](https://github.com/jokob-sk/Pi.Alert/actions/workflows/docker.yml)
+[![Docker](https://img.shields.io/github/actions/workflow/status/jokob-sk/Pi.Alert/docker_prod.yml?label=Build&logo=GitHub)](https://github.com/jokob-sk/Pi.Alert/actions/workflows/docker_prod.yml)
 [![GitHub Committed](https://img.shields.io/github/last-commit/jokob-sk/Pi.Alert?color=40ba12&label=Committed&logo=GitHub&logoColor=fff)](https://github.com/jokob-sk/Pi.Alert)
 [![Docker Size](https://img.shields.io/docker/image-size/jokobsk/pi.alert?label=Size&logo=Docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
 [![Docker Pulls](https://img.shields.io/docker/pulls/jokobsk/pi.alert?label=Pulls&logo=docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
 [![Docker Pushed](https://img.shields.io/badge/dynamic/json?color=0aa8d2&logoColor=fff&label=Pushed&query=last_updated&url=https%3A%2F%2Fhub.docker.com%2Fv2%2Frepositories%2Fjokobsk%2Fpi.alert%2F&logo=docker&link=http://left&link=https://hub.docker.com/repository/docker/jokobsk/pi.alert)](https://hub.docker.com/r/jokobsk/pi.alert)
 
-# 🐳 A docker image for Pi.Alert 
+# PiAlert 💻🔍 Network security scanner & notification framework
 
-🐳 [Docker hub](https://registry.hub.docker.com/r/jokobsk/pi.alert) | 📄 [Dockerfile](https://github.com/jokob-sk/Pi.Alert/blob/main/Dockerfile) | 📚 [Docker instructions](https://github.com/jokob-sk/Pi.Alert/blob/main//dockerfiles/README.md) | 🆕 [Release notes](https://github.com/jokob-sk/Pi.Alert/issues/138)
+  | 🐳 [Docker hub](https://registry.hub.docker.com/r/jokobsk/pi.alert) |  📑 [Docker guide](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md) |🆕 [Release notes](https://github.com/jokob-sk/Pi.Alert/releases) | 📚 [All Docs](https://github.com/jokob-sk/Pi.Alert/tree/main/docs) |
+  |----------------------|----------------------| ----------------------|  ----------------------| 
 
 <a href="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/devices_split.png" target="_blank">
   <img src="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/devices_split.png" width="300px" />
@@ -15,10 +16,9 @@
   <img src="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/network.png" width="300px" />
 </a>
 
-
 ## 📕 Basic Usage 
 
-- You will have to run the container on the host network, e.g: 
+- You will have to run the container on the `host` network, e.g: 
 
 ```yaml
 docker run -d --rm --network=host \
@@ -27,65 +27,86 @@ docker run -d --rm --network=host \
   -e TZ=Europe/Berlin \
   -e PORT=20211 \
   jokobsk/pi.alert:latest
-  ```
-- The initial scan can take up-to 15min (with 50 devices and MQTT). Subsequent ones 3 and 5 minutes so wait that long for all of the scans to run.
+```
+- The initial scan can take up to 15min (with 50 devices and MQTT). Subsequent ones 3 and 5 minutes so wait that long for all of the scans to run.
 
 ### Docker environment variables
 
 | Variable | Description | Default |
 | :------------- |:-------------| -----:|
 | `PORT`      |Port of the web interface  |  `20211` |
+| `LISTEN_ADDR`      |Set the specific IP Address for the listener address for the nginx webserver (web interface). This could be useful when using multiple subnets to hide the web interface from all untrusted networks. |  `0.0.0.0` |
 |`TZ` |Time zone to display stats correctly. Find your time zone [here](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)  |  `Europe/Berlin` |
 |`HOST_USER_GID`    |User ID (UID) to map the user in the container to a server user with sufficient read&write permissions on the mapped files   |  `1000` |
 |`HOST_USER_ID` |User Group ID (GID)  to map the user group in the container to a server user group with sufficient read&write permissions on the mapped files    |    `1000` |
 
 ### Docker paths
 
-| | Path | Description |
+| Required | Path | Description |
 | :------------- | :------------- |:-------------| 
-| **Required** | `:/home/pi/pialert/config` | Folder which will contain the `pialert.conf` file (see below for details)  | 
-| **Required** | `:/home/pi/pialert/db` | Folder which will contain the `pialert.db` file  | 
-|Optional| `:/home/pi/pialert/db/setting_darkmode` |  Map an empty file with the name `setting_darkmode` if you want to force the dark mode on container rebuilt  | 
-|Optional| `:/home/pi/pialert/front/log` |  Logs folder useful for debugging if you have issues setting up the container  | 
-|Optional| `:/etc/pihole/pihole-FTL.db` |  PiHole's `pihole-FTL.db` database file. Required if you want to use PiHole  | 
-|Optional| `:/etc/pihole/dhcp.leases` |  PiHole's `dhcp.leases` file. Required if you want to use PiHole  | 
+| ✅ | `:/home/pi/pialert/config` | Folder which will contain the `pialert.conf` file (see below for details)  | 
+| ✅ | `:/home/pi/pialert/db` | Folder which will contain the `pialert.db` file  | 
+| | `:/home/pi/pialert/front/log` |  Logs folder useful for debugging if you have issues setting up the container  | 
+| | `:/etc/pihole/pihole-FTL.db` |  PiHole's `pihole-FTL.db` database file. Required if you want to use PiHole  | 
+| | `:/etc/pihole/dhcp.leases` |  PiHole's `dhcp.leases` file. Required if you want to use PiHole `dhcp.leases` file. This has to be matched with a corresponding `DHCPLSS_paths_to_check` setting entry. (the path in the container must contain `pihole`)| 
+| | `:/home/pi/pialert/front/api` |  A simple [API endpoint](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md) containing static (but regularly updated) json and other files.   | 
+| | `:/home/pi/pialert/front/plugins/<plugin>/ignore_plugin` | Map a file `ignore_plugin` to ignore a plugin. Plugins can be soft-disabled via settings. More in the [Plugin docs](/front/plugins/README.md).  | 
 
 
-### Config (`pialert.conf`)
+### Modify the config (`pialert.conf`) only if UI is not available
 
-- Modify [pialert.conf](https://github.com/jokob-sk/Pi.Alert/tree/main/config) or manage the configuration via Settings.  
-- ❗ Set the `SCAN_SUBNETS` variable. 
-   * The adapter will probably be `eth0` or `eth1`. (Run `iwconfig` to find your interface name(s)) 
-   * Specify the network filter (which **significantly** speeds up the scan process). For example, the filter `192.168.1.0/24` covers IP ranges 192.168.1.0 to 192.168.1.255.
-   * Examples for one and two subnets  (❗ Note the `['...', '...']` format for two or more subnets):
-     * One subnet: `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0']`
-     * Two subnets:  `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0', '192.168.1.0/24 --interface=eth1']` 
+- The preferred way is to manage the configuration via the Settings section in the UI.
+- You can modify [pialert.conf](https://github.com/jokob-sk/Pi.Alert/tree/main/config) directly, if needed.
+- If unavailable, the app generates a default `pialert.conf` and `pialert.db` file on the first run.
+
+#### Important settings
+
+These are the most important settings to get at least some output in your Devices screen. Usually, only one approach is used, but you should be able to combine these approaches.
+
+##### For arp-scan: ARPSCAN_RUN, SCAN_SUBNETS
+
+- ❗ To use the arp-scan method, you need to set the `SCAN_SUBNETS` variable. See the documentation on how [to setup SUBNETS, VLANs & limitations](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/SUBNETS.md) 
+
+##### For pihole: PIHOLE_RUN, DHCPLSS_RUN
+
+There are 2 approaches how to get PiHole devices imported. Via the PiHole import (PIHOLE) plugin or DHCP leases (DHCPLSS) plugin.
+
+**PiHole (Device sync)**
+
+* `PIHOLE_RUN`: You need to map `:/etc/pihole/pihole-FTL.db` in the `docker-compose.yml` file if you enable this setting.
+
+**DHCP Leases (Device import)**
+
+* `DHCPLSS_RUN`: You need to map `:/etc/pihole/dhcp.leases` in the `docker-compose.yml` file if you enable this setting. 
+* The above setting has to be matched with a corresponding `DHCPLSS_paths_to_check` setting entry (the path in the container must contain `pihole` as PiHole uses a different format of the `dhcp.leases` file).
+
+> [!NOTE]
+> It's recommended to use the same schedule interval for all plugins responsible for discovering new devices.
 
 
-### 🛑 **Common issues** 
+#### 🧭 Community guides
+
+> Use the official installation guides at first and use community content as suplementary material. Open an issue if you'd like to add your link to the list 🙏 
+
+- 📄 [How to Install Pi.Alert on Your Synology NAS - Marius hosting (English)](https://mariushosting.com/how-to-install-pi-alert-on-your-synology-nas/) (Updated frequently)
+- 📄 [시놀/헤놀에서 네트워크 스캐너 Pi.Alert Docker로 설치 및 사용하기 (Korean)](https://blog.dalso.org/article/%EC%8B%9C%EB%86%80-%ED%97%A4%EB%86%80%EC%97%90%EC%84%9C-%EB%84%A4%ED%8A%B8%EC%9B%8C%ED%81%AC-%EC%8A%A4%EC%BA%90%EB%84%88-pi-alert-docker%EB%A1%9C-%EC%84%A4%EC%B9%98-%EB%B0%8F-%EC%82%AC%EC%9A%A9) (July 2023)
+- 📄 [网络入侵探测器Pi.Alert (Chinese)](https://codeantenna.com/a/VgUvIAjZ7J) (May 2023)
+- ▶ [Pi.Alert auf Synology & Docker by - Jürgen Barth (German)](https://www.youtube.com/watch?v=-ouvA2UNu-A) (March 2023)
+- ▶ [Top Docker Container for Home Server Security - VirtualizationHowto (English)](https://www.youtube.com/watch?v=tY-w-enLF6Q) (March 2023)
+- ▶ [Pi.Alert or WatchYourLAN can alert you to unknown devices appearing on your WiFi or LAN network - Danie van der Merwe (English)](https://www.youtube.com/watch?v=v6an9QG2xF0) (November 2022)
+
+> Ordered by last update time. 
+ 
+### **Common issues** 
 
 💡 Before creating a new issue, please check if a similar issue was [already resolved](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed). 
 
-**Permissions**
+⚠ Check also common issues and [debugging tips](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md). 
 
-* If facing issues (AJAX errors, can't write to DB, empty screen, etc,) make sure permissions are set correctly, and check the logs under `/home/pi/pialert/front/log`. 
-* To solve permission issues you can also try to create a DB backup and then run a DB Restore via the **Maintenance > Backup/Restore** section.
-* You can try also setting the owner and group of the `pialert.db` by executing the following on the host system: `docker exec pialert chown -R www-data:www-data /home/pi/pialert/db/pialert.db`. 
-* Map to local User and Group IDs. Specify the enviroment variables `HOST_USER_ID` and `HOST_USER_GID` if needed.
-* Map the pialert.db file (⚠ not folder) to `:/home/pi/pialert/db/pialert.db` (see Examples below for details)
+> [!NOTE]
+> You can bulk-update devices via the [CSV import method](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEVICES_BULK_EDITING.md).
 
-**Container restarts / crashes**
-
-* Check the logs for details. Often a required setting for a notification method is missing. 
-
-**unable to resolve host**
-
-* Check that your `SCAN_SUBNETS` variable is using the correct mask and `--interface` as outlined in the instructions above. 
- 
-
-Docker-compose examples can be found below.
-
-## 📄 Examples
+## 📄 docker-compose.yml Examples
 
 ### Example 1
 
@@ -94,14 +115,14 @@ version: "3"
 services:
   pialert:
     container_name: pialert
+    # use the below line if you want to test the latest dev image
+    # image: "jokobsk/pi.alert_dev:latest" 
     image: "jokobsk/pi.alert:latest"      
     network_mode: "host"        
     restart: unless-stopped
     volumes:
       - local/path/pialert/config:/home/pi/pialert/config
       - local/path/pialert/db:/home/pi/pialert/db      
-      # (optional) map an empty file with the name 'setting_darkmode' if you want to force the dark mode on container rebuilt
-      - local/path/pialert/db/setting_darkmode:/home/pi/pialert/db/setting_darkmode
       # (optional) useful for debugging if you have issues setting up the container
       - local/path/logs:/home/pi/pialert/front/log
     environment:
@@ -115,6 +136,29 @@ To run the container execute: `sudo docker-compose up -d`
 
 ### Example 2
 
+Example by [SeimuS](https://github.com/SeimusS).
+
+```yaml
+  pialert:
+    container_name: PiAlert
+    hostname: PiAlert
+    privileged: true
+    # use the below line if you want to test the latest dev image
+    # image: "jokobsk/pi.alert_dev:latest" 
+    image: jokobsk/pi.alert:latest
+    environment:
+      - TZ=Europe/Bratislava
+    restart: always
+    volumes:
+      - ./pialert/pialert_db:/home/pi/pialert/db
+      - ./pialert/pialert_config:/home/pi/pialert/config
+    network_mode: host
+```
+
+To run the container execute: `sudo docker-compose up -d`
+
+### Example 3
+
 `docker-compose.yml` 
 
 ```yaml
@@ -122,14 +166,14 @@ version: "3"
 services:
   pialert:
     container_name: pialert
+    # use the below line if you want to test the latest dev image
+    # image: "jokobsk/pi.alert_dev:latest" 
     image: "jokobsk/pi.alert:latest"      
     network_mode: "host"        
     restart: unless-stopped
     volumes:
       - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
       - ${APP_DATA_LOCATION}/pialert/db/pialert.db:/home/pi/pialert/db/pialert.db      
-      # (optional) map an empty file with the name 'setting_darkmode' if you want to force the dark mode on container rebuilt
-      - ${APP_DATA_LOCATION}/pialert/db/setting_darkmode:/home/pi/pialert/db/setting_darkmode
       # (optional) useful for debugging if you have issues setting up the container
       - ${LOGS_LOCATION}:/home/pi/pialert/front/log
     environment:
@@ -162,12 +206,14 @@ DEV_LOCATION=/path/to/local/source/code
 
 To run the container execute: `sudo docker-compose --env-file /path/to/.env up`
 
-### Example 3
+### Example 4
 
 Courtesy of [pbek](https://github.com/pbek). The volume `pialert_db` is used by the db directory. The two config files are mounted directly from a local folder to their places in the config folder. You can backup the `docker-compose.yaml` folder and the docker volumes folder.
 
 ```yaml
   pialert:
+    # use the below line if you want to test the latest dev image
+    # image: "jokobsk/pi.alert_dev:latest" 
     image: jokobsk/pi.alert
     ports:
       - "80:20211/tcp"
@@ -184,16 +230,21 @@ Courtesy of [pbek](https://github.com/pbek). The volume `pialert_db` is used by
 
 ## 🏅 Recognitions
 
-Big thanks to <a href="https://github.com/Macleykun">@Macleykun</a> for help and tips&tricks for Dockerfile(s):
+Big thanks to <a href="https://github.com/Macleykun">@Macleykun</a> for help and tips&tricks for Dockerfile(s).
 
-<a href="https://github.com/Macleykun">
-  <img src="https://avatars.githubusercontent.com/u/26381427?size=50"> 
-</a>
+## ❤ Support me
 
-## ☕ Support me
+Get:
+- Regular updates to keep your data and family safe 🔄 
+- Better and more functionality➕
+- I don't get burned out and the app survives longer🔥🤯
+- Quicker and better support with issues 🆘
+- Less grumpy me 😄
 
-Disclaimer: Please only donate if you don't have any debt yourself. Support yourself first, then others.
+| [![GitHub](https://i.imgur.com/emsRCPh.png)](https://github.com/sponsors/jokob-sk) | [![Buy Me A Coffee](https://i.imgur.com/pIM6YXL.png)](https://www.buymeacoffee.com/jokobsk) | [![Patreon](https://i.imgur.com/MuYsrq1.png)](https://www.patreon.com/user?u=84385063) | 
+| --- | --- | --- | 
 
-<a href="https://github.com/sponsors/jokob-sk" target="_blank"><img src="https://i.imgur.com/X6p5ACK.png" alt="Sponsor Me on GitHub" style="height: 30px !important;width: 117px !important;" width="150px" ></a>
-<a href="https://www.buymeacoffee.com/jokobsk" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 30px !important;width: 117px !important;" width="117px" height="30px" ></a>
-<a href="https://www.patreon.com/user?u=84385063" target="_blank"><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/8/82/Patreon_logo_with_wordmark.svg/512px-Patreon_logo_with_wordmark.svg.png" alt="Support me on patreon" style="height: 30px !important;width: 117px !important;" width="117px" ></a>
+- Bitcoin: `1N8tupjeCK12qRVU2XrV17WvKK7LCawyZM`
+- Ethereum: `0x6e2749Cb42F4411bc98501406BdcD82244e3f9C7`
+
+> 📧 Email me at [jokob@duck.com](mailto:jokob@duck.com?subject=PiAlert) if you want to get in touch or if I should add other sponsorship platforms.

dockerfiles/README_ES.md

@@ -0,0 +1,232 @@
+[![Docker](https://img.shields.io/github/actions/workflow/status/jokob-sk/Pi.Alert/docker_prod.yml?label=Build&logo=GitHub)](https://github.com/jokob-sk/Pi.Alert/actions/workflows/docker_prod.yml)
+[![GitHub Committed](https://img.shields.io/github/last-commit/jokob-sk/Pi.Alert?color=40ba12&label=Committed&logo=GitHub&logoColor=fff)](https://github.com/jokob-sk/Pi.Alert)
+[![Docker Size](https://img.shields.io/docker/image-size/jokobsk/pi.alert?label=Tamaño&logo=Docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
+[![Docker Pulls](https://img.shields.io/docker/pulls/jokobsk/pi.alert?label=Pulls&logo=docker&color=0aa8d2&logoColor=fff)](https://hub.docker.com/r/jokobsk/pi.alert)
+[![Docker Pushed](https://img.shields.io/badge/dynamic/json?color=0aa8d2&logoColor=fff&label=Pushed&query=last_updated&url=https%3A%2F%2Fhub.docker.com%2Fv2%2Frepositories%2Fjokobsk%2Fpi.alert%2F&logo=docker&link=http://left&link=https://hub.docker.com/repository/docker/jokobsk/pi.alert)](https://hub.docker.com/r/jokobsk/pi.alert)
+
+# 🐳 Una imagen docker para Pi.Alert
+
+🐳 [Docker hub](https://registry.hub.docker.com/r/jokobsk/pi.alert) | 📑 [Instrucciones para Docker](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md) | 🆕 [Release notes](https://github.com/jokob-sk/Pi.Alert/releases) | 📚 [Todos los Docs](https://github.com/jokob-sk/Pi.Alert/tree/main/docs)
+
+<a href="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/devices_split.png" target="_blank">
+  <img src="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/devices_split.png" width="300px" />
+</a>
+<a href="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/network.png" target="_blank">
+  <img src="https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/network.png" width="300px" />
+</a>
+
+
+## 📕 Uso básico
+
+- Tendrás que ejecutar el contenedor en la red del host, por ejemplo: 
+
+```yaml
+docker run -d --rm --network=host \
+  -v local/path/pialert/config:/home/pi/pialert/config \
+  -v local/path/pialert/db:/home/pi/pialert/db \
+  -e TZ=Europe/Berlin \
+  -e PORT=20211 \
+  jokobsk/pi.alert:latest
+  ```
+- El escaneo inicial puede tardar hasta 15 minutos (con 50 dispositivos y MQTT). Los siguientes pueden durar entre 3 y 5 minutos, así que espere a que se ejecuten todos los escaneos.
+
+### Variables de entorno Docker
+
+| Variable | Descripción | Predeterminado |
+| :------------- |:-------------| -----:|
+| `PORT`      |Puerto de la interfaz web  |  `20211` |
+|`TZ` |Zona horaria para mostrar correctamente las estadísticas. Encuentre su zona horaria [aquí](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)  |  `Europe/Berlin` |
+|`HOST_USER_GID`    |ID de usuario (UID) para asignar el usuario del contenedor a un usuario del servidor con suficientes permisos de lectura y escritura en los archivos asignados   |  `1000` |
+|`HOST_USER_ID` |ID de grupo de usuarios (GID) para asignar el grupo de usuarios del contenedor a un grupo de usuarios del servidor con suficientes permisos de lectura y escritura en los archivos asignados    |    `1000` |
+
+### Rutas Docker
+
+| | Ruta | Descripción |
+| :------------- | :------------- |:-------------| 
+| **Obligatorio** | `:/home/pi/pialert/config` | Carpeta que contendrá el archivo `pialert.conf` (para más detalles, véase más abajo)  | 
+| **Obligatorio** | `:/home/pi/pialert/db` | Carpeta que contendrá el archivo `pialert.db`  | 
+|Opcional| `:/home/pi/pialert/front/log` |  Carpeta de registros útil para depurar si tiene problemas al configurar el contenedor  | 
+|Opcional| `:/etc/pihole/pihole-FTL.db` |  Archivo de base de datos `pihole-FTL.db` de PiHole. Necesario si desea utilizar PiHole  | 
+|Opcional| `:/etc/pihole/dhcp.leases` |  Archivo `dhcp.leases` de PiHole. Obligatorio si desea utilizar el archivo `dhcp.leases` de PiHole. Tiene que coincidir con la correspondiente entrada de configuración `DHCPLSS_paths_to_check`. (La ruta en el contenedor debe contener `pihole`)| 
+|Opcional| `:/home/pi/pialert/front/api` |  Una simple [API endpoint](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md) que contiene archivos json estáticos (pero actualizados regularmente) y otros archivos.   | 
+
+
+### Configurar (`pialert.conf`)
+
+- Si no está disponible, la aplicación genera un archivo `pialert.conf` y `pialert.db` por defecto en la primera ejecución.
+- La forma preferida es gestionar la configuración a través de la sección "Configuración" de la interfaz de usuario.
+- Puede modificar [pialert.conf](https://github.com/jokob-sk/Pi.Alert/tree/main/config) directamente, si es necesario.
+
+#### Ajustes importantes
+
+Estos son los ajustes más importantes para obtener al menos alguna salida en la pantalla de tus Dispositivos. Por lo general, sólo se utiliza un enfoque, pero usted debe ser capaz de combinar estos enfoques.
+
+##### Para arp-scan: ARPSCAN_RUN, SCAN_SUBNETS
+
+- ❗ Para usar el método arp-scan, necesitas configurar la variable `SCAN_SUBNETS`. Consulte la documentación sobre cómo [configurar SUBNETS, VLANs y limitaciones](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/SUBNETS.md) 
+
+##### Para pihole: PIHOLE_RUN, DHCPLSS_RUN
+
+Hay dos maneras de importar dispositivos PiHole. A través del plugin de importación PiHole (PIHOLE) o del plugin DHCP leases (DHCPLSS).
+
+**PiHole (Sincronización de dispositivos)**
+
+* `PIHOLE_RUN`: Necesitas mapear `:/etc/pihole/pihole-FTL.db` en el fichero `docker-compose.yml` si activas esta opción.
+
+**DHCP Leases (Importación de dispositivos)**
+
+* `DHCPLSS_RUN`: Es necesario mapear `:/etc/pihole/dhcp.leases` en el fichero `docker-compose.yml` si se activa esta opción. 
+* La configuración anterior tiene que coincidir con una entrada de configuración correspondiente `DHCPLSS_paths_to_check` (la ruta en el contenedor debe contener `pihole` ya que PiHole utiliza un formato diferente del archivo `dhcp.leases`).
+
+> Se recomienda utilizar el mismo intervalo de programación para todos los plugins responsables de descubrir nuevos dispositivos.
+
+### **Problemas comunes** 
+
+💡 Antes de crear una nueva incidencia, comprueba si ya se ha resuelto una [incidencia similar](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed). 
+
+⚠ Compruebe también los problemas comunes y los [consejos de depuración](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md). 
+
+## 📄 Ejemplos
+
+### Ejemplo 1
+
+```yaml
+version: "3"
+services:
+  pialert:
+    container_name: pialert
+    # Utilice la siguiente línea si desea probar la última imagen de desarrollo
+    # image: "jokobsk/pi.alert_dev:latest" 
+    image: "jokobsk/pi.alert:latest"      
+    network_mode: "host"        
+    restart: unless-stopped
+    volumes:
+      - local/path/pialert/config:/home/pi/pialert/config
+      - local/path/pialert/db:/home/pi/pialert/db      
+      # (optional) useful for debugging if you have issues setting up the container
+      - local/path/logs:/home/pi/pialert/front/log
+    environment:
+      - TZ=Europe/Berlin      
+      - HOST_USER_ID=1000
+      - HOST_USER_GID=1000
+      - PORT=20211
+```
+
+Para ejecutar el contenedor ejecute: `sudo docker-compose up -d`
+
+### Ejemplo 2
+
+Ejemplo de [SeimuS](https://github.com/SeimusS).
+
+```yaml
+  pialert:
+    container_name: PiAlert
+    hostname: PiAlert
+    privileged: true
+    # Utilice la siguiente línea si desea probar la última imagen de desarrollo
+    # image: "jokobsk/pi.alert_dev:latest" 
+    image: jokobsk/pi.alert:latest
+    environment:
+      - TZ=Europe/Bratislava
+    restart: always
+    volumes:
+      - ./pialert/pialert_db:/home/pi/pialert/db
+      - ./pialert/pialert_config:/home/pi/pialert/config
+    network_mode: host
+```
+
+Para ejecutar el contenedor ejecute: `sudo docker-compose up -d`
+
+### Ejemplo 3
+
+`docker-compose.yml` 
+
+```yaml
+version: "3"
+services:
+  pialert:
+    container_name: pialert
+    # Utilice la siguiente línea si desea probar la última imagen de desarrollo
+    # image: "jokobsk/pi.alert_dev:latest" 
+    image: "jokobsk/pi.alert:latest"      
+    network_mode: "host"        
+    restart: unless-stopped
+    volumes:
+      - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
+      - ${APP_DATA_LOCATION}/pialert/db/pialert.db:/home/pi/pialert/db/pialert.db      
+      # (optional) useful for debugging if you have issues setting up the container
+      - ${LOGS_LOCATION}:/home/pi/pialert/front/log
+    environment:
+      - TZ=${TZ}      
+      - HOST_USER_ID=${HOST_USER_ID}
+      - HOST_USER_GID=${HOST_USER_GID}
+      - PORT=${PORT}
+```
+
+`.env` file
+
+```yaml
+#VARIABLES DE RUTA GLOBAL
+
+APP_DATA_LOCATION=/path/to/docker_appdata
+APP_CONFIG_LOCATION=/path/to/docker_config
+LOGS_LOCATION=/path/to/docker_logs
+
+#VARIABLES DE ENTORNO
+
+TZ=Europe/Paris
+HOST_USER_ID=1000
+HOST_USER_GID=1000
+PORT=20211
+
+#VARIABLES DE DESARROLLO
+
+DEV_LOCATION=/path/to/local/source/code
+```
+
+Para ejecutar el contenedor ejecute: `sudo docker-compose --env-file /path/to/.env up`
+
+### Example 4
+
+Por cortesía de [pbek](https://github.com/pbek). El volumen `pialert_db` es utilizado por el directorio db. Los dos archivos de configuración se montan directamente desde una carpeta local a sus lugares en la carpeta config. Puedes hacer una copia de seguridad de la carpeta `docker-compose.yaml` y de la carpeta docker volumes.
+
+
+```yaml
+  pialert:
+    # Utilice la siguiente línea si desea probar la última imagen de desarrollo
+    # image: "jokobsk/pi.alert_dev:latest" 
+    image: jokobsk/pi.alert
+    ports:
+      - "80:20211/tcp"
+    environment:
+      - TZ=Europe/Vienna
+    networks:
+      local:
+        ipv4_address: 192.168.1.2
+    restart: unless-stopped
+    volumes:
+      - pialert_db:/home/pi/pialert/db
+      - ./pialert/pialert.conf:/home/pi/pialert/config/pialert.conf      
+```
+
+## 🏅 Reconocimientos
+
+Muchas gracias a <a href="https://github.com/Macleykun">@Macleykun</a> por ayudarme en consejos y trucos para Dockerfile(s):
+
+<a href="https://github.com/Macleykun">
+  <img src="https://avatars.githubusercontent.com/u/26381427?size=50"> 
+</a>
+
+Muchas gracias a <a href="https://github.com/cvc90">@cvc90</a> por ayudarme y realizar esta traduccion:
+
+<a href="https://github.com/cvc90">
+  <img src="https://avatars.githubusercontent.com/u/76731844?size=50"> 
+</a>
+
+## ☕ Apóyame
+
+<a href="https://github.com/sponsors/jokob-sk" target="_blank"><img src="https://i.imgur.com/X6p5ACK.png" alt="Sponsor Me on GitHub" style="height: 30px !important;width: 117px !important;" width="150px" ></a>
+<a href="https://www.buymeacoffee.com/jokobsk" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 30px !important;width: 117px !important;" width="117px" height="30px" ></a>
+<a href="https://www.patreon.com/user?u=84385063" target="_blank"><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/8/82/Patreon_logo_with_wordmark.svg/512px-Patreon_logo_with_wordmark.svg.png" alt="Support me on patreon" style="height: 30px !important;width: 117px !important;" width="117px" ></a>
+
+BTC: 1N8tupjeCK12qRVU2XrV17WvKK7LCawyZM

dockerfiles/start.sh

@@ -1,31 +1,144 @@
-#!/bin/sh
-/home/pi/pialert/dockerfiles/user-mapping.sh
+#!/usr/bin/env bash
+
+echo "---------------------------------------------------------"
+echo "[INSTALL]                                    Run start.sh"
+echo "---------------------------------------------------------"
+
+
+INSTALL_DIR=/home/pi  # Specify the installation directory here
+
+# DO NOT CHANGE ANYTHING BELOW THIS LINE!
+WEB_UI_DIR=/var/www/html/pialert
+NGINX_CONFIG_FILE=/etc/nginx/conf.d/pialert.conf
+OUI_FILE="/usr/share/arp-scan/ieee-oui.txt" # Define the path to ieee-oui.txt and ieee-iab.txt
+FILEDB=$INSTALL_DIR/pialert/db/pialert.db
+# DO NOT CHANGE ANYTHING ABOVE THIS LINE!
 
 # if custom variables not set we do not need to do anything
 if [ -n "${TZ}" ]; then    
-  FILECONF=/home/pi/pialert/config/pialert.conf 
+  FILECONF=$INSTALL_DIR/pialert/config/pialert.conf 
   if [ -f "$FILECONF" ]; then
-    sed -ie "s|Europe/Berlin|${TZ}|g" /home/pi/pialert/config/pialert.conf 
+    sed -ie "s|Europe/Berlin|${TZ}|g" $INSTALL_DIR/pialert/config/pialert.conf 
   else 
-    sed -ie "s|Europe/Berlin|${TZ}|g" /home/pi/pialert/back/pialert.conf_bak 
+    sed -ie "s|Europe/Berlin|${TZ}|g" $INSTALL_DIR/pialert/back/pialert.conf_bak 
   fi
 fi
 
+# Check if script is run as root
+if [[ $EUID -ne 0 ]]; then
+    echo "This script must be run as root. Please use 'sudo'." 
+    exit 1
+fi
+
+# Run setup scripts
+echo "[INSTALL] Run setup scripts"
+
+"$INSTALL_DIR/pialert/dockerfiles/user-mapping.sh"
+"$INSTALL_DIR/pialert/install/install_dependencies.sh" # if modifying this file transfer the chanegs into the root Dockerfile as well!
+
+echo "[INSTALL] Setup NGINX"
+
+# Remove default NGINX site if it is symlinked, or backup it otherwise
+if [ -L /etc/nginx/sites-enabled/default ] ; then
+  echo "Disabling default NGINX site, removing sym-link in /etc/nginx/sites-enabled"
+  sudo rm /etc/nginx/sites-enabled/default
+elif [ -f /etc/nginx/sites-enabled/default ]; then
+  echo "Disabling default NGINX site, moving config to /etc/nginx/sites-available"
+  sudo mv /etc/nginx/sites-enabled/default /etc/nginx/sites-available/default.bkp_pialert
+fi
+
+# Clear existing directories and files
+if [ -d $WEB_UI_DIR ]; then
+  echo "Removing existing PiAlert web-UI"
+  sudo rm -R $WEB_UI_DIR
+fi
+
+if [ -f $NGINX_CONFIG_FILE ]; then
+  echo "Removing existing PiAlert NGINX config"
+  sudo rm $NGINX_CONFIG_FILE
+fi
+
+# create symbolic link to the pialert install directory
+ln -s $INSTALL_DIR/pialert/front $WEB_UI_DIR
+# create symbolic link to NGINX configuaration coming with PiAlert
+sudo ln -s "$INSTALL_DIR/pialert/install/pialert.conf" /etc/nginx/conf.d/pialert.conf
+
+# Use user-supplied port if set
 if [ -n "${PORT}" ]; then
-  sed -ie 's/listen 20211/listen '${PORT}'/g' /etc/nginx/sites-available/default
+  echo "Setting webserver to user-supplied port ($PORT)"
+  sudo sed -i 's/listen 20211/listen '"$PORT"'/g' /etc/nginx/conf.d/pialert.conf
 fi
 
-# I hope this will fix DB permission issues going forward
-FILEDB=/home/pi/pialert/db/pialert.db
+# Change web interface address if set
+if [ -n "${LISTEN_ADDR}" ]; then
+  echo "Setting webserver to user-supplied address ($LISTEN_ADDR)"
+  sed -ie 's/listen /listen '"${LISTEN_ADDR}":'/g' /etc/nginx/conf.d/pialert.conf
+fi
+
+# Run the hardware vendors update at least once
+echo "[INSTALL] Run the hardware vendors update"
+
+# Check if ieee-oui.txt or ieee-iab.txt exist
+if [ -f "$OUI_FILE" ]; then
+  echo "The file ieee-oui.txt exists. Skipping update_vendors.sh..."
+else
+  echo "The file ieee-oui.txt does not exist. Running update_vendors..."
+
+  # Run the update_vendors.sh script
+  if [ -f "$INSTALL_DIR/pialert/back/update_vendors.sh" ]; then
+    "$INSTALL_DIR/pialert/back/update_vendors.sh"
+  else
+    echo "update_vendors.sh script not found in $INSTALL_DIR."    
+  fi
+fi
+
+# Fixing file permissions
+echo "[INSTALL] Fixing file permissions"
+
+echo "[INSTALL] Fixing WEB_UI_DIR: $WEB_UI_DIR"
+
+chmod -R a+rwx $WEB_UI_DIR
+
+echo "[INSTALL] Fixing INSTALL_DIR: $INSTALL_DIR"
+
+chmod -R a+rw $INSTALL_DIR/pialert/front/log
+chmod -R a+rwx $INSTALL_DIR
+
+echo "[INSTALL] Copy starter pialert.db and pialert.conf if they don't exist"
+
+# Copy starter pialert.db and pialert.conf if they don't exist
+cp -n "$INSTALL_DIR/pialert/back/pialert.conf" "$INSTALL_DIR/pialert/config/pialert.conf" 
+cp -n "$INSTALL_DIR/pialert/back/pialert.db"  "$FILEDB"
+
+echo "[INSTALL] Fixing permissions after copied starter config & DB"
+
 if [ -f "$FILEDB" ]; then
-    chown -R www-data:www-data /home/pi/pialert/db/pialert.db
+    chown -R www-data:www-data $FILEDB
 fi
 
-chmod -R a+rw /home/pi/pialert/front/log
-chmod -R a+rw /home/pi/pialert/config
+chmod -R a+rwx $INSTALL_DIR # second time after we copied the files
+chmod -R a+rw $INSTALL_DIR/pialert/config
+sudo chgrp -R www-data  $INSTALL_DIR/pialert
 
-/etc/init.d/php7.4-fpm start
+# Check if buildtimestamp.txt doesn't exist
+if [ ! -f "$INSTALL_DIR/pialert/front/buildtimestamp.txt" ]; then
+    # Create buildtimestamp.txt
+    date +%s > "$INSTALL_DIR/pialert/front/buildtimestamp.txt"
+fi
+
+# start PHP
+/etc/init.d/php8.2-fpm start
 /etc/init.d/nginx start
 
-# cron -f
-python /home/pi/pialert/back/pialert.py
+# Start Nginx and your application to start at boot (if needed)
+# systemctl start nginx
+# systemctl enable nginx
+
+# # systemctl enable pi-alert
+# sudo systemctl restart nginx
+
+#  Activate the virtual python environment
+source myenv/bin/activate
+
+# Start the PiAlert python script
+python $INSTALL_DIR/pialert/pialert/

dockerfiles/user-mapping.sh

@@ -1,29 +1,39 @@
-#!/bin/bash
+#!/usr/bin/env bash
+
+echo "---------------------------------------------------------"
+echo "[INSTALL]                             Run user-mapping.sh"
+echo "---------------------------------------------------------"
 
 if [ -z "${USER}" ]; then
   echo "We need USER to be set!"; exit 100
 fi
 
 # if both not set we do not need to do anything
-if [ -z "${HOST_USER_ID}" -a -z "${HOST_USER_GID}" ]; then
+if [ -z "${HOST_USER_ID}" ] && [ -z "${HOST_USER_GID}" ]; then
     echo "Nothing to do here." ; exit 0
 fi
 
-# reset user_?id to either new id or if empty old (still one of above
+# reset user_id to either new id or if empty old (still one of above
 # might not be set)
 USER_ID=${HOST_USER_ID:=$USER_ID}
 USER_GID=${HOST_USER_GID:=$USER_GID}
 
 LINE=$(grep -F "${USER}" /etc/passwd)
 # replace all ':' with a space and create array
-array=( ${LINE//:/ } )
+array=( "${LINE//:/ }" )
 
 # home is 5th element
 USER_HOME=${array[4]}
 
+# print debug output
+echo  USER_ID"  ": "${USER_ID}";
+echo  USER_GID : "${USER_GID}";
+echo  USER_HOME: "${USER_HOME}";
+echo  TZ"       ": "${TZ}";
+
 sed -i -e "s/^${USER}:\([^:]*\):[0-9]*:[0-9]*/${USER}:\1:${USER_ID}:${USER_GID}/"  /etc/passwd
 sed -i -e "s/^${USER}:\([^:]*\):[0-9]*/${USER}:\1:${USER_GID}/"  /etc/group
 
-chown -R ${USER_ID}:${USER_GID} ${USER_HOME}
+chown -R "${USER_ID}:${USER_GID} ${USER_HOME}"
 
 exec su - "${USER}"

docs/API.md

@@ -0,0 +1,98 @@
+## API endpoints
+
+PiAlert comes with a simple API. These API endpoints are static files, that are periodically updated based on your settings. 
+
+
+### When are the endpoints updated
+
+The endpoints are updated when objects in the API endpoints are changed.
+
+### Location of the endpoints
+
+In the container, these files are located under the `/home/pi/pialert/front/api/` folder and thus on the `<pialert_url>/api/<File name>` url.
+
+### Available endpoints
+
+You can access the following files:
+
+  | File name | Description | 
+  |----------------------|----------------------| 
+  | `notification_text.txt` | The plain text version of the last notification. |
+  | `notification_text.html` | The full HTML of the last email notification. |
+  | `notification_json_final.json` | The json version of the last notification (e.g. used for webhooks - [sample JSON](https://github.com/jokob-sk/Pi.Alert/blob/main/back/webhook_json_sample.json)). |
+  | `table_devices.json` | The current (at the time of the last update as mentioned above on this page) state of all of the available Devices detected by the app. |  
+  | `table_pholus_scan.json` | The latest state of the [pholus](https://github.com/jokob-sk/Pi.Alert/tree/main/pholus) (A multicast DNS and DNS Service Discovery Security Assessment Tool) scan results. |
+  | `table_plugins_events.json` | The list of the unprocessed (pending) notification events (plugins_events DB table). |
+  | `table_plugins_history.json` | The list of notification events history. |  
+  | `table_plugins_objects.json` | The content of the plugins_objects table. Find more info on the [Plugin system here](https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins)|
+  | `language_strings.json` | The content of the language_strings table, which in turn is loaded from the plugins `config.json` definitions. |  
+  | `table_custom_endpoint.json` | A custom endpoint generated by the SQL query specified by the `API_CUSTOM_SQL` setting. |
+  | `table_settings.json` | The content of the settings table. |
+  | `app_state.json` | Contains the current application state. |
+  
+  Current/latest state of the aforementioned files depends on your settings.
+
+### JSON Data format
+
+The endpoints starting with the `table_` prefix contain most, if not all, data contained in the corresponding database table. The common format for those is:
+
+```JSON
+{
+  "data": [
+        {
+          "db_column_name": "data",
+          "db_column_name2": "data2"      
+        }, 
+        {
+          "db_column_name": "data3",
+          "db_column_name2": "data4" 
+        }
+    ]
+}
+
+```
+
+Example JSON of the `table_devices.json` endpoint with two Devices (database rows):
+
+```JSON
+{
+  "data": [
+        {
+          "dev_MAC": "Internet",
+          "dev_Name": "Net - Huawei",
+          "dev_DeviceType": "Router",
+          "dev_Vendor": null,
+          "dev_Group": "Always on",
+          "dev_FirstConnection": "2021-01-01 00:00:00",
+          "dev_LastConnection": "2021-01-28 22:22:11",
+          "dev_LastIP": "192.168.1.24",
+          "dev_StaticIP": 0,
+          "dev_PresentLastScan": 1,
+          "dev_LastNotification": "2023-01-28 22:22:28.998715",
+          "dev_NewDevice": 0,
+          "dev_Network_Node_MAC_ADDR": "",
+          "dev_Network_Node_port": "",
+          "dev_Icon": "globe"
+        }, 
+        {
+          "dev_MAC": "a4:8f:ff:aa:ba:1f",
+          "dev_Name": "Net - USG",
+          "dev_DeviceType": "Firewall",
+          "dev_Vendor": "Ubiquiti Inc",
+          "dev_Group": "",
+          "dev_FirstConnection": "2021-02-12 22:05:00",
+          "dev_LastConnection": "2021-07-17 15:40:00",
+          "dev_LastIP": "192.168.1.1",
+          "dev_StaticIP": 1,
+          "dev_PresentLastScan": 1,
+          "dev_LastNotification": "2021-07-17 15:40:10.667717",
+          "dev_NewDevice": 0,
+          "dev_Network_Node_MAC_ADDR": "Internet",
+          "dev_Network_Node_port": 1,
+          "dev_Icon": "shield-halved"
+      }
+    ]
+}
+
+```
+

docs/DATABASE.md

@@ -0,0 +1,39 @@
+  
+  # A high-level description of the datbase structure
+
+  ⚠ Disclaimer: As I'm not the original author, some of the information might be inaccurate. Feel free to submit a PR to correct anything within this page or documentation in general. 
+
+  The MAC address is used as a foreign key in most cases. 
+
+  ## 🔍Tables overview
+  
+  | Table name | Description  | Sample data |
+  |----------------------|----------------------| ----------------------| 
+  | CurrentScan | Result of the current scan | ![Screen1][screen1]  |  
+  | Devices     | The main devices database that also contains the Network tree mappings. If `ScanCycle` is set to `0` device is not scanned. | ![Screen2][screen2]  |   
+  | Events | Used to collect connection/disconnection events. | ![Screen4][screen4]  |   
+  | Online_History   | Used to display the `Device presence` chart  | ![Screen6][screen6]  | 
+  | Parameters       | Used to pass values between the frontend and backend. | ![Screen7][screen7]  | 
+  | Pholus_Scan      | Scan results of the Pholus python network penetration script. | ![Screen8][screen8]  |   
+  | Plugins_Events   | For capturing events exposed by a plugin via the `last_result.log` file. If unique then saved into the `Plugins_Objects` table. Entries are deleted once processed and stored in the `Plugins_History` and/or `Plugins_Objects` tables.  | ![Screen10][screen10]  | 
+  | Plugins_History  | History of all entries from the `Plugins_Events` table | ![Screen11][screen11]  | 
+  | Plugins_Language_Strings  | Language strings colelcted from the plugin `config.json` files used for string resolution in the frontend. | ![Screen12][screen12]  | 
+  | Plugins_Objects  | Unique objects detected by individual plugins. | ![Screen13][screen13]  | 
+  | Sessions  | Used to display sessions in the charts | ![Screen15][screen15]  | 
+  | Settings  | Database representation of the sum of all settings from `pialert.conf` and plugins coming from `config.json` files. | ![Screen16][screen16]  | 
+
+
+
+  [screen1]: /docs/img/DATABASE/CurrentScan.png
+  [screen2]: /docs/img/DATABASE/Devices.png
+  [screen4]: /docs/img/DATABASE/Events.png  
+  [screen6]: /docs/img/DATABASE/Online_History.png
+  [screen7]: /docs/img/DATABASE/Parameters.png
+  [screen8]: /docs/img/DATABASE/Pholus_Scan.png  
+  [screen10]: /docs/img/DATABASE/Plugins_Events.png
+  [screen11]: /docs/img/DATABASE/Plugins_History.png
+  [screen12]: /docs/img/DATABASE/Plugins_Language_Strings.png
+  [screen13]: /docs/img/DATABASE/Plugins_Objects.png  
+  [screen15]: /docs/img/DATABASE/Sessions.png
+  [screen16]: /docs/img/DATABASE/Settings.png
+

docs/DEBUG_INVALID_JSON.md

@@ -0,0 +1,36 @@
+# How to debug the Invalid JSON response error
+
+Check the the HTTP response of the failing backend call by following these steps:
+
+- Open developer console in your browser (usually, e. g. for Chrome, key F12 on the keyboard).
+- Follow the steps in this screenshot: 
+
+![F12DeveloperConsole][F12DeveloperConsole]
+
+- Copy the URL causing the error and enter it in the address bar of your browser directly and hit enter. The copied URLs could look something like this (notice the query strings at the end):
+  - `http://<pialert URL>:20211/api/table_devices.json?nocache=1704141103121`
+  - `http://<pialert URL>:20211/php/server/devices.php?action=getDevicesTotals`
+  - `http://<pialert URL>:20211/php/server/devices.php?action=getDevicesList&status=all`  
+
+- Post the error response in the existing issue thread on GitHub or create a new issue and include the redacted response of the failing query.
+
+For reference, the above queries should return results in the following format:
+
+## First URL:
+
+- Should yield a valid JSON file
+
+## Second URL:
+
+![array][array]
+
+## Third URL:
+
+![json][json]
+
+You can copy and paste any JSON result (result of the First and Third query) into an online JSON checker, such as [this one](https://jsonchecker.com/) to check if it's valid.
+
+
+[F12DeveloperConsole]:    ./img/DEBUG/Invalid_JSON_repsonse_debug.png           "F12DeveloperConsole"
+[array]:                  ./img/DEBUG/array_result_example.png                  "array"
+[json]:                   ./img/DEBUG/JSON_result_example.png                   "json"

docs/DEBUG_PLUGINS.md

@@ -0,0 +1,74 @@
+# Troubleshooting plugins
+
+## High-level overview
+
+If a Plugin supplies data to the main app it's doine either vie a SQL query or via a script that updates the `last_result.log` file in the plugin folder (`front/plugins/<plugin>`).
+
+For a more in-depth overview on how plugins work check the [Plugins development docs](https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README.md).
+
+### Prerequisites
+
+- Make sure you read and followed the specific plugin setup instructions.
+- Ensure you have [debug enabled (see More Logging)](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#1-more-logging-) 
+
+### Potential issues
+
+- Bugs
+- Unexpected input (e.g. special characters in names)
+- Dependencies changed how data is output
+
+#### Incorrect input data
+
+Input data from the plugin might cause mapping issues in specific edge cases. Look for a corresponding section in the `pialert.log` file, for example notice the first line of the execution run of the `PIHOLE` plugin below:
+
+```
+17:31:05 [Scheduler] - Scheduler run for PIHOLE: YES
+17:31:05 [Plugin utils] ---------------------------------------------
+17:31:05 [Plugin utils] display_name: PiHole (Device sync)
+17:31:05 [Plugins] CMD: SELECT n.hwaddr AS Object_PrimaryID, {s-quote}null{s-quote} AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr is not {s-quote}00:00:00:00:00:00{s-quote}  AND na.ip is not null
+17:31:05 [Plugins] setTyp: subnets
+17:31:05 [Plugin utils] Flattening the below array
+17:31:05 ['192.168.1.0/24 --interface=eth1']
+17:31:05 [Plugin utils] isinstance(arr, list) : False | isinstance(arr, str) : True
+17:31:05 [Plugins] Resolved value: 192.168.1.0/24 --interface=eth1
+17:31:05 [Plugins] Convert to Base64: True
+17:31:05 [Plugins] base64 value: b'MTkyLjE2OC4xLjAvMjQgLS1pbnRlcmZhY2U9ZXRoMQ=='
+17:31:05 [Plugins] Timeout: 10
+17:31:05 [Plugins] Executing: SELECT n.hwaddr AS Object_PrimaryID, 'null' AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, 'null' AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE 'ip-%' AND n.hwaddr is not '00:00:00:00:00:00'  AND na.ip is not null
+17:31:05 [Plugins] SUCCESS, received 2 entries
+17:31:05 [Plugins] sqlParam entries: [(0, 'PIHOLE', '01:01:01:01:01:01', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.1', 0, 'aaaa', 'vvvvvvvvv', 'not-processed', 'null', 'null', '01:01:01:01:01:01'), (0, 'PIHOLE', '02:42:ac:1e:00:02', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.2', 0, 'dddd', 'vvvvv2222', 'not-processed', 'null', 'null', '02:42:ac:1e:00:02')]
+17:31:05 [Plugins] Processing        : PIHOLE
+17:31:05 [Plugins] Existing objects from Plugins_Objects: 4
+17:31:05 [Plugins] Logged events from the plugin run    : 2
+17:31:05 [Plugins] pluginEvents      count: 2
+17:31:05 [Plugins] pluginObjects     count: 4
+17:31:05 [Plugins] events_to_insert  count: 0
+17:31:05 [Plugins] history_to_insert count: 4
+17:31:05 [Plugins] objects_to_insert count: 0
+17:31:05 [Plugins] objects_to_update count: 4
+17:31:05 [Plugin utils] In pluginEvents there are 2 events with the status "watched-not-changed" 
+17:31:05 [Plugin utils] In pluginObjects there are 2 events with the status "missing-in-last-scan" 
+17:31:05 [Plugin utils] In pluginObjects there are 2 events with the status "watched-not-changed" 
+17:31:05 [Plugins] Mapping objects to database table: CurrentScan
+17:31:05 [Plugins] SQL query for mapping: INSERT into CurrentScan ( "cur_MAC", "cur_IP", "cur_LastQuery", "cur_Name", "cur_Vendor", "cur_ScanMethod") VALUES ( ?, ?, ?, ?, ?, ?)
+17:31:05 [Plugins] SQL sqlParams for mapping: [('01:01:01:01:01:01', '172.30.0.1', 0, 'aaaa', 'vvvvvvvvv', 'PIHOLE'), ('02:42:ac:1e:00:02', '172.30.0.2', 0, 'dddd', 'vvvvv2222', 'PIHOLE')]
+17:31:05 [API] Update API starting
+17:31:06 [API] Updating table_plugins_history.json file in /front/api
+```
+
+In the above output notice the section logging how many events are produced by the plugin:
+
+```
+17:31:05 [Plugins] Existing objects from Plugins_Objects: 4
+17:31:05 [Plugins] Logged events from the plugin run    : 2
+17:31:05 [Plugins] pluginEvents      count: 2
+17:31:05 [Plugins] pluginObjects     count: 4
+17:31:05 [Plugins] events_to_insert  count: 0
+17:31:05 [Plugins] history_to_insert count: 4
+17:31:05 [Plugins] objects_to_insert count: 0
+17:31:05 [Plugins] objects_to_update count: 4
+```
+
+These values, if formatted correctly, will also show up in the UI:
+
+![Plugins table](/docs/img/DEBUG_PLUGINS/plugin_objects_pihole.png)

docs/DEBUG_TIPS.md

@@ -0,0 +1,84 @@
+# Debugging and troubleshooting
+
+Please follow tips 1 - 4 to get a more detailed error. 
+
+## 1. More Logging 📃
+
+When debugging an issue always set the highest log level:
+
+`LOG_LEVEL='debug'`
+
+
+## 2. Surfacing errors when container restarts 🔁
+
+Start the container via the **terminal** with a command similar to this one:
+
+```bash
+docker run --rm --network=host \
+  -v local/path/pialert/config:/home/pi/pialert/config \
+  -v local/path/pialert/db:/home/pi/pialert/db \
+  -e TZ=Europe/Berlin \
+  -e PORT=20211 \
+  jokobsk/pi.alert:latest
+
+```
+
+> ⚠ Please note, don't use the `-d` parameter so you see the error when the container crashes. Use this error in your issue description.
+
+## 3. Check the _dev image and open issues ❓
+
+If possible, check if your issue got fixed in the `_dev` image before opening a new issue. The container is:
+
+`jokobsk/pi.alert_dev:latest`
+
+> ⚠ Please backup your DB and config beforehand!
+
+Please also search [open issues](https://github.com/jokob-sk/Pi.Alert/issues).
+
+## 4. Disable restart behavior 🛑
+
+To prevent a Docker container from automatically restarting in a Docker Compose file, specify the restart policy as `no`:
+
+```yaml
+version: '3'
+
+services:
+  your-service:
+    image: your-image:tag
+    restart: no
+    # Other service configurations...
+```
+
+## 📃Common issues
+
+### Permissions
+
+* If facing issues (AJAX errors, can't write to DB, empty screen, etc,) make sure permissions are set correctly, and check the logs under `/home/pi/pialert/front/log`. 
+* To solve permission issues you can try setting the owner and group of the `pialert.db` by executing the following on the host system: `docker exec pialert chown -R www-data:www-data /home/pi/pialert/db/pialert.db`. 
+* Map to local User and Group IDs. Specify the enviroment variables `HOST_USER_ID` and `HOST_USER_GID` if needed.
+* If still facing issues, try to map the pialert.db file (⚠ not folder) to `:/home/pi/pialert/db/pialert.db` (see [docker-compose Examples](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md#-docker-composeyml-examples) for details)
+
+### Container restarts / crashes
+
+* Check the logs for details. Often a required setting for a notification method is missing. 
+
+### unable to resolve host
+
+* Check that your `SCAN_SUBNETS` variable is using the correct mask and `--interface` as outlined in the instructions above. 
+
+### Invalid JSON
+
+Check the [Invalid JSON errors debug help](/docs/DEBUG_INVALID_JSON.md) docs on how to proceed.
+
+### sudo execution failing (e.g.: on arpscan) on a Raspberry Pi 4 
+
+> sudo: unexpected child termination condition: 0
+
+Resolution based on [this issue](https://github.com/linuxserver/docker-papermerge/issues/4#issuecomment-1003657581)
+
+```
+wget ftp.us.debian.org/debian/pool/main/libs/libseccomp/libseccomp2_2.5.3-2_armhf.deb
+sudo dpkg -i libseccomp2_2.5.3-2_armhf.deb
+```
+
+The link above will probably break in time too. Go to https://packages.debian.org/sid/armhf/libseccomp2/download to find the new version number and put that in the url.

docs/DEVICES_BULK_EDITING.md

@@ -0,0 +1,24 @@
+# 📝Bulk-edit devices via CSV Export/Import
+
+> [!NOTE] 
+> As always, backup everything, just in case.
+
+1. In `Maintenance` > `Backup / Restore` click the `CSV Export` button.  
+2. A `devices.csv` is generated in the `/config` folder
+3. Edit the `devices.csv` file however you like. 
+
+![Maintenance > CSV Export](/docs/img/DEVICES_BULK_EDITING/MAINTENANCE_CSV_EXPORT.png)
+
+> [!NOTE] 
+> The file containing a list of Devices including the Network relationships between Network Nodes and connected devices. You can also trigger this by acessing this URL: `<your pialert url>/php/server/devices.php?action=ExportCSV` or via the `CSV Backup` plugin. (💡 You can schedule this)
+
+![Settings > CSV Backup](/docs/img/DEVICES_BULK_EDITING/CSV_BACKUP_SETTINGS.png)
+
+> [!NOTE] 
+> Keep Linux line endings (sugegsted editors: Nano, Notepad++)
+
+![Nodepad++ line endings](/docs/img/DEVICES_BULK_EDITING/NOTEPAD++.png)
+
+
+
+

docs/DEVICE_MANAGEMENT.md

@@ -9,6 +9,11 @@ To edit device information:
   - Press the "Save" button
 
 
+> [!NOTE] 
+>
+> [Bulk-edit devices](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEVICES_BULK_EDITING.md) by using the `CSV Export` functionality in the `Maintenance` section.
+
+
 ![Device Details][screen1]
 
 
@@ -18,7 +23,7 @@ To edit device information:
   - **Owner**: Device owner (The list is self-populated with existing owners)
   - **Type**: Select a device type from the dropdown list (Smartphone, Table,
       Laptop, TV, router, ....) or type a new device type
-  - **Vendor**: Automatically updated by Pi.Alert
+  - **Vendor**: Automatically updated by Pi.Alert when empty or unknown
   - **Favorite**: Mark the device as favorite and then it will appears at the
       begining of the device list
   - **Group**: Select a grouper ('Always on', 'Personal', Friends') or type
@@ -82,7 +87,15 @@ decides to change the MAC).
   [Read more here](../LICENSE.txt)
 
 ### Contact 
-  pi.alert.application@gmail.com
+
+  Always use the Issue tracker for the correct fork, for example: 
+  
+  [jokob-sk/Pi.Alert](https://github.com/jokob-sk/Pi.Alert/issues). Please also follow the guidelines on:
+
+  - ➕ [Pull Request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-pull-requests-prs) 
+  - 🙏 [Feature request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-feature-requests) 
+  - 🐛 [Issue guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-submitting-an-issue-or-bug)
+
     
   ***Suggestions and comments are welcome***
 

docs/FRONTEND_DEVELOPMENT.md

@@ -0,0 +1,46 @@
+# 🖼 Frontend development 
+
+This page contains tips for frontend development when extending PiAlert. Guiding principles are:
+
+1. Maintainability
+2. Extendability
+3. Reusability
+4. Placing more functionality into Plugins and enhancing core Plugins functionality
+
+That means that, when writing code, focus on reusing what's available instead of writing quick fixes. Or creating reusable functions, instead of bespoke functionaility. 
+
+## 🔍 Examples
+
+Some examples how to apply the above:
+
+> Example 1
+> 
+> I want to implement a scan fucntion. Options would be:
+>
+> 1. To add a manual scan functionality to the `deviceDetails.php` page. 
+> 2. To create a separate page that handles the execution of the scan.
+> 3. To create a configurable Plugin.
+>
+> From the above, number 3 would be the most appropriate solution. Then followed by number 2. Number 1 would be approved only in special circumstances.
+
+> Example 2
+>
+> I want to change the behavior of the application. Options to implement this could be:
+>
+> 1. Hard-code the changes in the code.
+> 2. Implement the changes and add settings to influence the behavior in the `initialize.py` file so the user can adjust these.
+> 3. Implement the changes and add settings via a setting-only plugin.
+> 4. Implement the changes in a way so the behavior can be toggled on each plugin so the core capabilities of Plugins get extended.
+> 
+> From the above, number 4 would be the most appropriate solution. Then followed by number 3. Number 1 or 2 would be approved only in special circumstances.
+
+## 💡 Frontend tips 
+
+Some useful frontend JavaScript functions:
+
+- `getDeviceDataByMacAddress(macAddress, devicesColumn)` - method to retrieve any device data (database column) based on MAC address in the frontend 
+- `getString(string stringKey)` - method to retrieve translated strings in the frontend 
+- `getSetting(string stringKey)` - method to retrieve settings in the frontend 
+
+
+Check the [pialert_common.js](https://github.com/jokob-sk/Pi.Alert/blob/main-2023-06-10/front/js/pialert_common.js) file for more frontend functions.

docs/HOME_ASSISTANT.md

@@ -0,0 +1,42 @@
+# Overview
+
+PiAlert comes with MQTT support, allowing you to show all detected devices as devices in Home Assistant. It also supplies a collection of stats, such as number of online devices.
+
+## ⚠ Note 
+
+- Please note that discovery takes about ~10s per device.
+- Deleting of devices is not handled automatically. Please use [MQTT Explorer](https://mqtt-explorer.com/) to delete devices in the broker (Home Assistant), if needed. 
+
+
+## 🧭 Guide
+
+> 💡 This guide was tested only with the Mosquitto MQTT broker
+
+1. Enable Mosquitto MQTT in Home Assistant by following the [documentation](https://www.home-assistant.io/integrations/mqtt/)
+
+2. Configure a user name and password on your broker.
+
+3. Note down the following details that you will need to configure PiAlert:
+   - MQTT host url (usually your Home Assistant IP)
+   - MQTT broker port
+   - User
+   - Password
+
+4. Ope the `PiAlert` > `Settings` > `MQTT` settings group
+   - Enable MQTT
+   - Fill in the details from above
+   - Fill in remaining settings as per description
+
+
+## 📷 Screenshots
+
+  | ![Screen 1][sensors] | ![Screen 2][history] | 
+  |----------------------|----------------------| 
+  | ![Screen 3][list] | ![Screen 4][overview] | 
+  
+
+  [sensors]:  /docs/img/HOME_ASISSTANT/PiAlert-HomeAssistant-Device-as-Sensors.png       "sensors"
+  [history]:  /docs/img/HOME_ASISSTANT/PiAlert-HomeAssistant-Device-Presence-History.png "history"
+  [list]:     /docs/img/HOME_ASISSTANT/PiAlert-HomeAssistant-Devices-List.png            "list"  
+  [overview]: /docs/img/HOME_ASISSTANT/PiAlert-HomeAssistant-Overview-Card.png           "overview"
+

docs/HW_INSTALL.md

@@ -0,0 +1,49 @@
+# How to install PiAlert on the server hardware
+
+To download and install PiAlert on the hardware/server directly use the `curl` or `wget` commands at the bottom of this page.
+
+> [!NOTE]
+> This is an Experimental feature 🧪 and it relies on community support.
+>
+> There is no guarantee that the install script or any other script will gracefully handle other installed software.
+> Data loss is a possibility, **it is recommended to install PiAlert using the supplied Docker image**.
+
+A warning to the installation method below: Piping to bash is [controversial](https://pi-hole.net/2016/07/25/curling-and-piping-to-bash) and may
+be dangerous, as you cannot see the code that's about to be executed on your system.
+
+Alternatively you can download the installation script `install/install.sh` from the repository and check the code yourself (beware other scripts are
+downloaded too - only from this repo).
+
+PiAlert will be installed in `home/pi/pialert/` and run on port number `20211`.
+
+Some facts about what and where something will be changed/installed by the HW install setup (may not contain everything!):
+
+- `/home/pi/pialert` directory will be deleted and newly created
+- `/home/pi/pialert` will contain the whole repository (downloaded by `install/install.sh`)
+- The default NGINX site `/etc/nginx/sites-enabled/default` will be disabled (sym-link deleted or backed up to `sites-available`)
+- `/var/www/html/pialert` directory will be deleted and newly created
+- `/etc/nginx/conf.d/pialert.conf` will be sym-linked to `/home/pi/pialert/install/pialert.conf`
+- Some files (IEEE device vendors info, ...) will be created in the directory where the installation script is executed
+
+## Limitations
+
+- No system service is provided. PiAlert must be started using `/home/pi/pialert/dockerfiles/start.sh`.
+- No checks for other running software is done.
+- Only tested to work on Debian Bookworm (Debian 12).
+- **EXPERIMENTAL** and not recommended way to install PiAlert.
+
+## 📥 Installation via CURL
+
+```bash
+curl -o install.sh https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/install/install.sh && sudo chmod +x install.sh && sudo ./install.sh
+```
+
+## 📥 Installation via WGET
+
+```bash
+wget https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/install/install.sh -O install.sh && sudo chmod +x install.sh && sudo ./install.sh
+```
+
+These commands will download the `install.sh` script from the GitHub repository, make it executable with `chmod`, and then run it using `./install.sh`.
+
+Make sure you have the necessary permissions to execute the script.

docs/ICONS.md

@@ -0,0 +1,32 @@
+## Icons overview
+
+Icons are used to visually distinguish devices in the app in most of the device listing tables and the [network tree](/docs/NETWORK_TREE.md). Currently only free [Font Awesome](https://fontawesome.com/search?o=r&m=free) icons (up-to v 6.4.0) are supported.
+
+![Raspberry Pi with a brand icon](/docs/img/ICONS/devices-icons.png)
+
+## ⚙ How to use custom device Icons
+
+You can assign icons individually on each device in the Details tab.
+
+![preview](/docs/img/ICONS/device_icons_preview.gif)
+
+![Raspberry Pi device details](/docs/img/ICONS/device-icon.png)
+
+- You can click into the `Icon` field or click the Pencil (2) icon in the above screenshot to enter any text. Only [free Font Awesome](https://fontawesome.com/search?o=r&m=free) icons in the following format will work:
+
+  1. For any value that is only prefixed with `fa-`, you can enter the value directly, such as `server`, `tv`, `ethernet`. 
+  2. If you want to add another classname, e.g. `fa-brands`, you can enter `brands fa-[fontawesome-icon-name]`, so for `apple` that is using the syntax`fa-brands fa-apple`, you would enter `brands fa-apple`.
+
+- If you want to mass-apply an icon to all devices of the same device type (Field marked (4) in the above screenshot), you can click the copy button (Marked (1) in the above screenshot). A confirmation prompt is displayed. If you proceed, icons of all devices set to the same device type as the current device, will be overwritten with the current device's icon.
+
+- The dropdown (3) contains all icons already used in the app for device icons. You need to navigate away or refresh the page once you add a new icon. 
+
+## 🌟 Pro Font Awesome icons
+
+If you own the premium package of Font Awesome icons you can mount it in your Docker container the following way:
+
+```yaml
+/font-awesome:/home/pi/pialert/front/lib/AdminLTE/bower_components/font-awesome:ro
+```
+
+You can use the full range of Font Awesome icons afterwards. 

docs/NETWORK_TREE.md

@@ -0,0 +1,63 @@
+## How to setup your Network page
+
+Make sure you have a root device with the MAC `Internet` (No other MAC addresses are currently supported as the root node).
+
+> 💡 Tip: You can add dummy devices via the [Undiscoverables plugin](https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/undiscoverables/README.md)
+
+> 💡 Tip: Export your configuration of the Network and Devices once in a while via the Export CSV feature under **Maintenance** -> **Backup/Restore** -> **CSV Export**.   
+
+## ⚡Quick setup:
+
+* Go to a Device you want to use as network device (network nodes, such as a Switch). 
+* Set the **Type** of such a device to one of the following: AP, Firewall, Gateway, PLC, Powerline, Router, Switch, USB LAN Adapter, USB WIFI Adapter and WLAN (you can create a custom network type device with in Settings -> General -> `NETWORK_DEVICE_TYPES`).
+* Save and go to Network where the devices you've marked as network devices (by selecting the Type as mentioned above) will show up as tabs.
+* You can now assign the Unassigend devices to the network node.
+* If port is empty or 0 a wifi icon is rendered, otherwise a ethernet port icon.
+
+
+> [!NOTE] 
+>
+> [Bulk-edit devices](/docs/DEVICES_BULK_EDITING.md) by using the `CSV Export` functionality in the `Maintenance` section. You can use this to fix `Internet` node assignment issues. 
+
+## 🔍Detailed example:
+
+In this example you will setup a device named `rapberrypi` as a `Switch` in our network. 
+
+### 1. Device details page
+
+- Go to the `Devices` (1) page:
+
+![Device details](/docs/img/NETWORK_TREE/Device_Details_Network_Type.png)
+
+- In the (2) `Details` tab navigate to the the `Type` (3) dropdown and select the type `Switch` (4).
+
+> Note: Only the following device types will show up as selectable Network nodes ( = devices you can connect other devices to):
+> AP, Firewall, Gateway, Hypervisor, PLC, Powerline, Router, Switch, USB LAN Adapter, USB WIFI Adapter and WLAN. Custom types can be added via the `NETWORK_DEVICE_TYPES` setting.
+
+- Assign a device to your root device from the `Node` (5) dropdown which has the MAC `Internet` (6) (Your name may differ, but the MAC needs to be set to `Internet` - this is done by default). 
+
+- Save your changes (7)
+
+### 2. Network page
+
+- Navigate to your `Network` (1) page:
+
+![Network page](/docs/img/NETWORK_TREE/Network_Page.png)
+
+- Notice the newly added `raspberrypi` (2) tab which now represents a network node, also showing up in the tree (3).
+- As we asssigned the `raspberrypi` in the previous (1) Device details page section to the `Internet` parent network node in step (6), the link is also showing up in the tree diagram (4)
+- We can now assign the device `(AppleTV)` (5) to this `raspberrypi` node, representing a network Switch in this example
+
+### 3. Network page with 2 levels
+
+- After clicking the `Assign` button in the previous section, the `(AppleTV)` (1) device is now connected to our `raspberrypi` (2).
+
+![Network page with 2 levels](/docs/img/NETWORK_TREE/Network_Page_2_Levels.png)
+
+- You can see the `raspberrypi` represents the Network node type `Switch` (3)
+- The `(AppleTV)` to `raspberrypi` connection is also displayed in the table of `Connected devices` (4).
+- You can also see that our `raspberrypi` node is connected to it's Parent network device node with the MAC `Internet` (5). This connection again shows up in the tree (6) as well.
+
+
+
+

docs/RANDOM_MAC.md

@@ -33,7 +33,14 @@ decides to change the MAC).
   [Read more here](../LICENSE.txt)
 
 ### Contact
-  pi.alert.application@gmail.com
+  Always use the Issue tracker for the correct fork, for example: 
+  
+  [jokob-sk/Pi.Alert](https://github.com/jokob-sk/Pi.Alert/issues). Please also follow the guidelines on:
+
+  - ➕ [Pull Request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-pull-requests-prs) 
+  - 🙏 [Feature request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-feature-requests) 
+  - 🐛 [Issue guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-submitting-an-issue-or-bug)
+
   
   ***Suggestions and comments are welcome***
 

docs/README.md

@@ -0,0 +1,129 @@
+## Documentation overview
+
+<details>
+  <summary>:information_source: In the app hover over settings or fields/labels or click blue in-app ❔ (question-mark) icons to get to relevant documentation pages.</summary>
+
+  ![In-app help](/docs/img/GENERAL/in-app-help.png)
+
+</details>
+
+There is also an in-app Help / FAQ section that should be answering frequently asked questions.
+
+### 📥 Installation
+
+#### 🐳 Docker (Fully supported)
+
+- The main installation method is as a [docker container - follow these instructions here](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md). 
+
+#### 💻 Bare-metal / On-server (Experimental/community supported 🧪)
+
+- [(Experimental 🧪) On-hardware instructions](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HW_INSTALL.md) 
+
+- Alternative bare-metal install forks: 
+  - [leiweibau's fork](https://github.com/leiweibau/Pi.Alert/) (maintained)
+  - [pucherot's original code](https://github.com/pucherot/Pi.Alert/) (un-maintained)
+
+### 📚 Table of contents
+
+#### 🐛 Debugging help & tips
+
+- [Debugging tips](/docs/DEBUG_TIPS.md)
+- [Invalid JSON errors debug help](/docs/DEBUG_INVALID_JSON.md)
+- [Troubleshooting Plugins](/docs/DEBUG_PLUGINS.md)
+
+#### 🔝 Popular/Suggested
+
+- [Network treemap configuration](/docs/NETWORK_TREE.md)
+- [SMTP server config](/docs/SMTP.md)
+- [Subnets and VLANs configuration for arp-scan](/docs/SUBNETS.md)
+- [Home Assistant](/docs/HOME_ASSISTANT.md)
+- [Bulk edit devices](/docs/DEVICES_BULK_EDITING.md)
+
+#### ⚙ System Management
+
+- [Manage devices (legacy docs)](/docs/DEVICE_MANAGEMENT.md)
+- [Random MAC/MAC icon meaning (legacy docs)](/docs/RANDOM_MAC.md)
+- [Custom Icon configuration and support](/docs/ICONS.md)
+
+#### 🔎 Examples
+
+- [N8N webhook example](/docs/WEBHOOK_N8N.md)
+
+#### ♻ Misc
+
+- [Version history (legacy)](/docs/VERSIONS_HISTORY.md)
+- [Reverse proxy (Nginx, Apache, SWAG)](/docs/REVERSE_PROXY.md)
+
+#### 👩‍💻For Developers👨‍💻
+
+- [APP code structure](/pialert/README.md)
+- [Database structure](/docs/DATABASE.md)
+- [API endpoints details](/docs/API.md)
+- [Plugin system details and how to develop your own](/front/plugins/README.md)
+- [Settings system](/docs/SETTINGS_SYSTEM.md)
+- [New Version notifications](/docs/VERSIONS.md)
+- [Frontend development tips](/docs/FRONTEND_DEVELOPMENT.md)
+- [Webhook secrets](/docs/WEBHOOK_SECRET.md)
+
+Feel free to suggest or submit new docs via a PR. 
+
+## 👨‍💻 Development priorities
+
+Priorities from highest to lowest:
+
+* 🔼 Fixing core functionality bugs not solvable with workarounds
+* 🔵 New core functionality unlocking other opportunities (e.g.: plugins) 
+* 🔵 Refactoring enabling faster implementation of future functionality 
+* 🔽 (low) UI functionality & improvements (PRs welcome 😉)
+
+Design philosophy: Focus on core functionality and leverage existing apps and tools to make PiAlert integrate into other workflows. 
+
+Examples: 
+
+    1. Supporting apprise makes more sense than implementing multiple individual notification gateways
+    2. Implementing regular expression support across settings for validation makes more sense than validating one setting with a specific expression. 
+
+UI-specific requests are a low priority as the framework picked by the original developer is not very extensible (and afaik doesn't support components) and has limited mobile support. Also, I argue the value proposition is smaller than working on something else.
+
+Feel free to submit PRs if interested. try to **keep the PRs small/on-topic** so they are easier to review and approve. 
+
+That being said, I'd reconsider if more people and or recurring sponsors file a request 😉.
+
+## 🙏 Feature requests
+
+Please be as detailed as possible with **workarounds** you considered and why a native feature is the better way. This gives me better context and will make it more likely to be implemented. Ideally, a feature request should be in the format "I want to be able to do XYZ so that ZYX. I considered these approaches XYZ".
+
+## ➕ Pull requests (PRs)
+
+If you submit a PR please:
+
+1. Check that your changes are backward compatible with existing installations and with a blank setup. 
+2. Existing features should always be preserved. 
+3. Keep the PR small, on-topic and don't change code that is not necessary for the PR to work
+4. New features code should ideally be re-usable for different purposes, not be for a very narrow use-case.
+5. New functionality should ideally be implemented via the Plugins system, if possible.
+
+Suggested test cases:
+
+- Blank setup with no DB or config
+- Existing DB / config
+- Sending a notification (e. g. Delete a device and wait for a scan to run) and testing all notification gateways, especially:
+   - Email, Apprise (e.g. via Telegram), webhook (e.g. via Discord), MQTT (e.g. via Home Assistant)
+- Saving settings
+- Test a couple of plugins
+- Check the Error log for anything unusual
+
+Some additional context:
+
+* Permanent settings/config is stored in the `pialert.conf` file
+* Currently temporary (session?) settings are stored in the `Parameters` DB table as key-value pairs. This table is wiped during a container rebuild/restart and its values are re-initialized from cookies/session data from the browser. 
+
+## 🐛 Submitting an issue or bug
+
+Before submitting a new issue please spend a couple of minutes on research:
+
+* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#common-issues) 
+* Check [💡 Closed issues](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed) if a similar issue was solved in the past.
+* When submitting an issue ❗[enable debug](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md)❗
+
+⚠ Please follow the pre-defined issue template to resolve your issue faster.

docs/REVERSE_PROXY.md

@@ -0,0 +1,479 @@
+# Reverse Proxy Configuration
+
+> Submitted by amazing [cvc90](https://github.com/cvc90) 🙏
+
+## NGINX HTTP Configuration (Direct Path)
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 80; 
+     server_name pi.alert; 
+     proxy_preserve_host on; 
+     proxy_pass http://localhost:20211/; 
+     proxy_pass_reverse http://localhost:20211/; 
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at http://pi.alert/
+
+<br>
+
+## NGINX HTTP Configuration (Sub Path)
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 80; 
+     server_name pi.alert; 
+     proxy_preserve_host on; 
+     location ^~ /pi.alert/ {
+          proxy_pass http://localhost:20211/;
+          proxy_pass_reverse http://localhost:20211/; 
+          proxy_redirect ~^/(.*)$ /pi.alert/$1;
+          rewrite ^/pi.alert/?(.*)$ /$1 break;			
+     }
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at http://pi.alert/pi.alert/
+
+<br>
+
+## NGINX HTTP Configuration (Sub Path) with module ngx_http_sub_module
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 80; 
+     server_name pi.alert; 
+     proxy_preserve_host on; 
+     location ^~ /pi.alert/ {
+          proxy_pass http://localhost:20211/;
+          proxy_pass_reverse http://localhost:20211/; 
+          proxy_redirect ~^/(.*)$ /pi.alert/$1;
+          rewrite ^/pi.alert/?(.*)$ /$1 break;
+	  sub_filter_once off;
+	  sub_filter_types *;
+	  sub_filter 'href="/' 'href="/pi.alert/';
+	  sub_filter '(?>$host)/css' '/pi.alert/css';
+	  sub_filter '(?>$host)/js'  '/pi.alert/js';
+	  sub_filter '/img' '/pi.alert/img';
+	  sub_filter '/lib' '/pi.alert/lib';
+	  sub_filter '/php' '/pi.alert/php';				
+     }
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at http://pi.alert/pi.alert/
+
+<br>
+
+**NGINX HTTPS Configuration (Direct Path)**
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 443; 
+     server_name pi.alert; 
+     SSLEngine On;
+     SSLCertificateFile /etc/ssl/certs/pi.alert.pem;
+     SSLCertificateKeyFile /etc/ssl/private/pi.alert.key;
+     proxy_preserve_host on; 
+     proxy_pass http://localhost:20211/; 
+     proxy_pass_reverse http://localhost:20211/; 
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at https://pi.alert/
+
+<br>
+
+**NGINX HTTPS Configuration (Sub Path)**
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 443; 
+     server_name pi.alert; 
+     SSLEngine On;
+     SSLCertificateFile /etc/ssl/certs/pi.alert.pem;
+     SSLCertificateKeyFile /etc/ssl/private/pi.alert.key;
+     location ^~ /pi.alert/ {
+          proxy_pass http://localhost:20211/;
+          proxy_pass_reverse http://localhost:20211/; 
+          proxy_redirect ~^/(.*)$ /pi.alert/$1;
+          rewrite ^/pi.alert/?(.*)$ /$1 break;		
+     }
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at https://pi.alert/pi.alert/
+
+<br>
+
+## NGINX HTTPS Configuration (Sub Path) with module ngx_http_sub_module
+
+1. On your NGINX server, create a new file called /etc/nginx/sites-available/pialert
+
+2. In this file, paste the following code:
+
+```
+   server { 
+     listen 443; 
+     server_name pi.alert; 
+     SSLEngine On;
+     SSLCertificateFile /etc/ssl/certs/pi.alert.pem;
+     SSLCertificateKeyFile /etc/ssl/private/pi.alert.key;
+     location ^~ /pi.alert/ {
+          proxy_pass http://localhost:20211/;
+          proxy_pass_reverse http://localhost:20211/; 
+          proxy_redirect ~^/(.*)$ /pi.alert/$1;
+          rewrite ^/pi.alert/?(.*)$ /$1 break;
+	  sub_filter_once off;
+	  sub_filter_types *;
+	  sub_filter 'href="/' 'href="/pi.alert/';
+	  sub_filter '(?>$host)/css' '/pi.alert/css';
+	  sub_filter '(?>$host)/js'  '/pi.alert/js';
+	  sub_filter '/img' '/pi.alert/img';
+	  sub_filter '/lib' '/pi.alert/lib';
+	  sub_filter '/php' '/pi.alert/php';		
+     }
+    }
+``` 
+
+3. Activate the new website by running the following command:
+
+   `nginx -s reload` or `systemctl restart nginx`
+
+4. Once NGINX restarts, you should be able to access the proxy website at https://pi.alert/pi.alert/
+
+<br>
+
+## Apache HTTP Configuration (Direct Path)
+
+1. On your Apache server, create a new file called /etc/apache2/sites-available/pialert.conf.
+
+2. In this file, paste the following code:
+
+```
+    <VirtualHost *:80>
+         ServerName pi.alert
+         ProxyPreserveHost On
+         ProxyPass / http://localhost:20211/
+         ProxyPassReverse / http://localhost:20211/
+    </VirtualHost>
+``` 
+
+3. Activate the new website by running the following command:
+
+   `a2ensite pialert` or `service apache2 reload`
+
+4. Once Apache restarts, you should be able to access the proxy website at http://pi.alert/
+
+<br>
+
+## Apache HTTP Configuration (Sub Path)
+
+1. On your Apache server, create a new file called /etc/apache2/sites-available/pialert.conf.
+
+2. In this file, paste the following code:
+
+```
+    <VirtualHost *:80>
+         ServerName pi.alert
+         location ^~ /pi.alert/ {
+               ProxyPreserveHost On
+               ProxyPass / http://localhost:20211/
+               ProxyPassReverse / http://localhost:20211/
+         }
+    </VirtualHost>
+```   
+
+3. Activate the new website by running the following command:
+
+   `a2ensite pialert` or `service apache2 reload`
+
+4. Once Apache restarts, you should be able to access the proxy website at http://pi.alert/
+
+<br>
+
+## Apache HTTPS Configuration (Direct Path)
+
+1. On your Apache server, create a new file called /etc/apache2/sites-available/pialert.conf.
+
+2. In this file, paste the following code:
+
+```
+    <VirtualHost *:443>
+         ServerName pi.alert
+         SSLEngine On
+         SSLCertificateFile /etc/ssl/certs/pi.alert.pem
+         SSLCertificateKeyFile /etc/ssl/private/pi.alert.key
+         ProxyPreserveHost On
+         ProxyPass / http://localhost:20211/
+         ProxyPassReverse / http://localhost:20211/
+    </VirtualHost>
+```   
+
+3. Activate the new website by running the following command:
+
+    `a2ensite pialert` or `service apache2 reload`
+
+4. Once Apache restarts, you should be able to access the proxy website at https://pi.alert/
+
+<br>
+
+## Apache HTTPS Configuration (Sub Path)
+
+1. On your Apache server, create a new file called /etc/apache2/sites-available/pialert.conf.
+
+2. In this file, paste the following code:
+                            
+```
+	<VirtualHost *:443> 
+        ServerName pi.alert
+        SSLEngine On 
+        SSLCertificateFile /etc/ssl/certs/pi.alert.pem
+        SSLCertificateKeyFile /etc/ssl/private/pi.alert.key
+        location ^~ /pi.alert/ {
+              ProxyPreserveHost On
+              ProxyPass / http://localhost:20211/
+              ProxyPassReverse / http://localhost:20211/
+        }
+    </VirtualHost>
+```       
+
+3. Activate the new website by running the following command:
+
+   `a2ensite pialert` or `service apache2 reload`
+
+4. Once Apache restarts, you should be able to access the proxy website at https://pi.alert/pi.alert/
+
+## Reverse proxy example by using LinuxServer's SWAG container.
+
+> Submitted by [s33d1ing](https://github.com/s33d1ing). 🙏
+
+## [linuxserver/swag](https://github.com/linuxserver/docker-swag)
+
+In the SWAG container create `/config/nginx/proxy-confs/pialert.subfolder.conf` with the following contents:
+
+``` nginx
+## Version 2023/02/05
+# make sure that your pialert container is named pialert
+# pialert does not require a base url setting
+
+# Since Pi.Alert uses a Host network, you may need to use the IP address of the system running Pi.Alert for $upstream_app.
+
+location /pialert {
+    return 301 $scheme://$host/pialert/;
+}
+
+location ^~ /pialert/ {
+    # enable the next two lines for http auth
+    #auth_basic "Restricted";
+    #auth_basic_user_file /config/nginx/.htpasswd;
+
+    # enable for ldap auth (requires ldap-server.conf in the server block)
+    #include /config/nginx/ldap-location.conf;
+
+    # enable for Authelia (requires authelia-server.conf in the server block)
+    #include /config/nginx/authelia-location.conf;
+
+    # enable for Authentik (requires authentik-server.conf in the server block)
+    #include /config/nginx/authentik-location.conf;
+
+    include /config/nginx/proxy.conf;
+    include /config/nginx/resolver.conf;
+
+    set $upstream_app pialert;
+    set $upstream_port 20211;
+    set $upstream_proto http;
+
+    proxy_pass $upstream_proto://$upstream_app:$upstream_port;
+    proxy_set_header Accept-Encoding "";
+
+    proxy_redirect ~^/(.*)$ /pialert/$1;
+    rewrite ^/pialert/?(.*)$ /$1 break;
+
+    sub_filter_once off;
+    sub_filter_types *;
+
+    sub_filter 'href="/' 'href="/pialert/';
+
+    sub_filter '(?>$host)/css' '/pialert/css';
+    sub_filter '(?>$host)/js'  '/pialert/js';
+
+    sub_filter '/img' '/pialert/img';
+    sub_filter '/lib' '/pialert/lib';
+    sub_filter '/php' '/pialert/php';
+}
+```
+
+
+## Traefik
+
+> Submitted by [Isegrimm](https://github.com/Isegrimm) 🙏 (based on this [discussion](https://github.com/jokob-sk/Pi.Alert/discussions/449#discussioncomment-7281442))
+
+Asuming the user already has a working Traefik setup, this is what's needed to make Pi.Alert work at a URL like www.domain.com/pialert/. 
+
+Note: Everything in these configs assumes '**www.domain.com**' as your domainname and '**section31**' as an arbitrary name for your certificate setup. You will have to substitute these with your own.
+
+Also, I use the prefix '**pialert**'. If you want to use another prefix, change it in these files: dynamic.toml and default.
+
+Content of my yaml-file (this is the generic Traefik config, which defines which ports to listen on, redirect http to https and sets up the certificate process).
+It also contains Authelia, which I use for authentication.
+This part contains nothing specific to Pi.Alert.
+
+```yaml
+version: '3.8'
+
+services:
+  traefik:
+    image: traefik
+    container_name: traefik
+    command:
+      - "--api=true"
+      - "--api.insecure=true"
+      - "--api.dashboard=true"
+      - "--entrypoints.web.address=:80"
+      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
+      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
+      - "--entrypoints.websecure.address=:443"
+      - "--providers.file.filename=/traefik-config/dynamic.toml"
+      - "--providers.file.watch=true"
+      - "--log.level=ERROR"
+      - "--certificatesresolvers.section31.acme.email=postmaster@domain.com"
+      - "--certificatesresolvers.section31.acme.storage=/traefik-config/acme.json"
+      - "--certificatesresolvers.section31.acme.httpchallenge=true"
+      - "--certificatesresolvers.section31.acme.httpchallenge.entrypoint=web"
+    ports:
+      - "80:80"
+      - "443:443"
+      - "8080:8080"
+    volumes:
+      - "/var/run/docker.sock:/var/run/docker.sock:ro"
+      - /appl/docker/traefik/config:/traefik-config
+    depends_on:
+      - authelia
+    restart: unless-stopped
+  authelia:
+    container_name: authelia
+    image: authelia/authelia:latest
+    ports:
+      - "9091:9091"
+    volumes:
+      - /appl/docker/authelia:/config
+    restart: u
+    nless-stopped
+```
+Snippet of the dynamic.toml file (referenced in the yml-file above) that defines the config for Pi.Alert:
+The following are self-defined keywords, everything else is traefik keywords:
+- pialert-router
+- pialert-service
+- auth
+- pialert-stripprefix
+
+
+```toml
+[http.routers]
+  [http.routers.pialert-router]
+    entryPoints = ["websecure"]
+    rule = "Host(`www.domain.com`) && PathPrefix(`/pialert`)"
+    service = "pialert-service"
+    middlewares = "auth,pialert-stripprefix"
+    [http.routers.pialert-router.tls]
+       certResolver = "section31"
+       [[http.routers.pialert-router.tls.domains]]
+         main = "www.domain.com"
+
+[http.services]
+  [http.services.pialert-service]
+    [[http.services.pialert-service.loadBalancer.servers]]
+      url = "http://internal-ip-address:20211/"
+
+[http.middlewares]
+  [http.middlewares.auth.forwardAuth]
+    address = "http://authelia:9091/api/verify?rd=https://www.domain.com/authelia/"
+    trustForwardHeader = true
+    authResponseHeaders = ["Remote-User", "Remote-Groups", "Remote-Name", "Remote-Email"]
+  [http.middlewares.pialert-stripprefix.stripprefix]
+    prefixes = "/pialert"
+    forceSlash = false
+
+```
+To make Pi.Alert work with this setup I modified the default file at `/etc/nginx/sites-available/default` in the docker container by copying it to my local filesystem, adding the changes as specified by [cvc90](https://github.com/cvc90) and mounting the new file into the docker container, overwriting the original one. By mapping the file instead of changing the file in-place, the changes persist if an updated dockerimage is pulled. This is also a downside when the default file is updated, so I only use this as a temporary solution, until the dockerimage is updated with this change.
+
+Default-file:
+
+```
+server {
+    listen 80 default_server;
+    root /var/www/html;
+    index index.php;
+    #rewrite /pialert/(.*) / permanent;
+    add_header X-Forwarded-Prefix "/pialert" always;
+    proxy_set_header X-Forwarded-Prefix "/pialert";
+
+  location ~* \.php$ {
+    fastcgi_pass unix:/run/php/php8.2-fpm.sock;
+    include         fastcgi_params;
+    fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
+    fastcgi_param   SCRIPT_NAME        $fastcgi_script_name;
+    fastcgi_connect_timeout 75;
+          fastcgi_send_timeout 600;
+          fastcgi_read_timeout 600;
+  }
+}
+```
+
+Mapping the updated file (on the local filesystem at `/appl/docker/pialert/default`) into the docker container:
+
+
+```bash
+docker run -d --rm --network=host \
+  --name=pi.alert \
+  -v /appl/docker/pialert/config:/home/pi/pialert/config \
+  -v /appl/docker/pialert/db:/home/pi/pialert/db \
+  -v /appl/docker/pialert/default:/etc/nginx/sites-available/default \
+  -e TZ=Europe/Amsterdam \
+  -e PORT=20211 \
+  jokobsk/pi.alert:latest
+
+```
+

docs/SETTINGS_SYSTEM.md

@@ -0,0 +1,77 @@
+## ⚙ Setting system
+
+This is an explanation how settings are handled intended for anyone thinking about writing their own plugin or contributing to the project. 
+
+If you are a user of the app, settings should be described in the `Settings` section of the app. Open an issue if you'd like to clarify any of the settings. 
+
+### 🛢 Data storage
+
+The source of truth for user-defined values is the `pialert.conf` file. Editing the file makes the App overwrite values in the `Settings` database table and in the `table_settings.json` file. 
+
+#### Settings database table
+
+The `Settings` database table contains settings for App run purposes. The table is recreated every time the App restarts. The settings are loaded from the source-of-truth, that is the `pialert.conf` file. A high-level overview on the database structure can be found in the [database documentation](/docs/DATABASE.md). 
+
+#### table_settings.json
+
+This is the [API endpoint](/docs/API.md) that reflects the state of the `Settings` database table. Settings can be accessed with the:
+
+* `getSetting(key)` JavaScript method
+
+The json file is also cached on the client-side local storage of the browser.
+
+#### pialert.conf
+
+> [!NOTE] 
+> This is the source of truth for settings. User-defined values in this files always override default values specified in the Plugin definition.
+
+The App generates two `pialert.conf` entries for every setting (Since version 23.8+). One entry is the setting value, the second is the `__metadata` associated with the setting. This `__metadata` entry contains the full setting definition in JSON format. This should helps the future extensibility of the Settings system.
+
+#### Plugin settings
+
+> [!NOTE] 
+> This is the preferred way adding settings going forward. I'll be likely migrating all app settings into plugin-based settings.
+
+Plugin settings are loaded dynamically from the `config.json` of individual plugins. If a setting isn't defined in the `pialert.conf` file, it is initialized via the `default_value` property of a setting from the `config.json` file. Check the [Plugins documentation](/front/plugins/README.md), section `⚙ Setting object structure` for details on the structure of the setting.
+
+![Screen 1][screen1]
+
+### Settings Process flow
+
+The process flow is mostly managed by the [initialise.py](/pialert/initialise.py) file. 
+
+The script is responsible for reading user-defined values from a configuration file (`pialert.conf`), initializing settings, and importing them into a database. It also handles plugins and their configurations.
+
+Here's a high-level description of the code:
+
+1. Function Definitions:
+   - `ccd`: This function is used to handle user-defined settings and configurations. It takes several parameters related to the setting's name, default value, input type, options, group, and more. It saves the settings and their metadata in different lists (`conf.mySettingsSQLsafe` and `conf.mySettings`).
+
+   - `importConfigs`: This function is the main entry point of the script. It imports user settings from a configuration file, processes them, and saves them to the database.
+
+   - `read_config_file`: This function reads the configuration file (`pialert.conf`) and returns a dictionary containing the key-value pairs from the file.
+
+2. Importing Configuration and Initializing Settings:
+   - The `importConfigs` function starts by checking the modification time of the configuration file to determine if it needs to be re-imported. If the file has not been modified since the last import, the function skips the import process.
+
+   - The function reads the configuration file using the `read_config_file` function, which returns a dictionary of settings.
+
+   - The script then initializes various user-defined settings using the `ccd` function, based on the values read from the configuration file. These settings are categorized into groups such as "General," "Email," "Webhooks," "Apprise," and more.
+
+3. Plugin Handling:
+   - The script loads and handles plugins dynamically. It retrieves plugin configurations and iterates through each plugin.
+   - For each plugin, it extracts the prefix and settings related to that plugin and processes them similarly to other user-defined settings.
+   - It also handles scheduling for plugins with specific `RUN_SCHD` settings.
+
+4. Saving Settings to the Database:
+   - The script clears the existing settings in the database and inserts the updated settings into the database using SQL queries.
+
+5. Updating the API and Performing Cleanup:
+   - After importing the configurations, the script updates the API to reflect the changes in the settings.
+   - It saves the current timestamp to determine the next import time.
+   - Finally, it logs the successful import of the new configuration.
+
+
+_____________________
+
+[screen1]: https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/docs/img/plugins_json_settings.png      "Screen 1"

docs/SMTP.md

@@ -0,0 +1,41 @@
+# 📧 SMTP guides
+
+## Using the GMX SMTP server
+
+1. Go to your GMX account https://account.gmx.com
+2. Under Security Options enable 2FA (Two-factor authentication)
+3. Under Security Options generate an Application-specific password
+4. Home -> Email Settings -> POP3 & IMAP -> Enable access to this account via POP3 and IMAP
+5. In PiAlert specify these settings:
+
+```python
+    REPORT_MAIL=True
+    SMTP_SERVER='mail.gmx.com'
+    SMTP_PORT=465
+    SMTP_USER='gmx_email@gmx.com'
+    SMTP_PASS='<your Application-specific password>'
+    SMTP_SKIP_TLS=True
+    SMTP_FORCE_SSL=True
+    SMTP_SKIP_LOGIN=False
+    REPORT_FROM='gmx_email@gmx.com' # this has to be the same email as in SMTP_USER
+    REPORT_TO='some_target_email@gmail.com'
+```
+
+
+## Using the Gmail SMTP server
+    
+1. Create an app password by following the instructions from Google, you need to Enable 2FA for this to work.
+[https://support.google.com/accounts/answer/185833](https://support.google.com/accounts/answer/185833)
+
+2. Specify the following settings:
+
+```python
+    REPORT_MAIL=True
+    SMTP_SKIP_TLS=True
+    SMTP_FORCE_SSL=True 
+    SMTP_PORT=465
+    SMTP_SERVER='smtp.gmail.com'
+    SMTP_PASS='16-digit passcode from google'
+    REPORT_TO='some_target_email@gmail.com'
+```
+

docs/SUBNETS.md

@@ -0,0 +1,100 @@
+# Subnets configuration for arp-scan
+
+You need to specify the network interface and the network mask. You can also configure multiple subnets and specify VLANS (see exceptions below).
+
+## Examples
+
+> [!NOTE] 
+> Please use the UI to configure settings as that ensures that the config file is in the correct format. Edit `pialert.conf` directly only when really necessary.
+> ![settings](/front/plugins/arp_scan/arp-scan-settings.png)
+
+* Examples for one and two subnets  (❗ Note the `['...', '...']` format):
+   * One subnet: `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0']`
+   * Two subnets:  `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0', '192.168.1.0/24 --interface=eth1 -vlan=107']` 
+
+## Explanation
+
+### Network mask
+
+**Example value: `192.168.1.0/24`**
+
+The arp-scan time itself depends on the number of IP addresses to check. 
+
+> The number of IPs to check depends on the [network mask](https://www.calculator.net/ip-subnet-calculator.html) you set on the `SCAN_SUBNETS` setting. 
+> For example, a `/24` mask results in 256 IPs to check, whereas a `/16` mask checks around 65,536. Every IP takes a couple of seconds. This means that with an incorrect configuration, the arp-scan will take hours to complete instead of seconds.
+
+Specify the network filter (which **significantly** speeds up the scan process). For example, the filter `192.168.1.0/24` covers IP ranges 192.168.1.0 to 192.168.1.255.
+
+### Network interface (adapter)
+
+**Example value: `--interface=eth0`**
+
+The adapter will probably be `eth0` or `eth1`. (Check `System info` > `Network Hardware` or run `iwconfig` in the container to find your interface name(s)) 
+
+> Run `iwconfig` in your container to find your interface name(s) (e.g.: `eth0`, `eth1`). 
+
+### VLANs
+
+**Example value: `-vlan=107`**
+
+- Append e.g.: ` -vlan=107` to the interface field (e.g.: `eth0 -vlan=107`) for multiple vlans. More details in this [comment in this issue](https://github.com/jokob-sk/Pi.Alert/issues/170#issuecomment-1419902988)
+
+
+#### VLANs on a Hyper-V setup
+
+> Community sourced content by [mscreations](https://github.com/mscreations) from this [discussion](https://github.com/jokob-sk/Pi.Alert/discussions/404).
+
+> [!NOTE] 
+> The setup this was tested on: Bare Metal -> Hyper-V on Win Server 2019 -> Ubuntu 22.04 VM -> Docker -> PiAlert. 
+
+**Approach 1 (may cause issues):**
+
+Configure multiple network adapters in Hyper-V with distinct VLANs connected to each one using Hyper-V's network setup. However, this action can potentially lead to the Docker host's inability to handle network traffic correctly. The issue may stem from the creation of routes for network time servers or domain controllers on every interface, thereby preventing proper synchronization of the underlying Ubuntu VM. This interference can affect the performance of other applications such as Authentik.
+
+**Approach 2 (working example)**
+
+Network connections to switches are configured as trunk and allow all VLANs access to the server. 
+
+By default Hyper-V only allows untagged packets through to the VM interface and no VLAN tagged packets get through. In order to fix this follow these steps:
+
+1) Run the following command in Powershell on the Hyper-V machine: 
+
+```shell
+Set-VMNetworkAdapterVlan -VMName <Docker VM Name> -Trunk -NativeVlanId 0 -AllowedVlanIdList "<comma separated list of vlans>"
+```
+
+(There might be other ways how adjust this.)
+
+2) Within the VM, set up sub-interfaces for each of the VLANs so they can be scanned. On Ubuntu 22.04 Netplan can be used.
+
+In /etc/netplan/00-installer-config.yaml, add vlan definitions:
+
+```
+network:
+  ethernets:
+    eth0:
+      dhcp4: yes
+  vlans:
+    eth0.2:
+      id: 2
+      link: eth0
+      addresses: [ "192.168.2.2/24" ]
+      routes:
+        - to: 192.168.2.0/24
+          via: 192.168.1.1 
+```
+
+3) Run `sudo netplan apply` and the interfaces are then available to scan in PiAlert. 
+4) In this case, use `192.168.2.0/24 --interface=eth0.2` in PiAlert
+
+#### VLAN 🔍Example:
+
+![Vlan configuration example](/docs/img/SUBNETS/subnets_vlan.png)
+
+#### Support for VLANS (& exceptions)
+
+Please note the accessibility of the macvlans when they are configured on the same computer. My understanding this is a general networking behavior, but feel free to clarify via a PR/issue.
+
+- Pi.Alert does not detect the macvlan container when it is running on the same computer.
+- Pi.Alert recognizes the macvlan container when it is running on a different computer.
+

docs/VERSIONS.md

@@ -0,0 +1,25 @@
+## Am I running the latest released version?
+
+Since version 23.01.14 PiAlert uses a simple timestamp-based version check to verify if a new version is available. You can check the [current and past releases here](https://github.com/jokob-sk/Pi.Alert/releases), or have a look at what I'm [currently working on](https://github.com/jokob-sk/Pi.Alert/issues/138). 
+
+If you are not on the latest version, the app will notify you, that a new released version is avialable the following way:
+
+### 📧 Via email on a notification event
+
+If any notification occurs and an email is sent, the email will contain a note that a new version is available. See the sample email below:
+
+![Sample email if a new version is available](/docs/img/VERSIONS/new-version-available-email.png)
+
+### 🆕 In the UI
+
+In the UI via a notification Icon and via a custom message in the Maintenance section.
+
+![UI screenshot if a new version is available](/docs/img/VERSIONS/new-version-available-maintenance.png)
+
+For a comparison, this is how the UI looks like if you are on the latest stable image:
+
+![UI screenshot if on latest version](/docs/img/VERSIONS/latest-version-maintenance.png)
+
+## Implementation details
+
+During build a [/home/pi/pialert/front/buildtimestamp.txt](https://github.com/jokob-sk/Pi.Alert/blob/092797e75ccfa8359444ad149e727358ac4da05f/Dockerfile#L44) file is created. The app then periodically checks if a new release is available with a newer timestamp in GitHub's rest-based JSON endpoint (check the `def isNewVersion():` method in `pialert.py` for details).   

docs/VERSIONS_HISTORY.md

@@ -77,4 +77,11 @@
   [Read more here](../LICENSE.txt)
 
 ### Contact
-  pi.alert.application@gmail.com
+  Always use the Issue tracker for the correct fork, for example: 
+  
+  [jokob-sk/Pi.Alert](https://github.com/jokob-sk/Pi.Alert/issues). Please also follow the guidelines on:
+
+  - ➕ [Pull Request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-pull-requests-prs) 
+  - 🙏 [Feature request guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-feature-requests) 
+  - 🐛 [Issue guidelines](https://github.com/jokob-sk/Pi.Alert/tree/main/docs#-submitting-an-issue-or-bug)
+

docs/WEBHOOK_N8N.md

@@ -0,0 +1,18 @@
+### Create a simple n8n workflow
+
+![n8n workflow](/docs/img/WEBHOOK_N8N/n8n_workflow.png)
+
+### Specify your email template 
+See [sample JSON](https://github.com/jokob-sk/Pi.Alert/blob/main/back/webhook_json_sample.json) if you want to see the JSON paths used in the email template below
+![Email template](/docs/img/WEBHOOK_N8N/n8n_send_email_settings.png)
+
+```
+Events count: {{ $json["body"]["attachments"][0]["text"]["events"].length }}
+New devices count: {{ $json["body"]["attachments"][0]["text"]["new_devices"].length }}
+```
+
+### Get your webhook in n8n
+![n8n webhook URL](/docs/img/WEBHOOK_N8N/n8n_webhook_settings.png)
+
+### Configure PiAlert to point to the above URL
+![PiAlert config](/docs/img/WEBHOOK_N8N/Webhook_settings.png)

docs/WEBHOOK_SECRET.md

@@ -0,0 +1,38 @@
+# Webhook Secrets
+
+## How does the signing work?
+
+Pi.Alert will use the configured secret to create a hash signature of the request body. This SHA256-HMAC signature will appear in the `X-Webhook-Signature` header of each request to the webhook target URL. You can use the value of this header to validate the request was sent by Pi.Alert.
+
+## Activating webhook signatures
+
+All you need to do in order to add a signature to the request headers is to set the `WEBHOOK_SECRET` config value to a non-empty string.
+
+## Validating webhook deliveries
+
+There are a few things to keep in mind when validating the webhook delivery:
+
+- Pi.Alert uses an HMAC hex digest to compute the hash
+- The signature in the `X-Webhook-Signature` header always starts with `sha256=`
+- The hash signature is generated using the configured `WEBHOOK_SECRET` and the request body.
+- Never use a plain `==` operator. Instead, consider using a method like [`secure_compare`](https://www.rubydoc.info/gems/rack/Rack%2FUtils:secure_compare) or [`crypto.timingSafeEqual`](https://nodejs.org/api/crypto.html#cryptotimingsafeequala-b), which performs a "constant time" string comparison to help mitigate certain timing attacks against regular equality operators, or regular loops in JIT-optimized languages.
+
+## Testing the webhook payload validation
+
+You can use the following secret and payload to verify that your implementation is working correctly.
+
+`secret`: 'this is my secret'
+
+`payload`: '{"test":"this is a test body"}'
+
+If your implementation is correct, the signature you generated should match the following:
+
+`signature`: bed21fcc34f98e94fd71c7edb75e51a544b4a3b38b069ebaaeb19bf4be8147e9
+
+`X-Webhook-Signature`: sha256=bed21fcc34f98e94fd71c7edb75e51a544b4a3b38b069ebaaeb19bf4be8147e9
+
+## More information
+
+If you want to learn more about webhook security, take a look at [GitHub's webhook documentation](https://docs.github.com/en/webhooks/about-webhooks).
+
+You can find examples for validating a webhook delivery [here](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#examples).