dig utility name discovery work #534

update issue template
update_vendors work #533 🔃
2025-12-07 01:26:11 -08:00 · 2024-01-18 17:10:07 +11:00 · 2024-01-16 07:25:17 +11:00 · 2024-01-13 08:56:05 +11:00 · 2024-01-13 08:42:31 +11:00 · 2024-01-08 08:08:51 +11:00
62 changed files with 1179 additions and 675 deletions
--- a/.github/ISSUE_TEMPLATE/feature_request.md
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -1,20 +0,0 @@
---
-name: Feature request
-about: Suggest an idea for this project
-title: ''
-labels: ''
-assignees: ''
-
---
-
-**Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
-
-**Describe the solution you'd like**
-A clear and concise description of what you want to happen.
-
-**Describe alternatives you've considered**
-A clear and concise description of any alternative solutions or features you've considered.
-
-**Additional context**
-Add any other context or screenshots about the feature request here.
--- a/.github/ISSUE_TEMPLATE/feature_request.yml
+++ b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,38 @@
+name: Feature Request
+description: 'Suggest an idea for PiAlert'
+labels: ['Feature request➕']
+body:
+- type: checkboxes
+  attributes:
+    label: Is there an existing issue for this?
+    description: Please search to see if an open or closed issue already exists for the feature you are requesting. 
+    options:
+    - label: I have searched the existing open and closed issues
+      required: true
+- type: textarea
+  attributes:
+    label: Is your feature request related to a problem? Please describe
+    description: A clear and concise description of what the problem is.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Describe the solution you'd like
+    description: A clear and concise description of what you want to happen.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Describe alternatives you've considered
+    description: A clear and concise description of any alternative solutions or features you've considered.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Anything else?
+    description: |
+      Links? References? Mockups? Anything that will give us more context about the feature you are encountering!
+      
+      Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
+  validations:
+    required: true
--- a/.github/ISSUE_TEMPLATE/i-have-an-issue.md
+++ b/.github/ISSUE_TEMPLATE/i-have-an-issue.md
@@ -1,46 +0,0 @@
---
-name: I have an issue
-about: Describe this issue template's purpose here.
-title: ''
-labels: ''
-assignees: ''
-
---
-
-## Describe the issue
-
-> When submitting an issue ❗[enable debug](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md)❗ and [have a look at the docs](https://github.com/jokob-sk/Pi.Alert/tree/main/docs)
-
-[describe your issue]
-
-## Paste your `pialert.conf` (remove personal info)
-
-```
-paste_here
-```
-
-## Paste your `docker-compose.yml` and `.env` (remove personal info)
-
-`docker-compose.yml` 
-
-```
-paste_here
-```
-
-`.env` 
-
-```
-paste_here
-```
-
-## Screenshots
-
-[If applicable, add screenshots to help explain your problem.]
-
-## Paste last few lines from `pialert.log`
-
-> You can use `tail -100 /home/pi/pialert/front/log/pialert.log`
-
-```bash
-
-# paste code below 
--- a/.github/ISSUE_TEMPLATE/i-have-an-issue.yml
+++ b/.github/ISSUE_TEMPLATE/i-have-an-issue.yml
@@ -0,0 +1,76 @@
+name: Bug Report
+description: 'When submitting an issue enable debug and have a look at the docs.'
+labels: ['bug 🐛']
+body:
+- type: checkboxes
+  attributes:
+    label: Is there an existing issue for this?
+    description: Please search to see if an open or closed issue already exists for the bug you encountered.
+    options:
+    - label: I have searched the existing open and closed issues and I checked the docs https://github.com/jokob-sk/Pi.Alert/tree/main/docs
+      required: true
+- type: textarea
+  attributes:
+    label: Current Behavior
+    description: A concise description of what you're experiencing.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Expected Behavior
+    description: A concise description of what you expected to happen.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Steps To Reproduce
+    description: Steps to reproduce the behavior.
+    placeholder: |
+      1. With these settings...
+      2. With this config...
+      3. Run '...'
+      4. See error...
+  validations:
+    required: false
+- type: textarea
+  attributes:
+    label: pialert.conf
+    description: |
+      Paste your `pialert.conf` (remove personal info)    
+    render: python
+  validations:
+    required: false
+- type: textarea
+  attributes:
+    label: docker-compose.yml
+    description: |
+      Paste your `docker-compose.yml`    
+    render: python
+  validations:
+    required: false
+- type: dropdown
+  attributes:
+    label: What branch are you running?
+    options:
+      - Production
+      - Dev
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: pialert.log
+    description: |
+      Logs with debug enabled (https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md) ⚠
+      ***Generally speaking, all bug reports should have logs provided.***
+      Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
+      Additionally, any additional info? Screenshots? References? Anything that will give us more context about the issue you are encountering!
+      You can use `tail -100 /home/pi/pialert/front/log/pialert.log` in teh container if you have troubles getting to the log files. 
+  validations:
+    required: false
+- type: checkboxes
+  attributes:
+    label: Debug enabled
+    description: I confirm I enabled `debug`
+    options:
+    - label: I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.
+      required: true
--- a/.github/workflows/docker_dev.yml
+++ b/.github/workflows/docker_dev.yml
@@ -82,7 +82,7 @@ jobs:
        uses: docker/build-push-action@v3
        with:
          context: .
-          platforms: linux/amd64,linux/arm64,linux/arm/v7
+          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
--- a/.github/workflows/docker_prod.yml
+++ b/.github/workflows/docker_prod.yml
@@ -76,7 +76,7 @@ jobs:
        uses: docker/build-push-action@v3
        with:
          context: .
-          platforms: linux/amd64,linux/arm64,linux/arm/v7
+          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
--- a/2
+++ b/2
@@ -6,7 +6,7 @@ The issue tracker is the preferred channel for bug reports, features requests an

 Before submitting a new issue please spend a couple of minutes on research:

-* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/tree/main/dockerfiles#-common-issues) 
+* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#common-issues) 
 * Check [💡 Closed issues](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed) if a similar issue was solved in the past.

 ## Pull-requests (PRs)
--- a/back/pialert.conf
+++ b/back/pialert.conf
@@ -21,7 +21,6 @@ SCAN_SUBNETS=['192.168.1.0/24 --interface=eth1']
 TIMEZONE='Europe/Berlin'
 PIALERT_WEB_PROTECTION=False
 PIALERT_WEB_PASSWORD='8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92'
-INCLUDED_SECTIONS=['internet','new_devices','down_devices','events']
 DAYS_TO_KEEP_EVENTS=90
 # Used for generating links in emails. Make sure not to add a trailing slash!
 REPORT_DASHBOARD_URL='http://pi.alert'
--- a/back/update_vendors.sh
+++ b/back/update_vendors.sh
@@ -29,10 +29,7 @@ sudo cp -- *.csv 2_backup
 echo ""
 echo Download Start
 echo ""
-sudo curl "$1"  -LO https://standards-oui.ieee.org/iab/iab.csv \
-              -LO https://standards-oui.ieee.org/iab/iab.txt \
-              -LO https://standards-oui.ieee.org/oui28/mam.csv \
-              -LO https://standards-oui.ieee.org/iab/iab.txt \
+sudo curl "$1" -LO https://standards-oui.ieee.org/oui28/mam.csv \              
              -LO https://standards-oui.ieee.org/oui28/mam.csv \
              -LO https://standards-oui.ieee.org/oui28/mam.txt \
              -LO https://standards-oui.ieee.org/oui36/oui36.csv \
--- a/back/webhook_json_sample.json
+++ b/back/webhook_json_sample.json
@@ -17,7 +17,17 @@
          "title": "Pi.Alert Notifications",
          "title_link": "",
          "text": {
-            "internet": [],
+            "new_devices_meta": {
+              "title": "New devices",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
            "new_devices": [
              {
                "MAC": "74:ac:74:ac:74:ac",
@@ -29,7 +39,29 @@
                "Device Vendor": null
              }
            ],
+            "down_devices_meta": {
+              "title": "Down devices",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
            "down_devices": [],
+            "events_meta": {
+              "title": "Events",
+              "columnNames": [
+                "MAC",
+                "Datetime",
+                "IP",
+                "Event Type",
+                "Device name",
+                "Comments"
+              ]
+            },
            "events": [
              {
                "MAC": "74:ac:74:ac:74:ac",
@@ -50,6 +82,20 @@
                "Device Vendor": null
              }
            ],
+            "plugins_meta": {
+              "title": "Plugins",
+              "columnNames": [
+                "Plugin",
+                "Object_PrimaryID",
+                "Object_SecondaryID",
+                "DateTimeChanged",
+                "Watched_Value1",
+                "Watched_Value2",
+                "Watched_Value3",
+                "Watched_Value4",
+                "Status"
+              ]
+            },
            "plugins": [
              {
                "Index": 138,
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -11,10 +11,10 @@ services:
    network_mode: host
    # restart: unless-stopped
    volumes:
-      # - ${APP_DATA_LOCATION}/pialert_dev/config:/home/pi/pialert/config
-      - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
-      # - ${APP_DATA_LOCATION}/pialert_dev/db:/home/pi/pialert/db      
-      - ${APP_DATA_LOCATION}/pialert/db:/home/pi/pialert/db           
+      - ${APP_DATA_LOCATION}/pialert_dev/config:/home/pi/pialert/config
+      # - ${APP_DATA_LOCATION}/pialert/config:/home/pi/pialert/config
+      - ${APP_DATA_LOCATION}/pialert_dev/db:/home/pi/pialert/db      
+      # - ${APP_DATA_LOCATION}/pialert/db:/home/pi/pialert/db           
      # (optional) useful for debugging if you have issues setting up the container
      - ${LOGS_LOCATION}:/home/pi/pialert/front/log      
      # ---------------------------------------------------------------------------
@@ -29,6 +29,7 @@ services:
      - ${APP_DATA_LOCATION}/pialert/php.ini:/etc/php/8.2/fpm/php.ini      
      - ${DEV_LOCATION}/install:/home/pi/pialert/install
      - ${DEV_LOCATION}/front/css:/home/pi/pialert/front/css
+      - ${DEV_LOCATION}/back/update_vendors.sh:/home/pi/pialert/back/update_vendors.sh
      - ${DEV_LOCATION}/front/lib/AdminLTE:/home/pi/pialert/front/lib/AdminLTE
      - ${DEV_LOCATION}/front/js:/home/pi/pialert/front/js
      - ${DEV_LOCATION}/dockerfiles/start.sh:/home/pi/pialert/dockerfiles/start.sh
--- a/dockerfiles/README.md
+++ b/dockerfiles/README.md
@@ -18,7 +18,7 @@

 ## 📕 Basic Usage 

- You will have to run the container on the host network, e.g: 
+- You will have to run the container on the `host` network, e.g: 

 ```yaml
 docker run -d --rm --network=host \
@@ -27,8 +27,8 @@ docker run -d --rm --network=host \
  -e TZ=Europe/Berlin \
  -e PORT=20211 \
  jokobsk/pi.alert:latest
-  ```
- The initial scan can take up-to 15min (with 50 devices and MQTT). Subsequent ones 3 and 5 minutes so wait that long for all of the scans to run.
+```
+- The initial scan can take up to 15min (with 50 devices and MQTT). Subsequent ones 3 and 5 minutes so wait that long for all of the scans to run.

 ### Docker environment variables

@@ -42,22 +42,22 @@ docker run -d --rm --network=host \

 ### Docker paths

-| | Path | Description |
+| Required | Path | Description |
 | :------------- | :------------- |:-------------| 
-| **Required** | `:/home/pi/pialert/config` | Folder which will contain the `pialert.conf` file (see below for details)  | 
-| **Required** | `:/home/pi/pialert/db` | Folder which will contain the `pialert.db` file  | 
-|Optional| `:/home/pi/pialert/front/log` |  Logs folder useful for debugging if you have issues setting up the container  | 
-|Optional| `:/etc/pihole/pihole-FTL.db` |  PiHole's `pihole-FTL.db` database file. Required if you want to use PiHole  | 
-|Optional| `:/etc/pihole/dhcp.leases` |  PiHole's `dhcp.leases` file. Required if you want to use PiHole `dhcp.leases` file. This has to be matched with a corresponding `DHCPLSS_paths_to_check` setting entry. (the path in the container must contain `pihole`)| 
-|Optional| `:/home/pi/pialert/front/api` |  A simple [API endpoint](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md) containing static (but regularly updated) json and other files.   | 
-|Optional| `:/home/pi/pialert/front/plugins/<plugin>/ignore_plugin` | Map a file `ignore_plugin` to ignore a plugin. Plugins can be soft-disabled via settings. More in the [Plugin docs](/front/plugins/README.md).  | 
+| ✅ | `:/home/pi/pialert/config` | Folder which will contain the `pialert.conf` file (see below for details)  | 
+| ✅ | `:/home/pi/pialert/db` | Folder which will contain the `pialert.db` file  | 
+| | `:/home/pi/pialert/front/log` |  Logs folder useful for debugging if you have issues setting up the container  | 
+| | `:/etc/pihole/pihole-FTL.db` |  PiHole's `pihole-FTL.db` database file. Required if you want to use PiHole  | 
+| | `:/etc/pihole/dhcp.leases` |  PiHole's `dhcp.leases` file. Required if you want to use PiHole `dhcp.leases` file. This has to be matched with a corresponding `DHCPLSS_paths_to_check` setting entry. (the path in the container must contain `pihole`)| 
+| | `:/home/pi/pialert/front/api` |  A simple [API endpoint](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/API.md) containing static (but regularly updated) json and other files.   | 
+| | `:/home/pi/pialert/front/plugins/<plugin>/ignore_plugin` | Map a file `ignore_plugin` to ignore a plugin. Plugins can be soft-disabled via settings. More in the [Plugin docs](/front/plugins/README.md).  | 


-### Config (`pialert.conf`)
+### Modify the config (`pialert.conf`) only if UI is not available

- If unavailable, the app generates a default `pialert.conf` and `pialert.db` file on the first run.
 - The preferred way is to manage the configuration via the Settings section in the UI.
 - You can modify [pialert.conf](https://github.com/jokob-sk/Pi.Alert/tree/main/config) directly, if needed.
+- If unavailable, the app generates a default `pialert.conf` and `pialert.db` file on the first run.

 #### Important settings

@@ -86,10 +86,11 @@ There are 2 approaches how to get PiHole devices imported. Via the PiHole import

 #### 🧭 Community guides

-> Primarily use the official installation guides in this document and use community content as suplementary material. Open an issue if you'd like to add your link to the list 🙏 
+> Use the official installation guides at first and use community content as suplementary material. Open an issue if you'd like to add your link to the list 🙏 

 - 📄 [How to Install Pi.Alert on Your Synology NAS - Marius hosting (English)](https://mariushosting.com/how-to-install-pi-alert-on-your-synology-nas/) (Updated frequently)
 - 📄 [시놀/헤놀에서 네트워크 스캐너 Pi.Alert Docker로 설치 및 사용하기 (Korean)](https://blog.dalso.org/article/%EC%8B%9C%EB%86%80-%ED%97%A4%EB%86%80%EC%97%90%EC%84%9C-%EB%84%A4%ED%8A%B8%EC%9B%8C%ED%81%AC-%EC%8A%A4%EC%BA%90%EB%84%88-pi-alert-docker%EB%A1%9C-%EC%84%A4%EC%B9%98-%EB%B0%8F-%EC%82%AC%EC%9A%A9) (July 2023)
+- 📄 [网络入侵探测器Pi.Alert (Chinese)](https://codeantenna.com/a/VgUvIAjZ7J) (May 2023)
 - ▶ [Pi.Alert auf Synology & Docker by - Jürgen Barth (German)](https://www.youtube.com/watch?v=-ouvA2UNu-A) (March 2023)
 - ▶ [Top Docker Container for Home Server Security - VirtualizationHowto (English)](https://www.youtube.com/watch?v=tY-w-enLF6Q) (March 2023)
 - ▶ [Pi.Alert or WatchYourLAN can alert you to unknown devices appearing on your WiFi or LAN network - Danie van der Merwe (English)](https://www.youtube.com/watch?v=v6an9QG2xF0) (November 2022)
@@ -229,11 +230,7 @@ Courtesy of [pbek](https://github.com/pbek). The volume `pialert_db` is used by

 ## 🏅 Recognitions

-Big thanks to <a href="https://github.com/Macleykun">@Macleykun</a> for help and tips&tricks for Dockerfile(s):
-
-<a href="https://github.com/Macleykun">
-  <img src="https://avatars.githubusercontent.com/u/26381427?size=50"> 
-</a>
+Big thanks to <a href="https://github.com/Macleykun">@Macleykun</a> for help and tips&tricks for Dockerfile(s).

 ## ❤ Support me

--- a/dockerfiles/start.sh
+++ b/dockerfiles/start.sh
@@ -11,6 +11,7 @@ INSTALL_DIR=/home/pi  # Specify the installation directory here
 WEB_UI_DIR=/var/www/html/pialert
 NGINX_CONFIG_FILE=/etc/nginx/conf.d/pialert.conf
 OUI_FILE="/usr/share/arp-scan/ieee-oui.txt" # Define the path to ieee-oui.txt and ieee-iab.txt
+FILEDB=$INSTALL_DIR/pialert/db/pialert.db
 # DO NOT CHANGE ANYTHING ABOVE THIS LINE!

 # if custom variables not set we do not need to do anything
@@ -94,23 +95,26 @@ fi
 # Fixing file permissions
 echo "[INSTALL] Fixing file permissions"

+echo "[INSTALL] Fixing WEB_UI_DIR: $WEB_UI_DIR"

 chmod -R a+rwx $WEB_UI_DIR
+
+echo "[INSTALL] Fixing INSTALL_DIR: $INSTALL_DIR"
+
 chmod -R a+rw $INSTALL_DIR/pialert/front/log
 chmod -R a+rwx $INSTALL_DIR

-FILEDB=$INSTALL_DIR/pialert/db/pialert.db
-
-if [ -f "$FILEDB" ]; then
-    chown -R www-data:www-data $INSTALL_DIR/pialert/db/pialert.db
-fi
-
 echo "[INSTALL] Copy starter pialert.db and pialert.conf if they don't exist"

 # Copy starter pialert.db and pialert.conf if they don't exist
 cp -n "$INSTALL_DIR/pialert/back/pialert.conf" "$INSTALL_DIR/pialert/config/pialert.conf" 
-cp -n "$INSTALL_DIR/pialert/back/pialert.db"  "$INSTALL_DIR/pialert/db/pialert.db" 
+cp -n "$INSTALL_DIR/pialert/back/pialert.db"  "$FILEDB"

+echo "[INSTALL] Fixing permissions after copied starter config & DB"
+
+if [ -f "$FILEDB" ]; then
+    chown -R www-data:www-data $FILEDB
+fi

 chmod -R a+rwx $INSTALL_DIR # second time after we copied the files
 chmod -R a+rw $INSTALL_DIR/pialert/config
@@ -122,7 +126,6 @@ if [ ! -f "$INSTALL_DIR/pialert/front/buildtimestamp.txt" ]; then
    date +%s > "$INSTALL_DIR/pialert/front/buildtimestamp.txt"
 fi

-
 # start PHP
 /etc/init.d/php8.2-fpm start
 /etc/init.d/nginx start
--- a/docs/DATABASE.md
+++ b/docs/DATABASE.md
@@ -12,7 +12,7 @@
  | CurrentScan | Result of the current scan | ![Screen1][screen1]  |  
  | Devices     | The main devices database that also contains the Network tree mappings. If `ScanCycle` is set to `0` device is not scanned. | ![Screen2][screen2]  |   
  | Events | Used to collect connection/disconnection events. | ![Screen4][screen4]  |   
-  | Online_History   | Used to display the `Device presence over time` chart  | ![Screen6][screen6]  | 
+  | Online_History   | Used to display the `Device presence` chart  | ![Screen6][screen6]  | 
  | Parameters       | Used to pass values between the frontend and backend. | ![Screen7][screen7]  | 
  | Pholus_Scan      | Scan results of the Pholus python network penetration script. | ![Screen8][screen8]  |   
  | Plugins_Events   | For capturing events exposed by a plugin via the `last_result.log` file. If unique then saved into the `Plugins_Objects` table. Entries are deleted once processed and stored in the `Plugins_History` and/or `Plugins_Objects` tables.  | ![Screen10][screen10]  | 
--- a/docs/DEBUG_INVALID_JSON.md
+++ b/docs/DEBUG_INVALID_JSON.md
@@ -8,22 +8,27 @@ Check the the HTTP response of the failing backend call by following these steps
 ![F12DeveloperConsole][F12DeveloperConsole]

 - Copy the URL causing the error and enter it in the address bar of your browser directly and hit enter. The copied URLs could look something like this (notice the query strings at the end):
+  - `http://<pialert URL>:20211/api/table_devices.json?nocache=1704141103121`
  - `http://<pialert URL>:20211/php/server/devices.php?action=getDevicesTotals`
-  - `http://<pialert URL>:20211/php/server/devices.php?action=getDevicesList&status=all`
+  - `http://<pialert URL>:20211/php/server/devices.php?action=getDevicesList&status=all`  

 - Post the error response in the existing issue thread on GitHub or create a new issue and include the redacted response of the failing query.

 For reference, the above queries should return results in the following format:

-First URL:
+## First URL:
+
+- Should yield a valid JSON file
+
+## Second URL:

 ![array][array]

-Second URL:
+## Third URL:

 ![json][json]

-You can copy and paste any JSON result (result of the second query) into an online JSON checker, such as [this one](https://jsonchecker.com/) to check if it's valid.
+You can copy and paste any JSON result (result of the First and Third query) into an online JSON checker, such as [this one](https://jsonchecker.com/) to check if it's valid.


 [F12DeveloperConsole]:    ./img/DEBUG/Invalid_JSON_repsonse_debug.png           "F12DeveloperConsole"
--- a/docs/DEBUG_PLUGINS.md
+++ b/docs/DEBUG_PLUGINS.md
@@ -0,0 +1,74 @@
+# Troubleshooting plugins
+
+## High-level overview
+
+If a Plugin supplies data to the main app it's doine either vie a SQL query or via a script that updates the `last_result.log` file in the plugin folder (`front/plugins/<plugin>`).
+
+For a more in-depth overview on how plugins work check the [Plugins development docs](https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README.md).
+
+### Prerequisites
+
+- Make sure you read and followed the specific plugin setup instructions.
+- Ensure you have [debug enabled (see More Logging)](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#1-more-logging-) 
+
+### Potential issues
+
+- Bugs
+- Unexpected input (e.g. special characters in names)
+- Dependencies changed how data is output
+
+#### Incorrect input data
+
+Input data from the plugin might cause mapping issues in specific edge cases. Look for a corresponding section in the `pialert.log` file, for example notice the first line of the execution run of the `PIHOLE` plugin below:
+
+```
+17:31:05 [Scheduler] - Scheduler run for PIHOLE: YES
+17:31:05 [Plugin utils] ---------------------------------------------
+17:31:05 [Plugin utils] display_name: PiHole (Device sync)
+17:31:05 [Plugins] CMD: SELECT n.hwaddr AS Object_PrimaryID, {s-quote}null{s-quote} AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr is not {s-quote}00:00:00:00:00:00{s-quote}  AND na.ip is not null
+17:31:05 [Plugins] setTyp: subnets
+17:31:05 [Plugin utils] Flattening the below array
+17:31:05 ['192.168.1.0/24 --interface=eth1']
+17:31:05 [Plugin utils] isinstance(arr, list) : False | isinstance(arr, str) : True
+17:31:05 [Plugins] Resolved value: 192.168.1.0/24 --interface=eth1
+17:31:05 [Plugins] Convert to Base64: True
+17:31:05 [Plugins] base64 value: b'MTkyLjE2OC4xLjAvMjQgLS1pbnRlcmZhY2U9ZXRoMQ=='
+17:31:05 [Plugins] Timeout: 10
+17:31:05 [Plugins] Executing: SELECT n.hwaddr AS Object_PrimaryID, 'null' AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, 'null' AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE 'ip-%' AND n.hwaddr is not '00:00:00:00:00:00'  AND na.ip is not null
+17:31:05 [Plugins] SUCCESS, received 2 entries
+17:31:05 [Plugins] sqlParam entries: [(0, 'PIHOLE', '01:01:01:01:01:01', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.1', 0, 'aaaa', 'vvvvvvvvv', 'not-processed', 'null', 'null', '01:01:01:01:01:01'), (0, 'PIHOLE', '02:42:ac:1e:00:02', 'null', 'null', '2023-12-25 06:31:05', '172.30.0.2', 0, 'dddd', 'vvvvv2222', 'not-processed', 'null', 'null', '02:42:ac:1e:00:02')]
+17:31:05 [Plugins] Processing        : PIHOLE
+17:31:05 [Plugins] Existing objects from Plugins_Objects: 4
+17:31:05 [Plugins] Logged events from the plugin run    : 2
+17:31:05 [Plugins] pluginEvents      count: 2
+17:31:05 [Plugins] pluginObjects     count: 4
+17:31:05 [Plugins] events_to_insert  count: 0
+17:31:05 [Plugins] history_to_insert count: 4
+17:31:05 [Plugins] objects_to_insert count: 0
+17:31:05 [Plugins] objects_to_update count: 4
+17:31:05 [Plugin utils] In pluginEvents there are 2 events with the status "watched-not-changed" 
+17:31:05 [Plugin utils] In pluginObjects there are 2 events with the status "missing-in-last-scan" 
+17:31:05 [Plugin utils] In pluginObjects there are 2 events with the status "watched-not-changed" 
+17:31:05 [Plugins] Mapping objects to database table: CurrentScan
+17:31:05 [Plugins] SQL query for mapping: INSERT into CurrentScan ( "cur_MAC", "cur_IP", "cur_LastQuery", "cur_Name", "cur_Vendor", "cur_ScanMethod") VALUES ( ?, ?, ?, ?, ?, ?)
+17:31:05 [Plugins] SQL sqlParams for mapping: [('01:01:01:01:01:01', '172.30.0.1', 0, 'aaaa', 'vvvvvvvvv', 'PIHOLE'), ('02:42:ac:1e:00:02', '172.30.0.2', 0, 'dddd', 'vvvvv2222', 'PIHOLE')]
+17:31:05 [API] Update API starting
+17:31:06 [API] Updating table_plugins_history.json file in /front/api
+```
+
+In the above output notice the section logging how many events are produced by the plugin:
+
+```
+17:31:05 [Plugins] Existing objects from Plugins_Objects: 4
+17:31:05 [Plugins] Logged events from the plugin run    : 2
+17:31:05 [Plugins] pluginEvents      count: 2
+17:31:05 [Plugins] pluginObjects     count: 4
+17:31:05 [Plugins] events_to_insert  count: 0
+17:31:05 [Plugins] history_to_insert count: 4
+17:31:05 [Plugins] objects_to_insert count: 0
+17:31:05 [Plugins] objects_to_update count: 4
+```
+
+These values, if formatted correctly, will also show up in the UI:
+
+![Plugins table](/docs/img/DEBUG_PLUGINS/plugin_objects_pihole.png)
--- a/docs/DEBUG_TIPS.md
+++ b/docs/DEBUG_TIPS.md
@@ -56,7 +56,7 @@ services:
 * If facing issues (AJAX errors, can't write to DB, empty screen, etc,) make sure permissions are set correctly, and check the logs under `/home/pi/pialert/front/log`. 
 * To solve permission issues you can try setting the owner and group of the `pialert.db` by executing the following on the host system: `docker exec pialert chown -R www-data:www-data /home/pi/pialert/db/pialert.db`. 
 * Map to local User and Group IDs. Specify the enviroment variables `HOST_USER_ID` and `HOST_USER_GID` if needed.
-* If still facing issues, try to map the pialert.db file (⚠ not folder) to `:/home/pi/pialert/db/pialert.db` (see Examples below for details)
+* If still facing issues, try to map the pialert.db file (⚠ not folder) to `:/home/pi/pialert/db/pialert.db` (see [docker-compose Examples](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md#-docker-composeyml-examples) for details)

 ### Container restarts / crashes

--- a/docs/HW_INSTALL.md
+++ b/docs/HW_INSTALL.md
@@ -1,6 +1,6 @@
 # How to install PiAlert on the server hardware

-To download and install PiAlert on the hardware/server directly use `curl` or `wget` commands.
+To download and install PiAlert on the hardware/server directly use the `curl` or `wget` commands at the bottom of this page.

 > [!NOTE]
 > This is an Experimental feature 🧪 and it relies on community support.
@@ -32,13 +32,13 @@ Some facts about what and where something will be changed/installed by the HW in
 - Only tested to work on Debian Bookworm (Debian 12).
 - **EXPERIMENTAL** and not recommended way to install PiAlert.

-## CURL
+## 📥 Installation via CURL

 ```bash
 curl -o install.sh https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/install/install.sh && sudo chmod +x install.sh && sudo ./install.sh
 ```

-## WGET
+## 📥 Installation via WGET

 ```bash
 wget https://raw.githubusercontent.com/jokob-sk/Pi.Alert/main/install/install.sh -O install.sh && sudo chmod +x install.sh && sudo ./install.sh
--- a/docs/NETWORK_TREE.md
+++ b/docs/NETWORK_TREE.md
@@ -8,12 +8,11 @@ Make sure you have a root device with the MAC `Internet` (No other MAC addresses

 ## ⚡Quick setup:

-* Go to Devices > Device Details. 
-* Find the device(s) you want to use as network devices (network nodes). 
-* Set the Type of such a device to one of the following: AP, Firewall, Gateway, PLC, Powerline, Router, Switch, USB LAN Adapter, USB WIFI Adapter and WLAN.
+* Go to a Device you want to use as network device (network nodes, such as a Switch). 
+* Set the **Type** of such a device to one of the following: AP, Firewall, Gateway, PLC, Powerline, Router, Switch, USB LAN Adapter, USB WIFI Adapter and WLAN (you can create a custom network type device with in Settings -> General -> `NETWORK_DEVICE_TYPES`).
 * Save and go to Network where the devices you've marked as network devices (by selecting the Type as mentioned above) will show up as tabs.
-* You can now assign the Unassigend devices to the correct network node.
-* If port is empty or 0 a wifi icon is rendered, otherwise a ethernet port icon
+* You can now assign the Unassigend devices to the network node.
+* If port is empty or 0 a wifi icon is rendered, otherwise a ethernet port icon.


 > [!NOTE] 
@@ -46,7 +45,7 @@ In this example you will setup a device named `rapberrypi` as a `Switch` in our
 ![Network page](/docs/img/NETWORK_TREE/Network_Page.png)

 - Notice the newly added `raspberrypi` (2) tab which now represents a network node, also showing up in the tree (3).
- As we asssigned the `raspberrypi` in the previous 1) Device details page section to the `Internet` parent network node in step (6), the link is also showing up in the tree diagram (4)
+- As we asssigned the `raspberrypi` in the previous (1) Device details page section to the `Internet` parent network node in step (6), the link is also showing up in the tree diagram (4)
 - We can now assign the device `(AppleTV)` (5) to this `raspberrypi` node, representing a network Switch in this example

 ### 3. Network page with 2 levels
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,16 +1,27 @@
 ## Documentation overview

-In the app hover over settings or fields/labels or click blue in-app ❔ (question-mark) icons to get to relevant documentation pages.
+<details>
+  <summary>:information_source: In the app hover over settings or fields/labels or click blue in-app ❔ (question-mark) icons to get to relevant documentation pages.</summary>

-![In-app help](/docs/img/GENERAL/in-app-help.png)
+  ![In-app help](/docs/img/GENERAL/in-app-help.png)
+
+</details>

 There is also an in-app Help / FAQ section that should be answering frequently asked questions.

 ### 📥 Installation

- ⚠ Only tested as a [docker container - follow these instructions here](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md). 
- > Check out [leiweibau's fork](https://github.com/leiweibau/Pi.Alert/) if you want to install Pi.Alert on the server directly or original instructions for [pucherot's original code](https://github.com/pucherot/Pi.Alert/)
+#### 🐳 Docker (Fully supported)

+- The main installation method is as a [docker container - follow these instructions here](https://github.com/jokob-sk/Pi.Alert/blob/main/dockerfiles/README.md). 
+
+#### 💻 Bare-metal / On-server (Experimental/community supported 🧪)
+
+- [(Experimental 🧪) On-hardware instructions](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HW_INSTALL.md) 
+
+- Alternative bare-metal install forks: 
+  - [leiweibau's fork](https://github.com/leiweibau/Pi.Alert/) (maintained)
+  - [pucherot's original code](https://github.com/pucherot/Pi.Alert/) (un-maintained)

 ### 📚 Table of contents

@@ -18,6 +29,7 @@ There is also an in-app Help / FAQ section that should be answering frequently a

 - [Debugging tips](/docs/DEBUG_TIPS.md)
 - [Invalid JSON errors debug help](/docs/DEBUG_INVALID_JSON.md)
+- [Troubleshooting Plugins](/docs/DEBUG_PLUGINS.md)

 #### 🔝 Popular/Suggested

@@ -31,7 +43,7 @@ There is also an in-app Help / FAQ section that should be answering frequently a

 - [Manage devices (legacy docs)](/docs/DEVICE_MANAGEMENT.md)
 - [Random MAC/MAC icon meaning (legacy docs)](/docs/RANDOM_MAC.md)
- [Custom Icons configuration and support](/docs/ICONS.md)
+- [Custom Icon configuration and support](/docs/ICONS.md)

 #### 🔎 Examples

@@ -96,7 +108,7 @@ Suggested test cases:
 - Blank setup with no DB or config
 - Existing DB / config
 - Sending a notification (e. g. Delete a device and wait for a scan to run) and testing all notification gateways, especially:
-   Email, Apprise (e.g. via Telegram), webhook (e.g. via Discord), MQTT (e.g. via HomeAssitant)
+   - Email, Apprise (e.g. via Telegram), webhook (e.g. via Discord), MQTT (e.g. via Home Assistant)
 - Saving settings
 - Test a couple of plugins
 - Check the Error log for anything unusual
@@ -110,7 +122,7 @@ Some additional context:

 Before submitting a new issue please spend a couple of minutes on research:

-* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/tree/main/dockerfiles#-common-issues) 
+* Check [🛑 Common issues](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md#common-issues) 
 * Check [💡 Closed issues](https://github.com/jokob-sk/Pi.Alert/issues?q=is%3Aissue+is%3Aclosed) if a similar issue was solved in the past.
 * When submitting an issue ❗[enable debug](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_TIPS.md)❗

--- a/docs/SUBNETS.md
+++ b/docs/SUBNETS.md
@@ -4,6 +4,10 @@ You need to specify the network interface and the network mask. You can also con

 ## Examples

+> [!NOTE] 
+> Please use the UI to configure settings as that ensures that the config file is in the correct format. Edit `pialert.conf` directly only when really necessary.
+> ![settings](/front/plugins/arp_scan/arp-scan-settings.png)
+
 * Examples for one and two subnets  (❗ Note the `['...', '...']` format):
   * One subnet: `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0']`
   * Two subnets:  `SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth0', '192.168.1.0/24 --interface=eth1 -vlan=107']` 
--- a/docs/img/DEBUG_PLUGINS/plugin_objects_pihole.png
+++ b/docs/img/DEBUG_PLUGINS/plugin_objects_pihole.png
--- a/front/devices.php
+++ b/front/devices.php
@@ -327,7 +327,7 @@ function filterDataByStatus(data, status) {
      case 'new':
        return item.dev_NewDevice === 1;
      case 'down':
-        return item.dev_PresentLastScan === 0 && item.dev_AlertDeviceDown  === 1;
+        return item.dev_PresentLastScan === 0 && item.dev_AlertDeviceDown  !== 0;
      case 'archived':
        return item.dev_Archived === 1;
      default:
@@ -343,7 +343,7 @@ function getDeviceStatus(item)
    {
      return 'On-line';
    }
-    else if(item.dev_PresentLastScan === 0 && item.dev_AlertDeviceDown  === 1)
+    else if(item.dev_PresentLastScan === 0 && item.dev_AlertDeviceDown  !== 0)
    {
      return 'Down';
    }
--- a/front/js/pialert_common.js
+++ b/front/js/pialert_common.js
@@ -349,6 +349,7 @@ function sanitize(data)
 // -----------------------------------------------------------------------------
 function numberArrayFromString(data)
 {
+  console.log(data)
  data = JSON.parse(sanitize(data));
  return data.replace(/\[|\]/g, '').split(',').map(Number);
 }
@@ -514,15 +515,40 @@ function getNameByMacAddress(macAddress) {

 // -----------------------------------------------------------------------------
 // A function used to make the IP address orderable
+function isValidIPv6(ipAddress) {
+  // Regular expression for IPv6 validation
+  const ipv6Regex = /^([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}$|^([0-9a-fA-F]{1,4}:){1,7}:|^([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}$|^([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}$|^([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}$|^([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}$|^([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}$|^[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})$/;
+
+  return ipv6Regex.test(ipAddress);
+}
+
 function formatIPlong(ipAddress) {
-  const parts = ipAddress.split('.');
-  if (parts.length !== 4) {
-      throw new Error('Invalid IP address format');
+  if (ipAddress.includes(':') && isValidIPv6(ipAddress)) {
+    const parts = ipAddress.split(':');
+
+    return parts.reduce((acc, part, index) => {
+      if (part === '') {
+        const remainingGroups = 8 - parts.length + 1;
+        return acc << (16 * remainingGroups);
+      }
+
+      const hexValue = parseInt(part, 16);
+      return acc | (hexValue << (112 - index * 16));
+    }, 0);
+  } else {
+    // Handle IPv4 address
+    const parts = ipAddress.split('.');
+
+    if (parts.length !== 4) {
+      console.log("⚠ Invalid IPv4 address: " + ipAddress);
+      return -1; // or any other default value indicating an error
+    }
+
+    return (parseInt(parts[0]) << 24) |
+           (parseInt(parts[1]) << 16) |
+           (parseInt(parts[2]) << 8) |
+           parseInt(parts[3]);
  }
-  return (parseInt(parts[0]) << 24) |
-         (parseInt(parts[1]) << 16) |
-         (parseInt(parts[2]) << 8) |
-         parseInt(parts[3]);
 }

 // -----------------------------------------------------------------------------
--- a/front/js/settings_utils.js
+++ b/front/js/settings_utils.js
@@ -19,6 +19,25 @@
    return result;
  }

+  // -------------------------------------------------------------------
+  // Get plugin type base on prefix
+  function getPluginCodeName(pluginsData, prefix)
+  {
+    var result = ""
+
+    pluginsData.forEach((plug) => {
+
+      if (plug.unique_prefix == prefix ) {
+        id = plug.code_name;
+        
+        // console.log(id)
+        result = plug.code_name;        
+      }
+    });
+
+    return result;
+  }
+

  // -------------------------------------------------------------------
  // Get plugin type base on prefix
@@ -61,19 +80,22 @@

      });

-      html += `
-            
+      html += `            
              <div class="col-sm-4 ">
                <div class="small-box bg-green " >
                <div class="inner ">
+                  <a href="#${prefix}_header" onclick="toggleAllSettings('open')">
                    <h5 class="card-title">
-                      ${getString(prefix+"_display_name")}
+                      <b>${getString(prefix+"_display_name")}</b>
                    </h5>
-                    ${includeSettings_html}
+                  </a>
+                  ${includeSettings_html}
                </div>
-                <div class="icon"> ${getString(prefix+"_icon")} </div> 
-                  
+                  <a href="#${prefix}_header" onclick="toggleAllSettings('open')">
+                    <div class="icon"> ${getString(prefix+"_icon")} </div> 
+                  </a>  
                </div>
+                
              </div>
            `
    });
@@ -81,6 +103,41 @@
    return html;    
  }

+
+  // -----------------------------------------------------------------------------
+  // Open or close all settings
+  // -----------------------------------------------------------------------------
+  function toggleAllSettings(openOrClose = '')
+  {
+    inStr = ' in';
+    allOpen = true;
+    openIcon = 'fa-angle-double-down';
+    closeIcon = 'fa-angle-double-up';    
+
+    $('.panel-collapse').each(function(){
+      if($(this).attr('class').indexOf(inStr) == -1)
+      {
+        allOpen = false;
+      }
+    })
+    
+    if(allOpen == false || openOrClose == 'open')
+    {
+      // open all
+      $('div[data-myid="collapsible"]').each(function(){$(this).attr('class', 'panel-collapse collapse in')})
+      $('div[data-myid="collapsible"]').each(function(){$(this).attr('style', 'height:inherit')})
+      $('#toggleSettings').attr('class', $('#toggleSettings').attr('class').replace(openIcon, closeIcon))
+      
+    }
+    else{
+      // close all
+      $('div[data-myid="collapsible"]').each(function(){$(this).attr('class', 'panel-collapse collapse  ')})      
+      $('#toggleSettings').attr('class', $('#toggleSettings').attr('class').replace(closeIcon, openIcon))
+    }
+    
+  }
+
+
  // -------------------------------------------------------------------
  // Checks if all schedules are the same
  function schedulesAreSynchronized(prefixesOfEnabledPlugins, pluginsData)
--- a/front/php/server/devices.php
+++ b/front/php/server/devices.php
@@ -77,7 +77,7 @@ function getDeviceData() {

  // Device Data
  $sql = 'SELECT rowid, *,
-            CASE WHEN dev_AlertDeviceDown=1 AND dev_PresentLastScan=0 THEN "Down"
+            CASE WHEN dev_AlertDeviceDown !=0 AND dev_PresentLastScan=0 THEN "Down"
                 WHEN dev_PresentLastScan=1 THEN "On-line"
                 ELSE "Off-line" END as dev_Status
          FROM Devices
@@ -626,7 +626,7 @@ function getDevicesList() {

  $sql = 'SELECT * FROM (
              SELECT rowid, *, CASE
-                      WHEN t1.dev_AlertDeviceDown=1 AND t1.dev_PresentLastScan=0 THEN "Down"
+                      WHEN t1.dev_AlertDeviceDown !=0 AND t1.dev_PresentLastScan=0 THEN "Down"
                      WHEN t1.dev_NewDevice=1 THEN "New"
                      WHEN t1.dev_PresentLastScan=1 THEN "On-line"
                      ELSE "Off-line"  END AS dev_Status
@@ -1133,14 +1133,14 @@ function copyFromDevice() {
 //------------------------------------------------------------------------------
 function getDeviceCondition ($deviceStatus) {
  switch ($deviceStatus) {
-    case 'all':        return 'WHERE dev_Archived=0';                                                      break;
-    case 'connected':  return 'WHERE dev_Archived=0 AND dev_PresentLastScan=1';                            break;
-    case 'favorites':  return 'WHERE dev_Archived=0 AND dev_Favorite=1';                                   break;
-    case 'new':        return 'WHERE dev_Archived=0 AND dev_NewDevice=1';                                  break;
-    case 'down':       return 'WHERE dev_Archived=0 AND dev_AlertDeviceDown=1 AND dev_PresentLastScan=0';  break;
-    case 'archived':   return 'WHERE dev_Archived=1';                                                      break;
-    default:           return 'WHERE 1=0';                                                                 break;
-  }
+    case 'all':        return 'WHERE dev_Archived=0';                                                        break;
+    case 'connected':  return 'WHERE dev_Archived=0 AND dev_PresentLastScan=1';                              break;
+    case 'favorites':  return 'WHERE dev_Archived=0 AND dev_Favorite=1';                                     break;
+    case 'new':        return 'WHERE dev_Archived=0 AND dev_NewDevice=1';                                    break;
+    case 'down':       return 'WHERE dev_Archived=0 AND dev_AlertDeviceDown !=0 AND dev_PresentLastScan=0';  break;
+    case 'archived':   return 'WHERE dev_Archived=1';                                                        break;
+    default:           return 'WHERE 1=0';                                                                   break;
+  }  
 }


--- a/front/php/templates/language/de_de.json
+++ b/front/php/templates/language/de_de.json
@@ -558,17 +558,13 @@
        "PIALERT_WEB_PROTECTION_name": "Login aktivieren",
        "PIALERT_WEB_PROTECTION_description": "Ein Loginfenster wird angezeigt wenn aktiviert. Untere Beschreibung genau durchlesen falls Sie sich aus Ihrer Instanz aussperren.",
        "PIALERT_WEB_PASSWORD_name": "Login-Passwort",
-        "PIALERT_WEB_PASSWORD_description": "Das Standardpasswort ist <code>123456</code>. Um das Passwort zu ändern, entweder <code>/home/pi/pialert/back/pialert-cli</code> im Container starten oder <a onclick=\"toggleAllSettings()\" href=\"#SETPWD_RUN\"><code>SETPWD_RUN</code> Set password plugin</a> nutzen.",
-        "INCLUDED_SECTIONS_name": "Benachrichtigungen",
-        "INCLUDED_SECTIONS_description": "Spezifiziert, bei welchen Events Benachrichtigungen versendet werden. Entfernen Sie die Eventtypen, bei welchen Sie nicht benachrichtigt werden wollen. Diese Einstellung überschreibt gerätespezifische Einstellungen im UI. (<code>STRG + klicken</code> zum aus-/abwählen).",
+        "PIALERT_WEB_PASSWORD_description": "Das Standardpasswort ist <code>123456</code>. Um das Passwort zu ändern, entweder <code>/home/pi/pialert/back/pialert-cli</code> im Container starten oder <a onclick=\"toggleAllSettings()\" href=\"#SETPWD_RUN\"><code>SETPWD_RUN</code> Set password plugin</a> nutzen.",       
        "DAYS_TO_KEEP_EVENTS_name": "Lösche Events älter als",
        "DAYS_TO_KEEP_EVENTS_description": "Dies ist eine Wartungseinstellung. Spezifiziert wie viele Tage Events gespeichert bleiben. Alle älteren Events werden periodisch gelöscht. Wird auch auf die Plugins History angewendet.",
        "HRS_TO_KEEP_NEWDEV_name": "Neue Geräte speichern für",
        "HRS_TO_KEEP_NEWDEV_description": "Dies ist eine Wartungseinstellung. Geräte markiert als <b>Neues Gerät</b> werden gelöscht, wenn ihre <b>Erste Sitzung</b> länger her ist als die angegebenen Stunden in dieser Einstellung. <code>0</code> deaktiviert diese Funktion. Nutzen Sie diese Einstellung, um <b>Neue Geräte</b> automatisch nach <code>X</code> Stunden zu löschen.",
        "REPORT_DASHBOARD_URL_name": "Pi.Alert URL",
        "REPORT_DASHBOARD_URL_description": "Diese URL wird als Basis fürs Erstellen von Links in E-Mails genutzt. Geben Sie die gesamte URL startend mit <code>http://</code> inklusive der genutzten Portnummer ein (keinen nachfolgenden Schrägstrich <code>/</code> nutzen).",
-        "DIG_GET_IP_ARG_name": "Erkennung externer IP (\"Internet IP\")",
-        "DIG_GET_IP_ARG_description": "Ändere die Argumente des <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">dig Dienstprogramms</a>, wenn Probleme beim Auflösen der externen IP auftreten. Argumente werden an das Ende des folgenden Befehls angehängt: <code>dig +short </code>.",
        "NETWORK_DEVICE_TYPES_name": "Netzwerkgeräte-Typen",
        "NETWORK_DEVICE_TYPES_description": "Welche Gerätetypen als Netzwerkgeräte in der Netzwerkansicht verwendet werden können. Der Gerätetyp muss genau der <code>Typ</code>-Einstellung eines spezifischen Geräts in den Gerätedetails übereinstimmen. Entfernen Sie keine existierenden Typen, sondern fügen Sie nur neue ein.",
        "UI_LANG_name": "UI Sprache",
--- a/front/php/templates/language/en_us.json
+++ b/front/php/templates/language/en_us.json
@@ -55,7 +55,7 @@
            "Device_Shortcut_DownAlerts" : "Down Alerts",
            "Device_Shortcut_Archived" : "Archived",
            "Device_Shortcut_Devices" : "Devices",
-            "Device_Shortcut_OnlineChart" : "Device presence over time",
+            "Device_Shortcut_OnlineChart" : "Device presence",
            "Device_TableHead_Name" : "Name",
            "Device_TableHead_Owner" : "Owner",
            "Device_TableHead_Type" : "Type",
@@ -340,7 +340,7 @@
            "Maintenance_Tool_del_ActHistory_noti" : "Delete network activity",
            "Maintenance_Tool_del_ActHistory_noti_text" : "Are you sure you want to reset the network activity?",
            "Maintenance_Tool_ExportCSV" : "CSV Export",
-            "Maintenance_Tool_ExportCSV_text" : "Generate a CSV (comma separated value) file containing the list of Devices including the Network relationships between Network Nodes and connected devices. You can also trigger this by accessing this URL <code>your pialert url/php/server/devices.php?action=ExportCSV</code>",
+            "Maintenance_Tool_ExportCSV_text" : "Generate a CSV (comma separated value) file containing the list of Devices including the Network relationships between Network Nodes and connected devices. You can also trigger this by accessing this URL <code>your pialert url/php/server/devices.php?action=ExportCSV</code> or by enabling the <a href=\"settings.php#CSVBCKP_header\">CSV Backup</a> plugin.",
            "Maintenance_Tool_ExportCSV_noti" : "CSV Export",
            "Maintenance_Tool_ExportCSV_noti_text" : "Are you sure you want to generate a CSV file?",
            "Maintenance_Tool_ImportCSV" : "CSV Import",
@@ -490,7 +490,7 @@
            "Setting_Override_Description" : "Enabling this option will override an App supplied default value with the value specified above.",
            "General_display_name" : "General",
            "General_icon" : "<i class=\"fa fa-gears\"></i>",        
-            "SCAN_SUBNETS_description" : "Arp-scan is a command-line tool that uses the ARP protocol to discover and fingerprint IP hosts on the local network. An alternative to ARP scan is to enable the <a onclick=\"toggleAllSettings()\" href=\"#PIHOLE_RUN\"><code>PIHOLE_RUN</code>PiHole integration settings</a>. The arp-scan time itself depends on the number of IP addresses to check so set this up carefully with the appropriate network mask and interface. Check the <a href=\"https://github.com/jokob-sk/Pi.Alert/blob/main/docs/SUBNETS.md\" target=\"_blank\">subnets documentation</a> for help on setting up VLANs, what VLANs are supported, or how to figure out the network mask and your interface.",
+            "SCAN_SUBNETS_description" : "Arp-scan is a command-line tool that uses the ARP protocol to discover and fingerprint IP hosts on the local network. An alternative to ARP scan is to enable some other Device scanners. The arp-scan time itself depends on the number of IP addresses to check so set this up carefully with the appropriate network mask and interface. Check the <a href=\"https://github.com/jokob-sk/Pi.Alert/blob/main/docs/SUBNETS.md\" target=\"_blank\">subnets documentation</a> for help on setting up VLANs, what VLANs are supported, or how to figure out the network mask and your interface.",
            "LOG_LEVEL_name" : "Print additional logging",
            "LOG_LEVEL_description" : "This setting will enable more verbose logging. Useful for debugging events writing into the database.",
            "TIMEZONE_name" : "Time zone",
@@ -499,28 +499,22 @@
            "ENABLE_PLUGINS_description" : "Enables the <a target=\"_blank\" href=\"https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins\">plugins</a> functionality. Loading plugins requires more hardware resources so you might want to disable them on low-powered system.",
            "PLUGINS_KEEP_HIST_name" : "Plugins History",
            "PLUGINS_KEEP_HIST_description" : "How many entries of Plugins History scan results should be kept (per Plugin, and not device specific).",
-            "DBCLNP_NOTIFI_HIST_name" : "Notifications History",
-            "DBCLNP_NOTIFI_HIST_description" : "How many historical entries of Notifications should be kept.",
            "PIALERT_WEB_PROTECTION_name" : "Enable login",
            "PIALERT_WEB_PROTECTION_description" : "When enabled a login dialog is displayed. Read below carefully if you get locked out of your instance.",
            "PIALERT_WEB_PASSWORD_name" : "Login password",
-            "PIALERT_WEB_PASSWORD_description" : "The default password is <code>123456</code>. To change the password run <code>/home/pi/pialert/back/pialert-cli</code> in the container or use the <a onclick=\"toggleAllSettings()\" href=\"#SETPWD_RUN\"><code>SETPWD_RUN</code> Set password plugin</a>.",
-            "INCLUDED_SECTIONS_name" : "Notify on",
-            "INCLUDED_SECTIONS_description" : "Specifies which events trigger notifications. Remove the event type(s) you do not want to get notified on. This setting overrides device-specific settings in the UI. (<code>CTRL + Click</code> to select/deselect).",        
+            "PIALERT_WEB_PASSWORD_description" : "The default password is <code>123456</code>. To change the password run <code>/home/pi/pialert/back/pialert-cli</code> in the container or use the <a onclick=\"toggleAllSettings()\" href=\"#SETPWD_RUN\"><code>SETPWD_RUN</code> Set password plugin</a>.",            
            "DAYS_TO_KEEP_EVENTS_name" : "Delete events older than",
            "DAYS_TO_KEEP_EVENTS_description" : "This is a maintenance setting. This specifies the number of days worth of event entries that will be kept. All older events will be deleted periodically. Also applies on Plugin Events History.",
            "HRS_TO_KEEP_NEWDEV_name" : "Keep new devices for",
            "HRS_TO_KEEP_NEWDEV_description" : "This is a maintenance setting. If enabled (<code>0</code> is disabled), devices marked as <b>New Device</b> will be deleted if their <b>First Session</b> time was older than the specified hours in this setting. Use this setting if you want to auto-delete <b>New Devices</b> after <code>X</code> hours.",
            "REPORT_DASHBOARD_URL_name" : "Pi.Alert URL",
            "REPORT_DASHBOARD_URL_description" : "This URL is used as the base for generating links in HTML reports (e.g.: emails). Enter full URL starting with <code>http://</code> including the port number (no trailing slash <code>/</code>).",
-            "DIG_GET_IP_ARG_name" : "Internet IP discovery",
-            "DIG_GET_IP_ARG_description" : "Change the <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">dig utility</a> arguments if you have issues resolving your Internet IP. Arguments are added at the end of the following command: <code>dig +short </code>.",
            "NETWORK_DEVICE_TYPES_name" : "Network device types",
            "NETWORK_DEVICE_TYPES_description" : "Which device types are allowed to be used as network devices in the Network view. The device type has to match exactly the <code>Type</code> setting on a specific device in Device details. Do not remove existing types, only add new ones.",
            "UI_LANG_name" : "UI Language",
            "UI_LANG_description" : "Select the preferred UI language.",
            "UI_PRESENCE_name" : "Show in presence chart",
-            "UI_PRESENCE_description" : "Select what statuses should be shown in the <b>Device presence over time</b>  chart in the <a href=\"/devices.php\" target=\"_blank\">Devices</a> page. (<code>CTRL + Click</code> to select/deselect)",
+            "UI_PRESENCE_description" : "Select what statuses should be shown in the <b>Device presence</b>  chart in the <a href=\"/devices.php\" target=\"_blank\">Devices</a> page. (<code>CTRL + Click</code> to select/deselect)",
            "Email_display_name" : "Email",
            "Email_icon" : "<i class=\"fa fa-at\"></i>",
            "REPORT_MAIL_name" : "Enable email",
--- a/front/php/templates/language/es_es.json
+++ b/front/php/templates/language/es_es.json
@@ -470,17 +470,13 @@
        "PIALERT_WEB_PROTECTION_name" : "Habilitar inicio de sesión",
        "PIALERT_WEB_PROTECTION_description" : "Cuando está habilitado, se muestra un cuadro de diálogo de inicio de sesión. Lea detenidamente a continuación si se le bloquea el acceso a su instancia.",
        "PIALERT_WEB_PASSWORD_name" : "Contraseña de inicio de sesión",
-        "PIALERT_WEB_PASSWORD_description" : "La contraseña predeterminada es <code>123456</code>. Para cambiar la contraseña, ejecute <code>/home/pi/pialert/back/pialert-cli</code> en el contenedor",
-        "INCLUDED_SECTIONS_name" : "Notificar en",
-        "INCLUDED_SECTIONS_description" : "Especifica que eventos envían notificaciones. Elimina los tipos de eventos de los que no quieras recibir notificaciones. Este ajuste sobreescribe los ajustes específicos de los dispositivos en la interfaz. (<code>CTRL + Clic</code> para seleccionar / deseleccionar).",        
+        "PIALERT_WEB_PASSWORD_description" : "La contraseña predeterminada es <code>123456</code>. Para cambiar la contraseña, ejecute <code>/home/pi/pialert/back/pialert-cli</code> en el contenedor", 
        "DAYS_TO_KEEP_EVENTS_name" : "Eliminar eventos anteriores a",
        "DAYS_TO_KEEP_EVENTS_description" : "Esta es una configuración de mantenimiento. Esto especifica el número de días de entradas de eventos que se guardarán. Todos los eventos anteriores se eliminarán periódicamente.",
        "HRS_TO_KEEP_NEWDEV_name": "Guardar nuevos dispositivos para",
        "HRS_TO_KEEP_NEWDEV_description": "Esta es una configuración de mantenimiento. Si está habilitado (<code>0</code> está deshabilitado), los dispositivos marcados como <b>Nuevo dispositivo</b> se eliminarán si su <b>Primera sesión</ b> el tiempo era anterior a las horas especificadas en esta configuración. Utilice esta configuración si desea eliminar automáticamente <b>Nuevos dispositivos</b> después de <code>X</code> horas.",
        "REPORT_DASHBOARD_URL_name" : "URL de Pi.Alert",
        "REPORT_DASHBOARD_URL_description" : "Esta URL se utiliza como base para generar enlaces en los correos electrónicos. Ingrese la URL completa que comienza con <code>http://</code>, incluido el número de puerto (sin barra inclinada al final <code>/</code>).",
-        "DIG_GET_IP_ARG_name" : "Descubrir de IP de Internet",
-        "DIG_GET_IP_ARG_description" : "Cambie los argumentos de la <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">utilidad de dig</a> si tiene problemas para resolver su IP de Internet. Los argumentos se agregan al final del siguiente comando: <code>dig +short </code>.",
        "UI_LANG_name" : "Idioma de interfaz",
        "UI_LANG_description" : "Seleccione el idioma de interfaz de usuario preferido.",
        "UI_PRESENCE_name" : "Mostrar en el gráfico de presencia",
--- a/front/plugins/README.md
+++ b/front/plugins/README.md
@@ -1,47 +1,41 @@
-## 📚 Docs for individual plugins 
+> Community translations of this file (might be out-of-date): <a href="https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README_ES.md">Spanish(<img src="https://github.com/lipis/flag-icons/blob/main/flags/4x3/es.svg" alt="README_ES.md" style="height: 16px !important;width: 20px !important;padding-inline:3px !important;">)</a>, <a href="https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README_DE.md">German(<img src="https://github.com/lipis/flag-icons/blob/main/flags/4x3/de.svg" alt="README_DE.md" style="height: 16px !important;width: 20px !important;padding-inline:3px !important;">)</a> 

-### 🏴 Community translations of this file
+# 📚 Docs for individual plugins 

-> Please note there might be a delay between English and community translations.
+>[!NOTE]
+> Please check this [Plugins debugging guide](https://github.com/jokob-sk/Pi.Alert/blob/main/docs/DEBUG_PLUGINS.md) and the corresponding Plugin documentation in the below table if you are facing issues.  

-* <a href="https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README_ES.md">
-   <img src="https://github.com/lipis/flag-icons/blob/main/flags/4x3/es.svg" alt="README_ES.md" style="height: 20px !important;width: 20px !important;"> Spanish (Spain)
-  </a> 
+## 🔌 Plugins & 📚 Docs 

-* <a href="https://github.com/jokob-sk/Pi.Alert/blob/main/front/plugins/README_DE.md">
-   <img src="https://github.com/lipis/flag-icons/blob/main/flags/4x3/de.svg" alt="README_DE.md" style="height: 20px !important;width: 20px !important;"> German (Germany)
-  </a> 
-
-### 🔌 Plugins & 📚 Docs 
-
-| Required | CurrentScan | Unique Prefix | Data source        |   Type         | Link + Docs                                                      | 
-|----------|-------------|---------------|--------------------|----------------|------------------------------------------------------------------|
-|          |             | APPRISE       | Script             | 💬 publisher   | 📚[_publisher_apprise](/front/plugins/_publisher_apprise/)      |
-|          |    Yes      | ARPSCAN       | Script             | 🔍dev scanner  | 📚[arp_scan](/front/plugins/arp_scan/)                          |
-|          |             | CSVBCKP       | Script             | ⚙ system       | 📚[csv_backup](/front/plugins/csv_backup/)                      |
-|  Yes*    |             | DBCLNP        | Script             | ⚙ system       | 📚[db_cleanup](/front/plugins/db_cleanup/)                      |
-|          |             | DDNS          | Script             | ⚙ system       | 📚[ddns_update](/front/plugins/ddns_update/)                    |
-|          |    Yes      | DHCPLSS       | Script             | 🔍dev scanner  | 📚[dhcp_leases](/front/plugins/dhcp_leases/)                    |
-|          |             | DHCPSRVS      | Script             | ♻ other        | 📚[dhcp_servers](/front/plugins/dhcp_servers/)                  |
-|          |    Yes      | INTRNT        | Script             | 🔍dev scanner  | 📚[internet_ip](/front/plugins/internet_ip/)                    |
-|          |             | INTRSPD       | Script             | ♻ other        | 📚[internet_speedtest](/front/plugins/internet_speedtest/)      |
-|          |             | MAINT         | Script             | ⚙ system       | 📚[maintenance](/front/plugins/maintenance/)                    |
-|          |             | MQTT          | Script             | 💬 publisher   | 📚[_publisher_mqtt](/front/plugins/_publisher_mqtt/)            |
-|  Yes     |             | NEWDEV        | Template           | ⚙ system       | 📚[newdev_template](/front/plugins/newdev_template/)            |
-|          |             | NMAP          | Script             | ♻ other        | 📚[nmap_scan](/front/plugins/nmap_scan/)                        |
-|          |             | NTFY          | Script             | 💬 publisher   | 📚[_publisher_ntfy](/front/plugins/_publisher_ntfy/)            |
-|          |             | PHOLUS        | Script             | ♻ other        | 📚[pholus_scan](/front/plugins/pholus_scan/)                    |
-|          |    Yes      | PIHOLE        | External SQLite DB | 🔍dev scanner  | 📚[pihole_scan](/front/plugins/pihole_scan/)                    |
-|          |             | PUSHSAFER     | Script             | 💬 publisher   | 📚[_publisher_pushsafer](/front/plugins/_publisher_pushsafer/)  |
-|          |             | SETPWD        | Script             | ⚙ system       | 📚[set_password](/front/plugins/set_password/)                  |
-|          |             | SMTP          | Script             | 💬 publisher   | 📚[_publisher_email](/front/plugins/_publisher_email/)          |
-|          |    Yes      | SNMPDSC       | Script             | 🔍dev scanner  | 📚[snmp_discovery](/front/plugins/snmp_discovery/)              |
-|          |    Yes**    | UNDIS         | Script             | ♻ other        | 📚[undiscoverables](/front/plugins/undiscoverables/)            |
-|          |    Yes      | UNFIMP        | Script             | 🔍dev scanner  | 📚[unifi_import](/front/plugins/unifi_import/)                  |
-|          |             | VNDRPDT       | Script             | ⚙ system       | 📚[vendor_update](/front/plugins/vendor_update/)                |
-|          |             | WEBHOOK       | Script             | 💬 publisher   | 📚[_publisher_webhook](/front/plugins/_publisher_webhook/)      |
-|          |             | WEBMON        | Script             | ♻ other        | 📚[website_monitor](/front/plugins/website_monitor/)            |
-|  N/A     |             | N/A           | SQL query          |                 | N/A, but the External SQLite DB plugins work similar            |
+| Required | CurrentScan | Unique Prefix | Data source        |   Type         | Link + Docs                                                        | 
+|----------|-------------|---------------|--------------------|----------------|---------------------------------------------------------------------|
+|          |             | APPRISE       | Script             | 💬 publisher   | 📚[_publisher_apprise](/front/plugins/_publisher_apprise/)          |
+|          |    Yes      | ARPSCAN       | Script             | 🔍dev scanner  | 📚[arp_scan](/front/plugins/arp_scan/)                              |
+|          |             | CSVBCKP       | Script             | ⚙ system       | 📚[csv_backup](/front/plugins/csv_backup/)                          |
+|  Yes*    |             | DBCLNP        | Script             | ⚙ system       | 📚[db_cleanup](/front/plugins/db_cleanup/)                          |
+|          |             | DDNS          | Script             | ⚙ system       | 📚[ddns_update](/front/plugins/ddns_update/)                        |
+|          |    Yes      | DHCPLSS       | Script             | 🔍dev scanner  | 📚[dhcp_leases](/front/plugins/dhcp_leases/)                        |
+|          |             | DHCPSRVS      | Script             | ♻ other        | 📚[dhcp_servers](/front/plugins/dhcp_servers/)                      |
+|          |    Yes      | INTRNT        | Script             | 🔍dev scanner  | 📚[internet_ip](/front/plugins/internet_ip/)                        |
+|          |             | INTRSPD       | Script             | ♻ other        | 📚[internet_speedtest](/front/plugins/internet_speedtest/)          |
+|          |             | MAINT         | Script             | ⚙ system       | 📚[maintenance](/front/plugins/maintenance/)                        |
+|          |             | MQTT          | Script             | 💬 publisher   | 📚[_publisher_mqtt](/front/plugins/_publisher_mqtt/)                |
+|  Yes     |             | NEWDEV        | Template           | ⚙ system       | 📚[newdev_template](/front/plugins/newdev_template/)                |
+|          |             | NMAP          | Script             | ♻ other        | 📚[nmap_scan](/front/plugins/nmap_scan/)                            |
+|  Yes     |             | NTFPRCS       | Template           | ⚙ system       | 📚[notification_processing](/front/plugins/notification_processing/)|
+|          |             | NTFY          | Script             | 💬 publisher   | 📚[_publisher_ntfy](/front/plugins/_publisher_ntfy/)                |
+|          |             | PHOLUS        | Script             | ♻ other        | 📚[pholus_scan](/front/plugins/pholus_scan/)                        |
+|          |    Yes      | PIHOLE        | External SQLite DB | 🔍dev scanner  | 📚[pihole_scan](/front/plugins/pihole_scan/)                        |
+|          |             | PUSHSAFER     | Script             | 💬 publisher   | 📚[_publisher_pushsafer](/front/plugins/_publisher_pushsafer/)      |
+|          |             | SETPWD        | Script             | ⚙ system       | 📚[set_password](/front/plugins/set_password/)                      |
+|          |             | SMTP          | Script             | 💬 publisher   | 📚[_publisher_email](/front/plugins/_publisher_email/)              |
+|          |    Yes      | SNMPDSC       | Script             | 🔍dev scanner  | 📚[snmp_discovery](/front/plugins/snmp_discovery/)                  |
+|          |    Yes**    | UNDIS         | Script             | ♻ other        | 📚[undiscoverables](/front/plugins/undiscoverables/)                |
+|          |    Yes      | UNFIMP        | Script             | 🔍dev scanner  | 📚[unifi_import](/front/plugins/unifi_import/)                      |
+|          |             | VNDRPDT       | Script             | ⚙ system       | 📚[vendor_update](/front/plugins/vendor_update/)                    |
+|          |             | WEBHOOK       | Script             | 💬 publisher   | 📚[_publisher_webhook](/front/plugins/_publisher_webhook/)          |
+|          |             | WEBMON        | Script             | ♻ other        | 📚[website_monitor](/front/plugins/website_monitor/)                |
+|  N/A     |             | N/A           | SQL query          |                 | N/A, but the External SQLite DB plugins work similarly             |


 > \* The database cleanup plugin (`DBCLNP`) is not _required_ but the app will become unusable after a while if not executed.
--- a/front/plugins/_publisher_mqtt/config.json
+++ b/front/plugins/_publisher_mqtt/config.json
@@ -254,7 +254,7 @@
            "events": ["test"],   
            "type": "text.select",            
            "default_value":"disabled",
-            "options": ["disabled", "on_notification" ],
+            "options": ["disabled", "on_notification", "once", "schedule", "always_after_scan", "on_new_device" ],
            "localized": ["name", "description"],
            "name" :[{
                "language_code": "en_us",
@@ -267,7 +267,7 @@
            "description": [
                {
                "language_code": "en_us",
-                "string" : "Enable sending notifications via <a target=\"_blank\" href=\"https://www.home-assistant.io/integrations/mqtt/\">MQTT</a> to your Home Assistance instance."
+                "string" : "Enable sending notifications via <a target=\"_blank\" href=\"https://www.home-assistant.io/integrations/mqtt/\">MQTT</a> to your Home Assistance instance. Usually, <code>on_notification</code> is recommended. See the <a target=\"_blank\" href=\"https://github.com/jokob-sk/Pi.Alert/blob/main/docs/HOME_ASSISTANT.md\">PiAlert Home Assistant guide</a> for details."
 				},
                {
                "language_code": "es_es",
@@ -298,6 +298,37 @@
                "string" : "Comando a ejecutar"	
            }]
        },
+        {
+            "function": "RUN_SCHD",
+            "type": "text",
+            "default_value":"0 2 * * 3",
+            "options": [],
+            "localized": ["name", "description"],
+            "name" : [{
+                "language_code":"en_us",
+                "string" : "Schedule"
+            },
+            {
+                "language_code":"es_es",
+                "string" : "Schedule"
+            },
+            {
+                "language_code":"de_de",
+                "string" : "Schedule"
+            }],
+            "description": [{
+                "language_code":"en_us",
+                "string" : "Only enabled if you select <code>schedule</code> in the <a href=\"#MQTT_RUN\"><code>MQTT_RUN</code> setting</a>. Make sure you enter the schedule in the correct cron-like format (e.g. validate at <a href=\"https://crontab.guru/\" target=\"_blank\">crontab.guru</a>). For example entering <code>0 4 * * *</code> will run the scan after 4 am in the <a onclick=\"toggleAllSettings()\" href=\"#TIMEZONE\"><code>TIMEZONE</code> you set above</a>. Will be run NEXT time the time passes."
+            },
+            {
+                "language_code":"es_es",
+                "string" : "Solo está habilitado si selecciona <code>schedule</code> en la configuración <a href=\"#MQTT_RUN\"><code>MQTT_RUN</code></a>. Asegúrese de ingresar la programación en el formato similar a cron correcto (por ejemplo, valide en <a href=\"https://crontab.guru/\" target=\"_blank\">crontab.guru</a>). Por ejemplo, ingresar <code>0 4 * * *</code> ejecutará el escaneo después de las 4 a.m. en el <a onclick=\"toggleAllSettings()\" href=\"#TIMEZONE\"><code>TIMEZONE</ código> que configuró arriba</a>. Se ejecutará la PRÓXIMA vez que pase el tiempo."
+            },
+            {
+                "language_code":"de_de",
+                "string" : "Nur aktiviert, wenn Sie <code>schedule</code> in der <a href=\"#MQTT_RUN\"><code>MQTT_RUN</code>-Einstellung</a> auswählen. Stellen Sie sicher, dass Sie den Zeitplan im richtigen Cron-ähnlichen Format eingeben (z. B. validieren unter <a href=\"https://crontab.guru/\" target=\"_blank\">crontab.guru</a>). Wenn Sie beispielsweise <code>0 4 * * *</code> eingeben, wird der Scan nach 4 Uhr morgens in der <a onclick=\"toggleAllSettings()\" href=\"#TIMEZONE\"><code>TIMEZONE</ ausgeführt. Code> den Sie oben festgelegt haben</a>. Wird das NÄCHSTE Mal ausgeführt, wenn die Zeit vergeht."
+            }]
+        },    
        {
            "function": "RUN_TIMEOUT",
            "type": "integer",
@@ -462,6 +493,40 @@
                "language_code": "es_es",
                "string" : "Un pequeño truco: retrase la adición a la cola en caso de que el proceso se reinicie y los procesos de publicación anteriores se anulen (se necesitan ~<code>2</code>s para actualizar la configuración de un sensor en el intermediario). Probado con <code>2</code>-<code>3</code> segundos de retraso. Este retraso solo se aplica cuando se crean dispositivos (durante el primer bucle de notificación). No afecta los escaneos o notificaciones posteriores."
            }]
+        },
+        {
+            "function": "SEND_STATS",
+            "type": "boolean",
+            "default_value":true,
+            "options": [],
+            "localized": ["name", "description"],
+            "name" : [{
+                "language_code":"en_us",
+                "string" : "Send stats"
+            }
+            ],
+            "description": [{
+                "language_code":"en_us",
+                "string" : "Check to send overal device stats, such as number of Online and Offline devices."
+            }
+            ]
+        },         
+        {
+            "function": "SEND_DEVICES",
+            "type": "boolean",
+            "default_value":true,
+            "options": [],
+            "localized": ["name", "description"],
+            "name" : [{
+                "language_code":"en_us",
+                "string" : "Send devices"
+            }
+            ],
+            "description": [{
+                "language_code":"en_us",
+                "string" : "Check to send individual devices to the broker with details, such as <code>is_new</code>, <code>is_present</code>, or <code>mac_address</code> of the devices."
+            }
+            ]
        }         
    ]
 }
--- a/front/plugins/_publisher_mqtt/mqtt.py
+++ b/front/plugins/_publisher_mqtt/mqtt.py
@@ -132,6 +132,12 @@ class sensor_config:

 def publish_mqtt(client, topic, message):
    status = 1
+
+
+
+    mylog('verbose', [f"[{pluginName}] Sending MQTT topic: {topic}"])
+    mylog('verbose', [f"[{pluginName}] Sending MQTT message: {message}"])
+
    while status != 0:
        result = client.publish(
                topic=topic,
@@ -251,76 +257,80 @@ def mqtt_start(db):
    # General stats    

    # Create a generic device for overal stats
-    create_generic_device(client)
+    if get_setting_value('MQTT_SEND_STATS') == True: 
+        # Create a new device representing overall PiAlert stats   
+        create_generic_device(client)

-    # Get the data
-    row = get_device_stats(db)   
+        # Get the data
+        row = get_device_stats(db)   

-    columns = ["online","down","all","archived","new","unknown"]
+        columns = ["online","down","all","archived","new","unknown"]

-    payload = ""
+        payload = ""

-    # Update the values 
-    for column in columns:       
-        payload += '"'+column+'": ' + str(row[column]) +','       
+        # Update the values 
+        for column in columns:       
+            payload += '"'+column+'": ' + str(row[column]) +','       

-    # Publish (warap into {} and remove last ',' from above)
-    publish_mqtt(client, "system-sensors/sensor/pialert/state",              
-            '{ \
-                '+ payload[:-1] +'\
-            }'
-        )
+        # Publish (wrap into {} and remove last ',' from above)
+        publish_mqtt(client, "system-sensors/sensor/pialert/state",              
+                '{ \
+                    '+ payload[:-1] +'\
+                }'
+            )

+    # Generate device-specific MQTT messages if enabled
+    if get_setting_value('MQTT_SEND_DEVICES') == True:

-    # Specific devices
+        # Specific devices

-    # Get all devices
-    devices = get_all_devices(db)
+        # Get all devices
+        devices = get_all_devices(db)

-    sec_delay = len(devices) * int(get_setting_value('MQTT_DELAY_SEC'))*5
+        sec_delay = len(devices) * int(get_setting_value('MQTT_DELAY_SEC'))*5

-    mylog('minimal', [f"[{pluginName}]         Estimated delay: ", (sec_delay), 's ', '(', round(sec_delay/60,1) , 'min)' ])
+        mylog('minimal', [f"[{pluginName}]         Estimated delay: ", (sec_delay), 's ', '(', round(sec_delay/60,1) , 'min)' ])

-    
-    for device in devices:      
+        
+        for device in devices:      

-    
-        # Create devices in Home Assistant - send config messages
-        deviceId = 'mac_' + device["dev_MAC"].replace(" ", "").replace(":", "_").lower()
-        deviceNameDisplay = re.sub('[^a-zA-Z0-9-_\s]', '', device["dev_Name"]) 
+        
+            # Create devices in Home Assistant - send config messages
+            deviceId = 'mac_' + device["dev_MAC"].replace(" ", "").replace(":", "_").lower()
+            deviceNameDisplay = re.sub('[^a-zA-Z0-9-_\s]', '', device["dev_Name"]) 

-        create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'last_ip', 'ip-network', device["dev_MAC"])
-        create_sensor(client, deviceId, deviceNameDisplay, 'binary_sensor', 'is_present', 'wifi', device["dev_MAC"])
-        create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'mac_address', 'folder-key-network', device["dev_MAC"])
-        create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'is_new', 'bell-alert-outline', device["dev_MAC"])
-        create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'vendor', 'cog', device["dev_MAC"])
-    
-        # update device sensors in home assistant              
+            create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'last_ip', 'ip-network', device["dev_MAC"])
+            create_sensor(client, deviceId, deviceNameDisplay, 'binary_sensor', 'is_present', 'wifi', device["dev_MAC"])
+            create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'mac_address', 'folder-key-network', device["dev_MAC"])
+            create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'is_new', 'bell-alert-outline', device["dev_MAC"])
+            create_sensor(client, deviceId, deviceNameDisplay, 'sensor', 'vendor', 'cog', device["dev_MAC"])
+        
+            # update device sensors in home assistant              

-        publish_mqtt(client, 'system-sensors/sensor/'+deviceId+'/state', 
-            '{ \
-                "last_ip": "' + device["dev_LastIP"] +'", \
-                "is_new": "' + str(device["dev_NewDevice"]) +'", \
-                "vendor": "' + sanitize_string(device["dev_Vendor"]) +'", \
-                "mac_address": "' + str(device["dev_MAC"]) +'" \
-            }'
-        ) 
+            publish_mqtt(client, 'system-sensors/sensor/'+deviceId+'/state', 
+                '{ \
+                    "last_ip": "' + device["dev_LastIP"] +'", \
+                    "is_new": "' + str(device["dev_NewDevice"]) +'", \
+                    "vendor": "' + sanitize_string(device["dev_Vendor"]) +'", \
+                    "mac_address": "' + str(device["dev_MAC"]) +'" \
+                }'
+            ) 

-        publish_mqtt(client, 'system-sensors/binary_sensor/'+deviceId+'/state', 
-            '{ \
-                "is_present": "' + to_binary_sensor(str(device["dev_PresentLastScan"])) +'"\
-            }'
-        ) 
+            publish_mqtt(client, 'system-sensors/binary_sensor/'+deviceId+'/state', 
+                '{ \
+                    "is_present": "' + to_binary_sensor(str(device["dev_PresentLastScan"])) +'"\
+                }'
+            ) 

-        # delete device / topic
-        #  homeassistant/sensor/mac_44_ef_bf_c4_b1_af/is_present/config
-        # client.publish(
-        #     topic="homeassistant/sensor/"+deviceId+"/is_present/config",
-        #     payload="",
-        #     qos=1,
-        #     retain=True,
-        # )        
-    # time.sleep(10)
+            # delete device / topic
+            #  homeassistant/sensor/mac_44_ef_bf_c4_b1_af/is_present/config
+            # client.publish(
+            #     topic="homeassistant/sensor/"+deviceId+"/is_present/config",
+            #     payload="",
+            #     qos=1,
+            #     retain=True,
+            # )        
+        # time.sleep(10)


 #===============================================================================
--- a/front/plugins/arp_scan/README.md
+++ b/front/plugins/arp_scan/README.md
@@ -12,3 +12,9 @@ Arp-scan is a command-line tool that uses the ARP protocol to discover and finge
 - SAVE
 - Wait for the next scan to finish

+#### Examples
+
+Settings:
+
+![settings](/front/plugins/arp_scan/arp-scan-settings.png)
+
--- a/front/plugins/arp_scan/arp-scan-settings.png
+++ b/front/plugins/arp_scan/arp-scan-settings.png
--- a/front/plugins/arp_scan/config.json
+++ b/front/plugins/arp_scan/config.json
@@ -106,7 +106,7 @@
      "description": [
        {
          "language_code": "en_us",
-          "string": "Specify when your Network-discovery scan will run. Typical setting would be <code>schedule</code> and then you specify a cron-like schedule in the <a href=\"#ARPSCAN_RUN_SCHD\"><code>ARPSCAN_RUN_SCHD</code>setting</a> "
+          "string": "Specify when your Network-discovery scan will run. Typical setting would be <code>schedule</code> and then you specify a cron-like schedule in the <a href=\"#ARPSCAN_RUN_SCHD\"><code>ARPSCAN_RUN_SCHD</code>setting</a>. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
        },
        {
          "language_code": "es_es",
--- a/front/plugins/csv_backup/README.md
+++ b/front/plugins/csv_backup/README.md
@@ -16,4 +16,4 @@ Plugin generating CSV backups of your Devices database table, including the netw

 ### Usage

- If the devices.csv file can be overwritten or the date and time timestamp added to the name. This is toggled with the `CSVBCKP_overwrite` setting.
+- The `devices.csv` file can be overwritten or the date and time timestamp added to the name. This is toggled with the `CSVBCKP_overwrite` setting.
--- a/front/plugins/csv_backup/script.py
+++ b/front/plugins/csv_backup/script.py
@@ -18,7 +18,7 @@ sys.path.append('/home/pi/pialert/pialert')
 from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64
 from logger import mylog, append_line_to_file
 from helper import timeNowTZ
-from const import logPath, pialertPath
+from const import logPath, pialertPath, fullDbPath


 CUR_PATH = str(pathlib.Path(__file__).parent.resolve())
@@ -43,7 +43,7 @@ def main():
    mylog('verbose', ['[CSVBCKP] In script'])     

    # Connect to the PiAlert SQLite database
-    conn = sqlite3.connect('/home/pi/pialert/db/pialert.db')
+    conn = sqlite3.connect(fullDbPath)
    cursor = conn.cursor()

    # Execute your SQL query
--- a/front/plugins/db_cleanup/config.json
+++ b/front/plugins/db_cleanup/config.json
@@ -172,6 +172,25 @@
          "string": "Maximale Zeit in Sekunden, die auf den Abschluss des Skripts gewartet werden soll. Bei Überschreitung dieser Zeit wird das Skript abgebrochen."
        }        
      ]
+    },
+    {
+      "function": "NOTIFI_HIST",
+      "type": "integer",
+      "default_value": 100,
+      "options": [],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Notifications History"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "How many historical entries of Notifications should be kept. This influences how mane entries are also available in the Report section in the UI"
+        }
+      ]
    }
  ],

--- a/front/plugins/db_cleanup/script.py
+++ b/front/plugins/db_cleanup/script.py
@@ -18,7 +18,7 @@ sys.path.append('/home/pi/pialert/pialert')
 from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64
 from logger import mylog, append_line_to_file
 from helper import timeNowTZ, get_setting_value
-from const import logPath, pialertPath
+from const import logPath, pialertPath, fullDbPath


 CUR_PATH = str(pathlib.Path(__file__).parent.resolve())
@@ -44,7 +44,7 @@ def main():


    # Execute cleanup/upkeep    
-    cleanup_database('/home/pi/pialert/db/pialert.db', DAYS_TO_KEEP_EVENTS, PHOLUS_DAYS_DATA, HRS_TO_KEEP_NEWDEV, PLUGINS_KEEP_HIST)
+    cleanup_database(fullDbPath, DAYS_TO_KEEP_EVENTS, PHOLUS_DAYS_DATA, HRS_TO_KEEP_NEWDEV, PLUGINS_KEEP_HIST)
    
    mylog('verbose', ['[DBCLNP] Cleanup complete file '])   
    
--- a/front/plugins/dhcp_leases/config.json
+++ b/front/plugins/dhcp_leases/config.json
@@ -478,7 +478,7 @@
      "description": [
        {
          "language_code": "en_us",
-          "string": "Enable import of devices from <code>dhcp.leases</code> files. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings."
+          "string": "Enable import of devices from <code>dhcp.leases</code> files. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
        },
        {
          "language_code": "es_es",
--- a/front/plugins/dhcp_servers/config.json
+++ b/front/plugins/dhcp_servers/config.json
@@ -290,7 +290,7 @@
            }],
            "description": [{
                "language_code":"en_us",
-                "string" : "Enable a regular scan of rogue DHCP servers. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings."
+                "string" : "Enable a regular scan of rogue DHCP servers. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
            },
            {
                "language_code":"es_es",
--- a/front/plugins/internet_ip/config.json
+++ b/front/plugins/internet_ip/config.json
@@ -57,9 +57,9 @@
      "value": "SELECT dev_LastIP FROM Devices WHERE dev_MAC = 'Internet' "
    },
    {
-      "name": "DIG_GET_IP_ARG",
+      "name": "INTRNT_DIG_GET_IP_ARG",
      "type": "setting",
-      "value": "DIG_GET_IP_ARG",
+      "value": "INTRNT_DIG_GET_IP_ARG",
      "base64": true
    }
  ],
@@ -109,7 +109,7 @@
    {
      "function": "CMD",
      "type": "readonly",
-      "default_value": "python3 /home/pi/pialert/front/plugins/internet_ip/script.py prev_ip={prev_ip} DIG_GET_IP_ARG={DIG_GET_IP_ARG}",
+      "default_value": "python3 /home/pi/pialert/front/plugins/internet_ip/script.py prev_ip={prev_ip} INTRNT_DIG_GET_IP_ARG={INTRNT_DIG_GET_IP_ARG}",
      "options": [],
      "localized": [
        "name",
@@ -144,6 +144,44 @@
        }
      ]
    },
+    {
+      "function": "DIG_GET_IP_ARG",
+      "type": "text",
+      "default_value": "-4 myip.opendns.com @resolver1.opendns.com",
+      "options": [],
+      "localized": [
+        "name",
+        "description"
+      ],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Internet IP discovery"
+        },
+        {
+          "language_code": "es_es",
+          "string": "Descubrir de IP de Internet"
+        },
+        {
+          "language_code": "de_de",
+          "string": "Erkennung externer IP (\"Internet IP\")"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "Change the <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">dig utility</a> arguments if you have issues resolving your Internet IP. Arguments are added at the end of the following command: <code>dig +short </code>."
+        },
+        {
+          "language_code": "es_es",
+          "string": "Cambie los argumentos de la <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">utilidad de dig</a> si tiene problemas para resolver su IP de Internet. Los argumentos se agregan al final del siguiente comando: <code>dig +short </code>."
+        },
+        {
+          "language_code": "de_de",
+          "string": "Ändere die Argumente des <a href=\"https://linux.die.net/man/1/dig\" target=\"_blank\">dig Dienstprogramms</a>, wenn Probleme beim Auflösen der externen IP auftreten. Argumente werden an das Ende des folgenden Befehls angehängt: <code>dig +short </code>."
+        }
+      ]
+    },
    {
      "function": "RUN_SCHD",
      "type": "text",
--- a/front/plugins/internet_ip/script.py
+++ b/front/plugins/internet_ip/script.py
@@ -20,7 +20,7 @@ sys.path.append('/home/pi/pialert/pialert')

 from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64
 from logger import mylog, append_line_to_file
-from helper import timeNowTZ, check_IP_format
+from helper import timeNowTZ, check_IP_format, get_setting_value
 from const import logPath, pialertPath, fullDbPath


@@ -37,19 +37,19 @@ def main():
    parser = argparse.ArgumentParser(description='Check internet connectivity and IP')
    
    parser.add_argument('prev_ip', action="store", help="Previous IP address to compare against the current IP")
-    parser.add_argument('DIG_GET_IP_ARG', action="store", help="Arguments for the 'dig' command to retrieve the IP address")
+    parser.add_argument('DIG_GET_IP_ARG', action="store", help="Arguments for the 'dig' command to retrieve the IP address") # unused

    values = parser.parse_args()

    PREV_IP         = values.prev_ip.split('=')[1]        
-    DIG_GET_IP_ARG  = values.DIG_GET_IP_ARG.split('=b')[1]   # byte64 encoded
+    DIG_GET_IP_ARG  = get_setting_value("INTRNT_DIG_GET_IP_ARG")

-    mylog('verbose', [f'[{pluginName}] DIG_GET_IP_ARG: ', DIG_GET_IP_ARG])     
+    mylog('verbose', [f'[{pluginName}] INTRNT_DIG_GET_IP_ARG: ', DIG_GET_IP_ARG])     

    # Decode the base64-encoded value to get the actual value in ASCII format.
-    DIG_GET_IP_ARG = base64.b64decode(DIG_GET_IP_ARG).decode('ascii')
+    # DIG_GET_IP_ARG = base64.b64decode(DIG_GET_IP_ARG).decode('ascii')
    
-    mylog('verbose', [f'[{pluginName}] DIG_GET_IP_ARG resolved: {DIG_GET_IP_ARG} ']) 
+    # mylog('verbose', [f'[{pluginName}] DIG_GET_IP_ARG resolved: {DIG_GET_IP_ARG} ']) 

    # perform the new IP lookup
    new_internet_IP, cmd_output = check_internet_IP( PREV_IP, DIG_GET_IP_ARG)   
--- a/front/plugins/newdev_template/config.json
+++ b/front/plugins/newdev_template/config.json
@@ -395,7 +395,7 @@
        {
          "function": "dev_NewDevice",
          "type": "integer.checkbox",
-          "default_value": true,
+          "default_value": 1,
          "options": [],
          "localized": ["name", "description"],
          "name": [
--- a/front/plugins/notification_processing/README.md
+++ b/front/plugins/notification_processing/README.md
@@ -0,0 +1,7 @@
+## Overview
+
+Plugin supplying settings for Notification Processing.
+
+### Usage
+
+- Check the Settings page for details.
--- a/front/plugins/notification_processing/config.json
+++ b/front/plugins/notification_processing/config.json
@@ -0,0 +1,129 @@
+{
+  "code_name": "notification_processing",
+  "unique_prefix": "NTFPRCS",
+  "plugin_type": "system",
+  "enabled": true,
+  "data_source": "script",  
+  "show_ui": false,
+  "localized": ["display_name", "description", "icon"],
+  "display_name": [
+    {
+      "language_code": "en_us",
+      "string": "Notification Processing"
+    }
+  ],
+  "icon": [
+    {
+      "language_code": "en_us",
+      "string": "<i class=\"fa-solid fa-envelopes-bulk\"></i>"
+    }
+  ],
+  "description": [
+    {
+      "language_code": "en_us",
+      "string": "A plugin to for advanced notification processing."
+    }  
+  ],
+    "params" : [
+  ],  
+
+  "settings": [   
+    {
+      "function": "INCLUDED_SECTIONS",
+      "type": "text.multiselect",
+      "default_value": ["new_devices", "down_devices", "events"],
+      "options": ["new_devices", "down_devices", "events", "plugins"],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Notify on"
+        },
+        {
+          "language_code": "de_de",
+          "string": "Benachrichtigungen"
+        },
+        {
+          "language_code": "es_es",
+          "string": "Notificar en"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "Specifies which events trigger notifications. Remove the event type(s) you do not want to get notified on. This setting overrides device-specific settings in the UI. (<code>CTRL + Click</code> to select/deselect)."
+        },    
+        {
+          "language_code": "de_de",
+          "string": "Spezifiziert, bei welchen Events Benachrichtigungen versendet werden. Entfernen Sie die Eventtypen, bei welchen Sie nicht benachrichtigt werden wollen. Diese Einstellung überschreibt gerätespezifische Einstellungen im UI. (<code>STRG + klicken</code> zum aus-/abwählen)."
+        },    
+        {
+          "language_code": "es_es",
+          "string": "Especifica que eventos envían notificaciones. Elimina los tipos de eventos de los que no quieras recibir notificaciones. Este ajuste sobreescribe los ajustes específicos de los dispositivos en la interfaz. (<code>CTRL + Clic</code> para seleccionar / deseleccionar)."
+        }
+      ]    
+    },
+    {
+      "function": "alert_down_time",
+      "type": "integer",
+      "default_value": 5,
+      "options": [],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Alert Down After"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "After how many minutes a device is reported as down and a notification is sent."
+        }    
+      ]    
+    },
+    {
+      "function": "new_dev_condition",
+      "type": "text",
+      "default_value": "",
+      "options": [],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "New Devices Filter"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "You can specify a SQL where condition to filter out New Devices from notifications. For example <code>AND dev_LastIP NOT LIKE '192.168.3.%'</code> will always exlude New Device notifications for all devices with the IP starting with <code>192.168.3.%</code>."
+        }    
+      ]        
+    },
+    {
+      "function": "event_condition",
+      "type": "text",
+      "default_value": "",
+      "options": [],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Events Filter"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "You can specify a SQL where condition to filter out Events from notifications. For example <code>AND dev_LastIP NOT LIKE '192.168.3.%'</code> will always exlude New Device notifications for all devices with the IP starting with <code>192.168.3.%</code>."
+        }    
+      ]    
+    }
+  ],
+
+  "database_column_definitions": 
+  [          
+           
+  ]
+}
--- a/front/plugins/pihole_scan/config.json
+++ b/front/plugins/pihole_scan/config.json
@@ -73,7 +73,7 @@
        }],
        "description": [{
            "language_code":"en_us",
-            "string" : "Specify when your PiHole device import from the PiHole database will run. The typical setting would be <code>schedule</code> and then you specify a cron-like schedule in the <a href=\"#PIHOLE_RUN_SCHD\"><code>PIHOLE_RUN_SCHD</code>setting</a>. If enabled, you must map the pihole db into your container to the <code>:/etc/pihole/pihole-FTL.db</code> mount path as specified in the <code>DB_PATH</code> setting."
+            "string" : "Specify when your PiHole device import from the PiHole database will run. The typical setting would be <code>schedule</code> and then you specify a cron-like schedule in the <a href=\"#PIHOLE_RUN_SCHD\"><code>PIHOLE_RUN_SCHD</code>setting</a>. If enabled, you must map the pihole db into your container to the <code>:/etc/pihole/pihole-FTL.db</code> mount path as specified in the <code>DB_PATH</code> setting. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
        },
        {
            "language_code":"es_es",
@@ -83,7 +83,7 @@
    {
        "function": "CMD",
        "type": "text",
-        "default_value":"SELECT n.hwaddr AS Object_PrimaryID, {s-quote}null{s-quote} AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr <> {s-quote}00:00:00:00:00:00{s-quote} AND na.ip <> null;",
+        "default_value":"SELECT n.hwaddr AS Object_PrimaryID, {s-quote}null{s-quote} AS Object_SecondaryID, datetime() AS DateTime, na.ip  AS Watched_Value1, n.lastQuery AS Watched_Value2, na.name AS Watched_Value3, n.macVendor AS Watched_Value4, {s-quote}null{s-quote} AS Extra, n.hwaddr AS ForeignKey FROM EXTERNAL_PIHOLE.Network AS n LEFT JOIN EXTERNAL_PIHOLE.Network_Addresses AS na ON na.network_id = n.id WHERE n.hwaddr NOT LIKE {s-quote}ip-%{s-quote} AND n.hwaddr is not {s-quote}00:00:00:00:00:00{s-quote}  AND na.ip is not null",
        "options": [],
        "localized": ["name", "description"],
        "name" : [{
--- a/front/plugins/set_password/config.json
+++ b/front/plugins/set_password/config.json
@@ -67,7 +67,7 @@
  },
  {
      "function": "CMD",
-      "type": "text",
+      "type": "readonly",
      "default_value":"/home/pi/pialert/back/pialert-cli set_password {password}",
      "options": [],
      "localized": ["name", "description"],
@@ -108,7 +108,7 @@
    "description": [
      {
        "language_code": "en_us",
-        "string": "The default password is <code>123456</code>. To change the password run <code>/home/pi/pialert/back/pialert-cli set_password {password}</code> in the container"
+        "string": "The default password is <code>123456</code>. To change it, you can either use this plugin (follow the instructions in the <code>SETPWD_RUN</code> setting) or run <code>/home/pi/pialert/back/pialert-cli set_password {password}</code> in the container."
      },
      {
        "language_code": "es_es",
--- a/front/plugins/snmp_discovery/config.json
+++ b/front/plugins/snmp_discovery/config.json
@@ -322,7 +322,7 @@
            }],
            "description": [{
                "language_code":"en_us",
-                "string" : "Enable import of devices from a SNMP enabled device. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings."
+                "string" : "Enable import of devices from a SNMP enabled device. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
            },
            {
                "language_code":"es_es",
--- a/front/plugins/unifi_import/config.json
+++ b/front/plugins/unifi_import/config.json
@@ -479,7 +479,7 @@
            "description": [
                {
                    "language_code": "en_us",
-                    "string": "Enable import of devices from a UNIFI controller. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings."
+                    "string": "Enable import of devices from a UNIFI controller. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings. ⚠ Use the same schedule if you have multiple <i class=\"fa-solid fa-magnifying-glass-plus\"></i> Device scanners enabled."
                },
                {
                    "language_code": "es_es",
--- a/front/plugins/vendor_update/script.py
+++ b/front/plugins/vendor_update/script.py
@@ -19,7 +19,7 @@ sys.path.append('/home/pi/pialert/pialert')
 from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64, handleEmpty
 from logger import mylog, append_line_to_file
 from helper import timeNowTZ
-from const import logPath, pialertPath
+from const import logPath, pialertPath, fullDbPath
 from device import query_MAC_vendor


@@ -37,7 +37,7 @@ def main():
    # Resolve missing vendors
    plugin_objects = Plugin_Objects(RESULT_FILE)

-    plugin_objects = update_vendors('/home/pi/pialert/db/pialert.db', plugin_objects)
+    plugin_objects = update_vendors(fullDbPath, plugin_objects)

    plugin_objects.write_result_file()
    
@@ -60,8 +60,8 @@ def update_vendor_database():
        update_output = subprocess.check_output (update_args)
    except subprocess.CalledProcessError as e:
        # An error occured, handle it
-        mylog('none', ['    FAILED: Updating vendors DB, set LOG_LEVEL=debug for more info'])  
-        mylog('none', [e.output])        
+        mylog('verbose', ['    FAILED: Updating vendors DB, set LOG_LEVEL=debug for more info'])  
+        mylog('verbose', [e.output])        

 # ------------------------------------------------------------------------------
 # resolve missing vendors
--- a/front/settings.php
+++ b/front/settings.php
@@ -245,21 +245,11 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
        `)
    } 
    
-    // Start constructing the main settings HTML 
-    let pluginHtml = `
-          <div class="row table_row">
-            <div class="table_cell bold">
-              <i class="fa-regular fa-book fa-sm"></i>
-              <a href="https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins" target="_blank">
-                ${getString('Gen_ReadDocs')}
-              </a>
-            </div>
-          </div>
-        `;
+

    let isIn = ' in '; // to open the active panel in AdminLTE

-    for (const group of settingGroups) {     
+    for (const group of settingGroups) {   

      // enabled / disabled icons
      enabledHtml = ''
@@ -277,7 +267,20 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
                      `
      }      

-      headerHtml = `<div class="box box-solid box-primary panel panel-default">
+      // Start constructing the main settings HTML 
+      let pluginHtml = `
+        <div class="row table_row">
+          <div class="table_cell bold">
+            <i class="fa-regular fa-book fa-sm"></i>
+            <a href="https://github.com/jokob-sk/Pi.Alert/tree/main/front/plugins/${getPluginCodeName(pluginsData, group)}" target="_blank">
+              ${getString('Gen_ReadDocs')}
+            </a>
+          </div>
+        </div>
+      `;
+
+      // Plugin HEADER
+      headerHtml = `<div class="box box-solid box-primary panel panel-default" id="${group}_header">
                  <a data-toggle="collapse" data-parent="#accordion_gen" href="#${group}">
                    <div class="panel-heading">
                      <h4 class="panel-title">
@@ -787,37 +790,6 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {

  }
  
-  
-  
-  // -----------------------------------------------------------------------------
-  function toggleAllSettings()
-  {
-    inStr = ' in';
-    allOpen = true;
-    openIcon = 'fa-angle-double-down';
-    closeIcon = 'fa-angle-double-up';    
-
-    $('.panel-collapse').each(function(){
-      if($(this).attr('class').indexOf(inStr) == -1)
-      {
-        allOpen = false;
-      }
-    })
-    
-    if(allOpen)
-    {
-      // close all
-      $('div[data-myid="collapsible"]').each(function(){$(this).attr('class', 'panel-collapse collapse  ')})      
-      $('#toggleSettings').attr('class', $('#toggleSettings').attr('class').replace(closeIcon, openIcon))
-    }
-    else{
-      // open all
-      $('div[data-myid="collapsible"]').each(function(){$(this).attr('class', 'panel-collapse collapse in')})
-      $('div[data-myid="collapsible"]').each(function(){$(this).attr('style', 'height:inherit')})
-      $('#toggleSettings').attr('class', $('#toggleSettings').attr('class').replace(openIcon, closeIcon))
-    }
-    
-  }

  getData()

--- a/pialert/README.md
+++ b/pialert/README.md
@@ -7,18 +7,22 @@ The original pilaert.py code is now moved to this new folder and split into diff
 |```__main__.py```| The MAIN program of Pi.Alert|
 |```__init__.py```| an empty init file|
 |```README.md```| this readme file|
-|**publishers**| a folder containing all modules used to publish the results|
-|```api.py```| updating the API endpoints with the relevant data. (Should move to publishers)|
+|```../front/plugins ```| a folder containing all [plugins](/front/plugins/) that publish notifications or scan for devices|
+|```api.py```| updating the API endpoints with the relevant data. |
+|```appevent.py```| TBC |
 |```const.py```| A place to define the constants for Pi.Alert like log path or config path.|
 |```conf.py```| conf.py holds the configuration variables and makes them available for all modules. It is also the <b>workaround</b> for global variables that need to be resolved at some point|
 |```database.py```| This module connects to the DB, makes sure the DB is up to date and defines some standard queries and interfaces. |
 |```device.py```| The device module looks after the devices and saves the scan results into the devices |
+|```flows.py```| TBC |
 |```helper.py```| Helper as the name suggest contains multiple little functions and methods used in many of the other modules and helps keep things clean |
 |```initialise.py```| Initiatlise sets up the environment and makes everything ready to go |
 |```logger.py```| Logger is there the keep all the logs organised and looking identical. |
-|```networscan.py```| Networkscan orchestrates the actual scanning of the network, calling the individual scanners and managing the results |
+|```networscan.py```| Networkscan collects the scan results (maybe to merge with `reporting.py`) |
+|```notification.py```| Creates and handles the notification object and generates ther HTML and text variants of the message |
 |```plugin.py```| This is where the plugins get integrated into the backend of Pi.Alert |
-|```reporting.py```| Reporting generates the email, html and json reports to be sent by the publishers |
+|```plugin_utils.py```| Helper utilities for `plugin.py` |
+|```reporting.py```| Reporting collects the data for the notification reports |
 |```scheduler.py```| All things scheduling |


--- a/pialert/main.py
+++ b/pialert/main.py
@@ -151,32 +151,19 @@ def main ():
            # ----------------------------------------

            # send all configured notifications
-            notiStructure = get_notifications(db)
+            final_json = get_notifications(db)

            # Write the notifications into the DB
            notification    = Notification_obj(db)
-            notificationObj = notification.create(notiStructure.json, notiStructure.text, notiStructure.html, "")
+            notificationObj = notification.create(final_json, "")

            # run all enabled publisher gateways 
            if notificationObj.HasNotifications:
                pluginsState = run_plugin_scripts(db, 'on_notification', pluginsState) 
                notification.setAllProcessed()
+                notification.clearPendingEmailFlag()

-                # Clean Pending Alert Events
-                sql.execute ("""UPDATE Devices SET dev_LastNotification = ?
-                                    WHERE dev_MAC IN (
-                                        SELECT eve_MAC FROM Events
-                                            WHERE eve_PendingAlertEmail = 1
-                                    )
-                             """, (timeNowTZ(),) )
-                sql.execute ("""UPDATE Events SET eve_PendingAlertEmail = 0
-                                    WHERE eve_PendingAlertEmail = 1""")
-
-                # clear plugin events
-                sql.execute ("DELETE FROM Plugins_Events")
-
-                # DEBUG - print number of rows updated
-                mylog('minimal', ['[Notification] Notifications changes: ', sql.rowcount])
+                
            else:
                mylog('verbose', ['[Notification] No changes to report'])

--- a/pialert/conf.py
+++ b/pialert/conf.py
@@ -32,12 +32,10 @@ arpscan_devices = []
 SCAN_SUBNETS    = ['192.168.1.0/24 --interface=eth1', '192.168.1.0/24 --interface=eth0']   
 LOG_LEVEL       = 'verbose' 
 TIMEZONE        = 'Europe/Berlin'
-DIG_GET_IP_ARG  = '-4 myip.opendns.com @resolver1.opendns.com' 
 UI_LANG         = 'English' 
 UI_PRESENCE     = ['online', 'offline', 'archived']  
 PIALERT_WEB_PROTECTION  = False 
 PIALERT_WEB_PASSWORD    = '8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92' 
-INCLUDED_SECTIONS       = ['new_devices', 'down_devices', 'events']   
 DAYS_TO_KEEP_EVENTS     = 90 
 REPORT_DASHBOARD_URL    = 'http://pi.alert/' 

--- a/pialert/device.py
+++ b/pialert/device.py
@@ -41,8 +41,8 @@ def print_scan_stats(db):
    SELECT
        (SELECT COUNT(*) FROM CurrentScan) AS devices_detected,
        (SELECT COUNT(*) FROM CurrentScan WHERE NOT EXISTS (SELECT 1 FROM Devices WHERE dev_MAC = cur_MAC)) AS new_devices,
-        (SELECT COUNT(*) FROM Devices WHERE dev_AlertDeviceDown = 1 AND NOT EXISTS (SELECT 1 FROM CurrentScan WHERE dev_MAC = cur_MAC)) AS down_alerts,
-        (SELECT COUNT(*) FROM Devices WHERE dev_AlertDeviceDown = 1 AND dev_PresentLastScan = 1 AND NOT EXISTS (SELECT 1 FROM CurrentScan WHERE dev_MAC = cur_MAC)) AS new_down_alerts,
+        (SELECT COUNT(*) FROM Devices WHERE dev_AlertDeviceDown != 0 AND NOT EXISTS (SELECT 1 FROM CurrentScan WHERE dev_MAC = cur_MAC)) AS down_alerts,
+        (SELECT COUNT(*) FROM Devices WHERE dev_AlertDeviceDown != 0 AND dev_PresentLastScan = 1 AND NOT EXISTS (SELECT 1 FROM CurrentScan WHERE dev_MAC = cur_MAC)) AS new_down_alerts,
        (SELECT COUNT(*) FROM Devices WHERE dev_PresentLastScan = 0) AS new_connections,
        (SELECT COUNT(*) FROM Devices WHERE dev_PresentLastScan = 1 AND NOT EXISTS (SELECT 1 FROM CurrentScan WHERE dev_MAC = cur_MAC)) AS disconnections,
        (SELECT COUNT(*) FROM Devices, CurrentScan WHERE dev_MAC = cur_MAC AND dev_LastIP <> cur_IP) AS ip_changes,
@@ -365,10 +365,15 @@ def query_MAC_vendor (pMAC):
    try:
        with open(vendorsPath, 'r') as f:
            for line in f:
-                if line.startswith(mac_start_string6):
-                    vendor = line.split(' ', 1)[1].strip()
-                    mylog('debug', [f"[Vendor Check] Found '{vendor}' for '{pMAC}' in {vendorsPath}"])
-                    return vendor
+                if line.startswith(mac_start_string6):                    
+                    parts = line.split(' ', 1)
+                    if len(parts) > 1:
+                        vendor = parts[1].strip()
+                        mylog('debug', [f"[Vendor Check] Found '{vendor}' for '{pMAC}' in {vendorsPath}"])
+                        return vendor
+                    else:
+                        mylog('debug', [f'[Vendor Check] ⚠ ERROR: Match found, but line could not be processed: "{line}"'])
+                        return -1


        return -1  # MAC address not found in the database
--- a/pialert/helper.py
+++ b/pialert/helper.py
@@ -37,6 +37,12 @@ def timeNowTZ():
 def timeNow():
    return datetime.datetime.now().replace(microsecond=0)

+def get_timezone_offset():    
+    now = datetime.datetime.now(conf.tz)
+    offset_hours = now.utcoffset().total_seconds() / 3600        
+    offset_formatted =  "{:+03d}:{:02d}".format(int(offset_hours), int((offset_hours % 1) * 60))
+    return offset_formatted
+

 #-------------------------------------------------------------------------------
 # App state
@@ -376,7 +382,7 @@ def resolve_device_name_dig (pMAC, pIP):
        # Cleanup
        newName = cleanDeviceName(newName, True)

-        if newName == "" or  len(newName) == 0 or newName == '-1' or newName == -1 or "communications error" in newName: 
+        if newName == "" or  len(newName) == 0 or newName == '-1' or newName == -1 or "communications error" in newName or 'malformed message packet' in newName : 
            return nameNotFound

        # all checks passed
@@ -470,58 +476,6 @@ def resolve_device_name_pholus (pMAC, pIP, allRes, nameNotFound, match_IP = Fals
        if 'PTR Class:IN' in value and len(value.split('"')) > 1:
            return cleanDeviceName(value.split('"')[1], match_IP)

-
-    # # airplay matches contain a lot of information
-    # # Matches for example: 
-    # # Brand Tv (50)._airplay._tcp.local. TXT Class:32769 "acl=0 deviceid=66:66:66:66:66:66 features=0x77777,0x38BCB46 rsf=0x3 fv=p20.T-FFFFFF-03.1 flags=0x204 model=XXXX manufacturer=Brand serialNumber=XXXXXXXXXXX protovers=1.1 srcvers=777.77.77 pi=FF:FF:FF:FF:FF:FF psi=00000000-0000-0000-0000-FFFFFFFFFF gid=00000000-0000-0000-0000-FFFFFFFFFF gcgl=0 pk=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and '._airplay._tcp.local. TXT Class:32769' in str(allRes[i]["Value"]) :
-    #         return cleanDeviceName(allRes[i]["Value"].split('._airplay._tcp.local. TXT Class:32769')[0], match_IP)
-    
-    # # second best - contains airplay
-    # # Matches for example: 
-    # # _airplay._tcp.local. PTR Class:IN "Brand Tv (50)._airplay._tcp.local."
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and '_airplay._tcp.local. PTR Class:IN' in allRes[i]["Value"] and ('._googlecast') not in allRes[i]["Value"]:
-    #         return cleanDeviceName(allRes[i]["Value"].split('"')[1], match_IP)    
-
-    # # Contains PTR Class:32769
-    # # Matches for example: 
-    # # 3.1.168.192.in-addr.arpa. PTR Class:32769 "MyPc.local."
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and 'PTR Class:32769' in allRes[i]["Value"]:
-    #         return cleanDeviceName(allRes[i]["Value"].split('"')[1], match_IP)
-
-    # # Contains AAAA Class:IN
-    # # Matches for example: 
-    # # DESKTOP-SOMEID.local. AAAA Class:IN "fe80::fe80:fe80:fe80:fe80"
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and 'AAAA Class:IN' in allRes[i]["Value"]:
-    #         return cleanDeviceName(allRes[i]["Value"].split('.local.')[0], match_IP)
-
-    # # Contains _googlecast._tcp.local. PTR Class:IN
-    # # Matches for example: 
-    # # _googlecast._tcp.local. PTR Class:IN "Nest-Audio-ff77ff77ff77ff77ff77ff77ff77ff77._googlecast._tcp.local."
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and '_googlecast._tcp.local. PTR Class:IN' in allRes[i]["Value"] and ('Google-Cast-Group') not in allRes[i]["Value"]:
-    #         return cleanDeviceName(allRes[i]["Value"].split('"')[1], match_IP)
-
-    # # Contains A Class:32769
-    # # Matches for example: 
-    # # Android.local. A Class:32769 "192.168.1.6"
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and ' A Class:32769' in allRes[i]["Value"]:
-    #         return cleanDeviceName(allRes[i]["Value"].split(' A Class:32769')[0], match_IP)
-
-
-    # # # Contains PTR Class:IN
-    # # Matches for example: 
-    # # _esphomelib._tcp.local. PTR Class:IN "ceiling-light-1._esphomelib._tcp.local."
-    # for i in pholusMatchesIndexes:
-    #     if checkIPV4(allRes[i]['IP_v4_or_v6']) and 'PTR Class:IN' in allRes[i]["Value"]:
-    #         if allRes[i]["Value"] and len(allRes[i]["Value"].split('"')) > 1:
-    #             return cleanDeviceName(allRes[i]["Value"].split('"')[1], match_IP)
-
    return nameNotFound
    
    
--- a/pialert/initialise.py
+++ b/pialert/initialise.py
@@ -103,15 +103,12 @@ def importConfigs (db):
    conf.TIMEZONE = ccd('TIMEZONE', 'Europe/Berlin' , c_d, 'Time zone', 'text', '', 'General')    
    conf.PLUGINS_KEEP_HIST = ccd('PLUGINS_KEEP_HIST', 250 , c_d, 'Keep history entries', 'integer', '', 'General') 
    conf.PIALERT_WEB_PROTECTION = ccd('PIALERT_WEB_PROTECTION', False , c_d, 'Enable logon', 'boolean', '', 'General')
-    conf.PIALERT_WEB_PASSWORD = ccd('PIALERT_WEB_PASSWORD', '8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92' , c_d, 'Logon password', 'readonly', '', 'General')
-    conf.INCLUDED_SECTIONS = ccd('INCLUDED_SECTIONS', ['new_devices', 'down_devices', 'events']   , c_d, 'Notify on', 'text.multiselect', "['new_devices', 'down_devices', 'events', 'plugins']", 'General')    
+    conf.PIALERT_WEB_PASSWORD = ccd('PIALERT_WEB_PASSWORD', '8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92' , c_d, 'Logon password', 'readonly', '', 'General')    
    conf.REPORT_DASHBOARD_URL = ccd('REPORT_DASHBOARD_URL', 'http://pi.alert/' , c_d, 'PiAlert URL', 'text', '', 'General')
-    conf.DIG_GET_IP_ARG = ccd('DIG_GET_IP_ARG', '-4 myip.opendns.com @resolver1.opendns.com' , c_d, 'DIG arguments', 'text', '', 'General')
    conf.UI_LANG = ccd('UI_LANG', 'English' , c_d, 'Language Interface', 'text.select', "['English', 'German', 'Spanish']", 'General')
    conf.UI_PRESENCE = ccd('UI_PRESENCE', ['online', 'offline', 'archived']   , c_d, 'Include in presence', 'text.multiselect', "['online', 'offline', 'archived']", 'General')    
    conf.DAYS_TO_KEEP_EVENTS = ccd('DAYS_TO_KEEP_EVENTS', 90 , c_d, 'Delete events days', 'integer', '', 'General')
-    conf.HRS_TO_KEEP_NEWDEV = ccd('HRS_TO_KEEP_NEWDEV', 0 , c_d, 'Keep new devices for', 'integer', "0", 'General')    
-    conf.DBCLNP_NOTIFI_HIST = ccd('DBCLNP_NOTIFI_HIST', 100 , c_d, 'Keep notification', 'integer', "0", 'General')    
+    conf.HRS_TO_KEEP_NEWDEV = ccd('HRS_TO_KEEP_NEWDEV', 0 , c_d, 'Keep new devices for', 'integer', "0", 'General')        
    conf.API_CUSTOM_SQL = ccd('API_CUSTOM_SQL', 'SELECT * FROM Devices WHERE dev_PresentLastScan = 0' , c_d, 'Custom endpoint', 'text', '', 'General')
    conf.NETWORK_DEVICE_TYPES = ccd('NETWORK_DEVICE_TYPES', ['AP', 'Gateway', 'Firewall', 'Hypervisor', 'Powerline', 'Switch', 'WLAN', 'PLC', 'Router','USB LAN Adapter', 'USB WIFI Adapter', 'Internet'] , c_d, 'Network device types', 'list', '', 'General')

@@ -251,6 +248,7 @@ def read_config_file(filename):

 #-------------------------------------------------------------------------------
 # DEPERECATED soonest after 3/3/2024
+# 🤔Idea/TODO: Check and compare versions/timestamps amd only perform a replacement if config/version older than...
 replacements = {
    r'\bREPORT_TO\b': 'SMTP_REPORT_TO',
    r'\bREPORT_FROM\b': 'SMTP_REPORT_FROM',
@@ -259,7 +257,10 @@ replacements = {
    r'REPORT_NTFY=True': 'NTFY_RUN=\'on_notification\'',
    r'REPORT_WEBHOOK=True': 'WEBHOOK_RUN=\'on_notification\'',
    r'REPORT_PUSHSAFER=True': 'PUSHSAFER_RUN=\'on_notification\'',
-    r'REPORT_MQTT=True': 'MQTT_RUN=\'on_notification\''
+    r'REPORT_MQTT=True': 'MQTT_RUN=\'on_notification\'',
+    r'PIHOLE_CMD=': 'PIHOLE_CMD_OLD=',
+    r'\bINCLUDED_SECTIONS\b': 'NTFPRCS_INCLUDED_SECTIONS',
+    r'\bDIG_GET_IP_ARG\b': 'INTRNT_DIG_GET_IP_ARG'
 }

 def renameSettings(config_file):
--- a/pialert/networkscan.py
+++ b/pialert/networkscan.py
@@ -187,7 +187,7 @@ def insert_events (db):
                        eve_PendingAlertEmail)
                    SELECT dev_MAC, dev_LastIP, '{startTime}', 'Device Down', '', 1
                    FROM Devices 
-                    WHERE dev_AlertDeviceDown = 1
+                    WHERE dev_AlertDeviceDown != 0
                      AND dev_PresentLastScan = 1                      
                      AND NOT EXISTS (SELECT 1 FROM CurrentScan
                                      WHERE dev_MAC = cur_MAC
--- a/pialert/notification.py
+++ b/pialert/notification.py
@@ -1,13 +1,16 @@
 import datetime
 import json
 import uuid
+import socket
+import subprocess
+from json2table import convert

 # PiAlert modules
 import conf
 import const
 from const import pialertPath, logPath, apiPath
 from logger import logResult, mylog, print_log
-from helper import  timeNowTZ
+from helper import generate_mac_links, removeDuplicateNewLines, timeNowTZ, get_file_content, write_file, get_setting_value, get_timezone_offset

 #-------------------------------------------------------------------------------
 # Notification object handling
@@ -34,11 +37,22 @@ class Notification_obj:

        self.save()

-    # Create a new DB entry if new notiifcations available, otherwise skip
-    def create(self, JSON, Text, HTML, Extra=""):        
+    # Method to override processing of notifications
+    def on_before_create(self, JSON, Extra):  
+
+        return JSON, Extra
+
+
+    # Create a new DB entry if new notifications available, otherwise skip
+    def create(self, JSON, Extra=""):  
+
+        JSON, Extra = self.on_before_create(JSON, Extra)
+
+        #  Write output data for debug
+        write_file (logPath + '/report_output.json', json.dumps(JSON))
        
        # Check if nothing to report, end
-        if JSON["internet"] == [] and JSON["new_devices"] == [] and JSON["down_devices"] == [] and JSON["events"] == [] and JSON["plugins"] == []:
+        if JSON["new_devices"] == [] and JSON["down_devices"] == [] and JSON["events"] == [] and JSON["plugins"] == []:
            self.HasNotifications = False
        else:            
            self.HasNotifications = True 
@@ -48,12 +62,108 @@ class Notification_obj:
        self.DateTimePushed     = ""
        self.Status             = "new"
        self.JSON               = JSON
-        self.Text               = Text
-        self.HTML               = HTML
+        self.Text               = ""
+        self.HTML               = ""
        self.PublishedVia       = ""
        self.Extra              = Extra

        if self.HasNotifications:
+
+
+            # if not notiStruc.json['data'] and not notiStruc.text and not notiStruc.html:
+            #     mylog('debug', '[Notification] notiStruc is empty')
+            # else:
+            #     mylog('debug', ['[Notification] notiStruc:', json.dumps(notiStruc.__dict__, indent=4)])
+            
+            Text = ""
+            HTML = ""                        
+
+
+            # Open text Template
+            mylog('verbose', ['[Notification] Open text Template'])
+            template_file = open(pialertPath + '/back/report_template.txt', 'r')
+            mail_text = template_file.read()
+            template_file.close()
+
+            # Open html Template
+            mylog('verbose', ['[Notification] Open html Template'])
+
+            # select template type depoending if running latest version or an older one
+            if conf.newVersionAvailable :
+                template_file_path = '/back/report_template_new_version.html'
+            else:
+                template_file_path = '/back/report_template.html'
+
+            mylog('verbose', ['[Notification] Using template', template_file_path])
+            template_file = open(pialertPath + template_file_path, 'r')   
+            mail_html = template_file.read()
+            template_file.close()
+
+
+            # Report "REPORT_DATE" in Header & footer
+            timeFormated = timeNowTZ().strftime ('%Y-%m-%d %H:%M')
+            mail_text = mail_text.replace ('<REPORT_DATE>', timeFormated)
+            mail_html = mail_html.replace ('<REPORT_DATE>', timeFormated)
+
+            # Report "SERVER_NAME" in Header & footer
+            mail_text = mail_text.replace ('<SERVER_NAME>', socket.gethostname() )
+            mail_html = mail_html.replace ('<SERVER_NAME>', socket.gethostname() )
+
+            # Report "VERSION" in Header & footer
+            VERSIONFILE = subprocess.check_output(['php', pialertPath + '/front/php/templates/version.php']).decode('utf-8')
+            mail_text = mail_text.replace ('<VERSION_PIALERT>', VERSIONFILE)
+            mail_html = mail_html.replace ('<VERSION_PIALERT>', VERSIONFILE)	
+
+            # Report "BUILD" in Header & footer
+            BUILDFILE = subprocess.check_output(['php', pialertPath + '/front/php/templates/build.php']).decode('utf-8')
+            mail_text = mail_text.replace ('<BUILD_PIALERT>', BUILDFILE)
+            mail_html = mail_html.replace ('<BUILD_PIALERT>', BUILDFILE)
+
+            # Start generating the TEXT & HTML notification messages
+            html, text = construct_notifications(self.JSON, "new_devices")
+
+            mail_text = mail_text.replace ('<NEW_DEVICES_TABLE>', text + '\n')
+            mail_html = mail_html.replace ('<NEW_DEVICES_TABLE>', html)
+            mylog('verbose', ['[Notification] New Devices sections done.'])
+
+            html, text = construct_notifications(self.JSON, "down_devices")
+
+
+            mail_text = mail_text.replace ('<DOWN_DEVICES_TABLE>', text + '\n')
+            mail_html = mail_html.replace ('<DOWN_DEVICES_TABLE>', html)
+            mylog('verbose', ['[Notification] Down Devices sections done.'])
+
+            html, text = construct_notifications(self.JSON, "events")           
+            
+
+            mail_text = mail_text.replace ('<EVENTS_TABLE>', text + '\n')
+            mail_html = mail_html.replace ('<EVENTS_TABLE>', html)
+            mylog('verbose', ['[Notification] Events sections done.'])    
+
+
+            html, text = construct_notifications(self.JSON, "plugins")
+
+            mail_text = mail_text.replace ('<PLUGINS_TABLE>', text + '\n')
+            mail_html = mail_html.replace ('<PLUGINS_TABLE>', html)
+            
+            mylog('verbose', ['[Notification] Plugins sections done.'])
+
+            final_text = removeDuplicateNewLines(mail_text)
+
+            # Create clickable MAC links            
+            final_html = generate_mac_links (mail_html, conf.REPORT_DASHBOARD_URL + '/deviceDetails.php?mac=')    
+
+            send_api(self.JSON, mail_text, mail_html)
+
+            #  Write output data for debug            
+            write_file (logPath + '/report_output.txt', final_text)
+            write_file (logPath + '/report_output.html', final_html)
+
+            mylog('minimal', ['[Notification] Udating API files'])
+
+            self.Text               = final_text
+            self.HTML               = final_html
+
            self.upsert()
        
        return self
@@ -107,8 +217,108 @@ class Notification_obj:

        self.save()

-        
+    def clearPendingEmailFlag(self):
+
+        # Clean Pending Alert Events
+        self.db.sql.execute ("""UPDATE Devices SET dev_LastNotification = ?
+                            WHERE dev_MAC IN (
+                                SELECT eve_MAC FROM Events
+                                    WHERE eve_PendingAlertEmail = 1
+                            )
+                        """, (timeNowTZ(),) )
+
+        self.db.sql.execute ("""UPDATE Events SET eve_PendingAlertEmail = 0
+                                    WHERE eve_PendingAlertEmail = 1 
+                                    AND eve_EventType !='Device Down' """)
+
+        # Clear down events flag after the reporting window passed
+        self.db.sql.execute (f"""UPDATE Events SET eve_PendingAlertEmail = 0
+                                    WHERE eve_PendingAlertEmail = 1 
+                                    AND eve_EventType =='Device Down' 
+                                    AND eve_DateTime < datetime('now', '-{get_setting_value('NTFPRCS_alert_down_time')} minutes', '{get_timezone_offset()}')
+                            """)
+
+        # clear plugin events
+        self.db.sql.execute ("DELETE FROM Plugins_Events")
+
+        # DEBUG - print number of rows updated
+        mylog('minimal', ['[Notification] Notifications changes: ', self.db.sql.rowcount])    
+
+        self.save()

    def save(self):
        # Commit changes
-        self.db.commitDB()
+        self.db.commitDB()
+
+#-------------------------------------------------------------------------------
+# Reporting
+#-------------------------------------------------------------------------------
+
+#-------------------------------------------------------------------------------
+def construct_notifications(JSON, section):
+
+    jsn         = JSON[section]
+
+    # Return if empty
+    if jsn == []:
+        return '',''
+
+    tableTitle  = JSON[section + "_meta"]["title"]
+    headers     = JSON[section + "_meta"]["columnNames"]
+
+    html = ''
+    text = ''
+
+    table_attributes = {"style" : "border-collapse: collapse; font-size: 12px; color:#70707", "width" : "100%", "cellspacing" : 0, "cellpadding" : "3px", "bordercolor" : "#C0C0C0", "border":"1"}
+    headerProps = "width='120px' style='color:white; font-size: 16px;' bgcolor='#64a0d6' "
+    thProps = "width='120px' style='color:#F0F0F0' bgcolor='#64a0d6' "
+
+    build_direction = "TOP_TO_BOTTOM"
+    text_line = '{}\t{}\n'
+
+
+    if len(jsn) > 0:
+        text = tableTitle + "\n---------\n"
+
+        # Convert a JSON into an HTML table
+        html = convert({"data": jsn}, build_direction=build_direction, table_attributes=table_attributes)
+        
+        # Cleanup the generated HTML table notification
+        html = format_table(html, "data", headerProps, tableTitle).replace('<ul>','<ul style="list-style:none;padding-left:0">').replace("<td>null</td>", "<td></td>")        
+
+        # prepare text-only message
+        for device in jsn:
+            for header in headers:
+                padding = ""
+                if len(header) < 4:
+                    padding = "\t"
+                text += text_line.format ( header + ': ' + padding, device[header])
+            text += '\n'
+
+        #  Format HTML table headers
+        for header in headers:
+            html = format_table(html, header, thProps)
+
+    return html, text
+
+#-------------------------------------------------------------------------------
+def send_api(json_final, mail_text, mail_html):
+        mylog('verbose', ['[Send API] Updating notification_* files in ', apiPath])
+
+        write_file(apiPath + 'notification_text.txt'  , mail_text)
+        write_file(apiPath + 'notification_text.html'  , mail_html)
+        write_file(apiPath + 'notification_json_final.json'  , json.dumps(json_final))
+
+
+
+#-------------------------------------------------------------------------------
+# Replacing table headers
+def format_table (html, thValue, props, newThValue = ''):
+
+    if newThValue == '':
+        newThValue = thValue
+
+    return html.replace("<th>"+thValue+"</th>", "<th "+props+" >"+newThValue+"</th>" )
+
+
+
--- a/pialert/plugin.py
+++ b/pialert/plugin.py
@@ -327,6 +327,8 @@ def execute_plugin(db, plugin, pluginsState = plugins_state() ):
        try:
            sql.execute ("ATTACH DATABASE '"+ fullSqlitePath +"' AS EXTERNAL_"+plugin["unique_prefix"])
            arr = db.get_sql_array (q) 
+            sql.execute ("DETACH DATABASE EXTERNAL_"+plugin["unique_prefix"])
+             
        except sqlite3.Error as e:            
            mylog('none',[f'[Plugins] ⚠ ERROR: DB_PATH setting ({fullSqlitePath}) for plugin {plugin["unique_prefix"]}. Did you mount it correctly?'])
            mylog('none',[f'[Plugins] ⚠ ERROR: ATTACH DATABASE failed with SQL ERROR: ', e])
@@ -787,7 +789,7 @@ def handle_test(runType, db, pluginsState):
    
    # Create fake notification
    notification    = Notification_obj(db)
-    notificationObj = notification.create(sample_json, sample_txt, sample_html, "")
+    notificationObj = notification.create(sample_json, "")

    # Run test
    pluginsState = handle_run(runType, db, pluginsState)
--- a/pialert/reporting.py
+++ b/pialert/reporting.py
@@ -12,103 +12,37 @@

 import datetime
 import json
-import socket
-import subprocess
-import requests
-from json2table import convert

 # pialert modules
 import conf
 import const
 from const import pialertPath, logPath, apiPath
-from helper import noti_obj, generate_mac_links, removeDuplicateNewLines, timeNowTZ, hide_email,  updateState, get_file_content, write_file
+from helper import timeNowTZ, get_file_content, write_file, get_timezone_offset, get_setting_value
 from logger import logResult, mylog, print_log


 #===============================================================================
 # REPORTING
 #===============================================================================
-# create a json of the notifications to provide further integration options
-json_final = []


 #-------------------------------------------------------------------------------
-def construct_notifications(db, sqlQuery, tableTitle, skipText = False, suppliedJsonStruct = None):
-
-    if suppliedJsonStruct is None and sqlQuery == "":
-        return noti_obj("", "", "")
-
-    table_attributes = {"style" : "border-collapse: collapse; font-size: 12px; color:#70707", "width" : "100%", "cellspacing" : 0, "cellpadding" : "3px", "bordercolor" : "#C0C0C0", "border":"1"}
-    headerProps = "width='120px' style='color:white; font-size: 16px;' bgcolor='#64a0d6' "
-    thProps = "width='120px' style='color:#F0F0F0' bgcolor='#64a0d6' "
-
-    build_direction = "TOP_TO_BOTTOM"
-    text_line = '{}\t{}\n'
-
-    if suppliedJsonStruct is None:
-        json_obj = db.get_table_as_json(sqlQuery)
-    else:
-        json_obj = suppliedJsonStruct
-
-    jsn  = json_obj.json
-    html = ""
-    text = ""
-
-    if len(jsn["data"]) > 0:
-        text = tableTitle + "\n---------\n"
-
-        # Convert a JSON into an HTML table
-        html = convert(jsn, build_direction=build_direction, table_attributes=table_attributes)
-        
-        # Cleanup the generated HTML table notification
-        html = format_table(html, "data", headerProps, tableTitle).replace('<ul>','<ul style="list-style:none;padding-left:0">').replace("<td>null</td>", "<td></td>")
-
-        headers = json_obj.columnNames
-
-        # prepare text-only message
-        if skipText == False:
-
-            for device in jsn["data"]:
-                for header in headers:
-                    padding = ""
-                    if len(header) < 4:
-                        padding = "\t"
-                    text += text_line.format ( header + ': ' + padding, device[header])
-                text += '\n'
-
-        #  Format HTML table headers
-        for header in headers:
-            html = format_table(html, header, thProps)
-
-    notiStruc = noti_obj(jsn, text, html)
-
-    
-    if not notiStruc.json['data'] and not notiStruc.text and not notiStruc.html:
-        mylog('debug', '[Notification] notiStruc is empty')
-    else:
-        mylog('debug', ['[Notification] notiStruc:', json.dumps(notiStruc.__dict__, indent=4)])
-
-    return notiStruc
-
-
 def get_notifications (db):

    sql = db.sql  #TO-DO
-    global mail_text, mail_html, json_final, partial_html, partial_txt, partial_json
-
-    deviceUrl              = conf.REPORT_DASHBOARD_URL + '/deviceDetails.php?mac='
-    plugins_report         = False
-
+    
    # Reporting section
    mylog('verbose', ['[Notification] Check if something to report'])

-    # prepare variables for JSON construction
-    json_internet = []
+    # prepare variables for JSON construction    
    json_new_devices = []
+    json_new_devices_meta = {}
    json_down_devices = []
-    json_events = []
-    json_ports = []
+    json_down_devices_meta = {}
+    json_events = []    
+    json_events_meta = {}
    json_plugins = []
+    json_plugins_meta = {}

    # Disable reporting on events for devices where reporting is disabled based on the MAC address
    sql.execute ("""UPDATE Events SET eve_PendingAlertEmail = 0
@@ -121,192 +55,106 @@ def get_notifications (db):
                        (
                            SELECT dev_MAC FROM Devices WHERE dev_AlertDeviceDown = 0
 						)""")
+    
+    sections = get_setting_value('NTFPRCS_INCLUDED_SECTIONS')

-    # Open text Template
-    mylog('verbose', ['[Notification] Open text Template'])
-    template_file = open(pialertPath + '/back/report_template.txt', 'r')
-    mail_text = template_file.read()
-    template_file.close()
+    mylog('verbose', ['[Notification] Included sections: ', sections ])

-    # Open html Template
-    mylog('verbose', ['[Notification] Open html Template'])
-
-
-    # select template type depoending if running latest version or an older one
-    if conf.newVersionAvailable :
-        template_file_path = '/back/report_template_new_version.html'
-    else:
-        template_file_path = '/back/report_template.html'
-
-    mylog('verbose', ['[Notification] Using template', template_file_path])
-    template_file = open(pialertPath + template_file_path, 'r')
-
-    mail_html = template_file.read()
-    template_file.close()
-
-    # Report "REPORT_DATE" in Header & footer
-    timeFormated = timeNowTZ().strftime ('%Y-%m-%d %H:%M')
-    mail_text = mail_text.replace ('<REPORT_DATE>', timeFormated)
-    mail_html = mail_html.replace ('<REPORT_DATE>', timeFormated)
-
-    # Report "SERVER_NAME" in Header & footer
-    mail_text = mail_text.replace ('<SERVER_NAME>', socket.gethostname() )
-    mail_html = mail_html.replace ('<SERVER_NAME>', socket.gethostname() )
-
-    # Report "VERSION" in Header & footer
-    VERSIONFILE = subprocess.check_output(['php', pialertPath + '/front/php/templates/version.php']).decode('utf-8')
-    mail_text = mail_text.replace ('<VERSION_PIALERT>', VERSIONFILE)
-    mail_html = mail_html.replace ('<VERSION_PIALERT>', VERSIONFILE)	
-
-    # Report "BUILD" in Header & footer
-    BUILDFILE = subprocess.check_output(['php', pialertPath + '/front/php/templates/build.php']).decode('utf-8')
-    mail_text = mail_text.replace ('<BUILD_PIALERT>', BUILDFILE)
-    mail_html = mail_html.replace ('<BUILD_PIALERT>', BUILDFILE)
-	
-    mylog('verbose', ['[Notification] included sections: ', conf.INCLUDED_SECTIONS ])
-
-    if 'new_devices' in conf.INCLUDED_SECTIONS :
+    if 'new_devices' in sections:
        # Compose New Devices Section
-        sqlQuery = """SELECT eve_MAC as MAC, eve_DateTime as Datetime, dev_LastIP as IP, eve_EventType as "Event Type", dev_Name as "Device name", dev_Comments as Comments  FROM Events_Devices
+        sqlQuery = f"""SELECT eve_MAC as MAC, eve_DateTime as Datetime, dev_LastIP as IP, eve_EventType as "Event Type", dev_Name as "Device name", dev_Comments as Comments  FROM Events_Devices
                        WHERE eve_PendingAlertEmail = 1
-                        AND eve_EventType = 'New Device'
-                        ORDER BY eve_DateTime"""
+                        AND eve_EventType = 'New Device' 
+                        {get_setting_value('NTFPRCS_new_dev_condition').replace('{s-quote}',"'")} 
+                        ORDER BY eve_DateTime"""   

-        notiStruc = construct_notifications(db, sqlQuery, "New devices")
+        mylog('debug', ['[Notification] new_devices SQL query: ', sqlQuery ])

-        # collect "new_devices" for the json
-        json_new_devices = notiStruc.json["data"]
+        # Get the events as JSON
+        json_obj = db.get_table_as_json(sqlQuery)

-        mail_text = mail_text.replace ('<NEW_DEVICES_TABLE>', notiStruc.text + '\n')
-        mail_html = mail_html.replace ('<NEW_DEVICES_TABLE>', notiStruc.html)
-        mylog('verbose', ['[Notification] New Devices sections done.'])
+        json_new_devices_meta = {
+            "title": "New devices",
+            "columnNames": json_obj.columnNames
+        }

-    if 'down_devices' in conf.INCLUDED_SECTIONS :
-        # Compose Devices Down Section
-        sqlQuery = """SELECT eve_MAC as MAC, eve_DateTime as Datetime, dev_LastIP as IP, eve_EventType as "Event Type", dev_Name as "Device name", dev_Comments as Comments  FROM Events_Devices
-                        WHERE eve_PendingAlertEmail = 1
-                        AND eve_EventType = 'Device Down'
-                        ORDER BY eve_DateTime"""
+        json_new_devices = json_obj.json["data"]    

-        notiStruc = construct_notifications(db, sqlQuery, "Down devices")
+    if 'down_devices' in sections:
+        # Compose Devices Down Section 
+        # - select only Down Alerts with pending email of devices that didn't reconnect within the specified time window
+        sqlQuery = f"""
+                    SELECT dev_Name, eve_MAC, dev_Vendor, eve_IP, eve_DateTime, eve_EventType  
+                        FROM Events_Devices AS down_events
+                        WHERE eve_PendingAlertEmail = 1 
+                        AND down_events.eve_EventType = 'Device Down' 
+                        AND eve_DateTime < datetime('now', '-{get_setting_value('NTFPRCS_alert_down_time')} minutes', '{get_timezone_offset()}')
+                        AND NOT EXISTS (
+                            SELECT 1
+                            FROM Events AS connected_events
+                            WHERE connected_events.eve_MAC = down_events.eve_MAC
+                                AND connected_events.eve_EventType = 'Connected'
+                                AND connected_events.eve_DateTime > down_events.eve_DateTime        
+                        )
+                        ORDER BY down_events.eve_DateTime;
+                    """
+        
+        # Get the events as JSON        
+        json_obj = db.get_table_as_json(sqlQuery)

-        # collect "down_devices" for the json
-        json_down_devices = notiStruc.json["data"]
+        json_down_devices_meta = {
+            "title": "Down devices",
+            "columnNames": json_obj.columnNames
+        }
+        json_down_devices = json_obj.json["data"]     

-        mail_text = mail_text.replace ('<DOWN_DEVICES_TABLE>', notiStruc.text + '\n')
-        mail_html = mail_html.replace ('<DOWN_DEVICES_TABLE>', notiStruc.html)
-        mylog('verbose', ['[Notification] Down Devices sections done.'])
-
-    if 'events' in conf.INCLUDED_SECTIONS :
+    if 'events' in sections:
        # Compose Events Section
-        sqlQuery = """SELECT eve_MAC as MAC, eve_DateTime as Datetime, dev_LastIP as IP, eve_EventType as "Event Type", dev_Name as "Device name", dev_Comments as Comments  FROM Events_Devices
+        sqlQuery = f"""SELECT eve_MAC as MAC, eve_DateTime as Datetime, dev_LastIP as IP, eve_EventType as "Event Type", dev_Name as "Device name", dev_Comments as Comments  FROM Events_Devices
                        WHERE eve_PendingAlertEmail = 1
                        AND eve_EventType IN ('Connected','Disconnected',
-                            'IP Changed')
-                        ORDER BY eve_DateTime"""
+                            'IP Changed') 
+                            {get_setting_value('NTFPRCS_event_condition').replace('{s-quote}',"'")} 
+                        ORDER BY eve_DateTime"""      

-        notiStruc = construct_notifications(db, sqlQuery, "Events")
+        mylog('debug', ['[Notification] events SQL query: ', sqlQuery ])
+        
+        # Get the events as JSON        
+        json_obj = db.get_table_as_json(sqlQuery)

-        # collect "events" for the json
-        json_events = notiStruc.json["data"]
+        json_events_meta = {
+            "title": "Events",
+            "columnNames": json_obj.columnNames
+        }
+        json_events = json_obj.json["data"]     

-        mail_text = mail_text.replace ('<EVENTS_TABLE>', notiStruc.text + '\n')
-        mail_html = mail_html.replace ('<EVENTS_TABLE>', notiStruc.html)
-        mylog('verbose', ['[Notification] Events sections done.'])    
-
-    if 'plugins' in conf.INCLUDED_SECTIONS:
+    if 'plugins' in sections:
        # Compose Plugins Section
-        sqlQuery = """SELECT Plugin, Object_PrimaryId, Object_SecondaryId, DateTimeChanged, Watched_Value1, Watched_Value2, Watched_Value3, Watched_Value4, Status from Plugins_Events"""
+        sqlQuery = """SELECT Plugin, Object_PrimaryId, Object_SecondaryId, DateTimeChanged, Watched_Value1, Watched_Value2, Watched_Value3, Watched_Value4, Status from Plugins_Events"""        
+        
+        # Get the events as JSON        
+        json_obj = db.get_table_as_json(sqlQuery)

-        notiStruc = construct_notifications(db, sqlQuery, "Plugins")
+        json_plugins_meta = {
+            "title": "Plugins",
+            "columnNames": json_obj.columnNames
+        }
+        json_plugins = json_obj.json["data"]   

-        # collect "plugins" for the json
-        json_plugins = notiStruc.json["data"]

-        mail_text = mail_text.replace ('<PLUGINS_TABLE>', notiStruc.text + '\n')
-        mail_html = mail_html.replace ('<PLUGINS_TABLE>', notiStruc.html)
-
-        # check if we need to report something
-        plugins_report = len(json_plugins) > 0
-        mylog('verbose', ['[Notification] Plugins sections done.'])
-
-    final_json = {
-                    "internet": json_internet,
+    final_json = {                    
                    "new_devices": json_new_devices,
+                    "new_devices_meta": json_new_devices_meta,
                    "down_devices": json_down_devices,
+                    "down_devices_meta": json_down_devices_meta,
                    "events": json_events,                    
+                    "events_meta": json_events_meta,                    
                    "plugins": json_plugins,
-                    }
+                    "plugins_meta": json_plugins_meta,
+                }

-    final_text = removeDuplicateNewLines(mail_text)
+    return final_json

-    # Create clickable MAC links
-    final_html = generate_mac_links (mail_html, deviceUrl)    
-
-    #  Write output emails for debug
-    write_file (logPath + '/report_output.json', json.dumps(final_json))
-    write_file (logPath + '/report_output.txt', final_text)
-    write_file (logPath + '/report_output.html', final_html)
-
-    mylog('minimal', ['[Notification] Udating API files'])
-    send_api()
-
-    return noti_obj(final_json, final_text, final_html)   
-
-
-#-------------------------------------------------------------------------------
-# Replacing table headers
-def format_table (html, thValue, props, newThValue = ''):
-
-    if newThValue == '':
-        newThValue = thValue
-
-    return html.replace("<th>"+thValue+"</th>", "<th "+props+" >"+newThValue+"</th>" )
-
-#-------------------------------------------------------------------------------
-def format_report_section (pActive, pSection, pTable, pText, pHTML):
-
-
-    # Replace section text
-    if pActive :
-        conf.mail_text = conf.mail_text.replace ('<'+ pTable +'>', pText)
-        conf.mail_html = conf.mail_html.replace ('<'+ pTable +'>', pHTML)
-
-        conf.mail_text = remove_tag (conf.mail_text, pSection)
-        conf.mail_html = remove_tag (conf.mail_html, pSection)
-    else:
-        conf.mail_text = remove_section (conf.mail_text, pSection)
-        conf.mail_html = remove_section (conf.mail_html, pSection)
-
-#-------------------------------------------------------------------------------
-def remove_section (pText, pSection):
-    # Search section into the text
-    if pText.find ('<'+ pSection +'>') >=0 \
-    and pText.find ('</'+ pSection +'>') >=0 :
-        # return text without the section
-        return pText[:pText.find ('<'+ pSection+'>')] + \
-               pText[pText.find ('</'+ pSection +'>') + len (pSection) +3:]
-    else :
-        # return all text
-        return pText
-
-#-------------------------------------------------------------------------------
-def remove_tag (pText, pTag):
-    # return text without the tag
-    return pText.replace ('<'+ pTag +'>','').replace ('</'+ pTag +'>','')
-
-
-#-------------------------------------------------------------------------------
-# Reporting
-#-------------------------------------------------------------------------------
-
-#-------------------------------------------------------------------------------
-def send_api():
-        mylog('verbose', ['[Send API] Updating notification_* files in ', apiPath])
-
-        write_file(apiPath + 'notification_text.txt'  , mail_text)
-        write_file(apiPath + 'notification_text.html'  , mail_html)
-        write_file(apiPath + 'notification_json_final.json'  , json.dumps(json_final))


 #-------------------------------------------------------------------------------
@@ -314,7 +162,8 @@ def skip_repeated_notifications (db):

    # Skip repeated notifications
    # due strfime : Overflow --> use  "strftime / 60"
-    mylog('verbose','[Skip Repeated Notifications] Skip Repeated start')
+    mylog('verbose','[Skip Repeated Notifications] Skip Repeated')
+    
    db.sql.execute ("""UPDATE Events SET eve_PendingAlertEmail = 0
                    WHERE eve_PendingAlertEmail = 1 AND eve_MAC IN
                        (
@@ -326,7 +175,7 @@ def skip_repeated_notifications (db):
                              (strftime('%s','now','localtime')/60 )
                        )
                 """ )
-    mylog('verbose','[Skip Repeated Notifications] Skip Repeated end')
+   

    db.commitDB()
Author	SHA1	Message	Date
Jokob-sk	508b0c2d83	dig utility name discovery work #534	2024-01-18 17:10:07 +11:00
Jokob-sk	b354e72489	update issue template	2024-01-16 07:25:17 +11:00
Jokob-sk	90bfa70d1b	update_vendors work #533 🔃	2024-01-13 08:56:05 +11:00
Jokob-sk	7561a8478d	update_vendors work #533 🔃	2024-01-13 08:42:31 +11:00
Jokob-sk	b5afdb2bce	docs📚	2024-01-08 08:08:51 +11:00
Jokob-sk	5207162d0a	clickable settings cards ⚙	2024-01-07 11:40:09 +11:00
Jokob-sk	8eecc54217	docs 📚	2024-01-07 11:18:52 +11:00
Jokob-sk	ddfd0d3cb3	docs 📚, links to plugin docs	2024-01-07 10:26:08 +11:00
Jokob-sk	bcc5b2f28a	Issue templates	2024-01-06 18:59:09 +11:00
Jokob-sk	2e6be21cd9	Issue templates	2024-01-06 13:12:00 +11:00
Jokob-sk	abb28c4e5b	Issue templates	2024-01-06 12:48:30 +11:00
Jokob-sk	44f0ba0924	MQTT settings to select what is send #364📩	2024-01-06 12:09:59 +11:00
Jokob-sk	a6f5e6c499	MQTT enable scheduler #522	2024-01-06 10:26:39 +11:00
Jokob-sk	d992edf6b4	MQTT more logging #522	2024-01-05 18:42:32 +11:00
Jokob-sk	d7ba540377	renamed DIG_GET_IP_ARG to INTRNT_DIG_GET_IP_ARG v0.2	2024-01-05 08:33:30 +11:00
Jokob-sk	30c95f0d5e	renamed DIG_GET_IP_ARG to INTRNT_DIG_GET_IP_ARG v0.1	2024-01-05 08:29:27 +11:00
Jokob-sk	b670e3a8b1	renamed DIG_GET_IP_ARG to INTRNT_DIG_GET_IP_ARG	2024-01-05 08:18:52 +11:00
Jokob-sk	467e24d167	ARMv6 test #527 - added to prod image	2024-01-05 08:00:54 +11:00
Jokob-sk	7b70d61dd8	ARMv6 test #527	2024-01-05 07:42:43 +11:00
Jokob-sk	d530576e9b	NTFPRCS work + docs	2024-01-04 15:28:43 +11:00
Jokob-sk	cff6f6393d	PIHOLE_CMD_OLD bug	2024-01-04 14:37:22 +11:00
Jokob-sk	f33d753cc1	work #504 - New Dev + Events filter setting⚙	2024-01-04 14:33:26 +11:00
Jokob-sk	6809688623	docs 📚	2024-01-04 13:26:28 +11:00
Jokob-sk	68de633143	work on #523 - ip v6 ordering support 0.1	2024-01-04 09:03:35 +11:00
Jokob-sk	a3e21ac17d	work on #523 - ip v6 ordering support	2024-01-03 07:31:32 +11:00
Jokob-sk	c8267f75fa	Re-initialize PIHOLE_CMD + docs	2024-01-02 08:06:00 +11:00
Jokob-sk	9e6a52ca4b	work #520 - install logging	2024-01-01 10:15:25 +11:00
Jokob-sk	13c68efb8a	PiHole work #513	2023-12-21 21:17:05 +11:00
Jokob-sk	47a3f7073b	_meta unbound #519	2023-12-18 07:55:41 +11:00
Jokob-sk	8e0eb6a480	Notifications refactor and #242 📩	2023-12-17 22:43:50 +11:00
Jokob-sk	911c897b00	Notifications refactor 🏗	2023-12-17 10:39:14 +11:00