Retry support for INTRNT #667

README.md

@@ -60,12 +60,12 @@ Get visibility of what's going on on your WIFI/LAN network. Schedule scans for d
 |-------------|-------------|
 | 📥🐳  | [Docker instructions](https://github.com/jokob-sk/NetAlertX/blob/main/dockerfiles/README.md) 
 | 📥💻  | [HW install (experimental 🧪)](https://github.com/jokob-sk/NetAlertX/blob/main/docs/HW_INSTALL.md) |
+| 📥🟧  | [Unraid App 🧪)](https://unraid.net/community/apps) |
 | 📚     | [All Documentation](https://github.com/jokob-sk/NetAlertX/blob/main/docs/README.md) (App Usage and Configuration) |
  
 > Other Alternatives
 >
 > - Check out [leiweibau's on HW installed fork](https://github.com/leiweibau/Pi.Alert/) (maintained)
-> - Check instructions for [pucherot's original code](https://github.com/pucherot/Pi.Alert/) (unmaintained)
 > - [WatchYourLAN](https://github.com/aceberg/WatchYourLAN) - Lightweight network IP scanner with web GUI (Open source)
 > - [Fing](https://www.fing.com/) - Network scanner app for your Internet security (Commercial, Phone App, Proprietary hardware)
 > - [NetBox](https://netboxlabs.com/) - Network management software (Commercial)

front/js/common.js

@@ -1072,7 +1072,9 @@ function clearCache() {
   showSpinner();
   sessionStorage.clear();
   localStorage.clear();
+  setTimeout(() => {
   window.location.reload();
+}, 500);
 }
 
 // -----------------------------------------------------------------------------

front/plugins/README.md

@@ -1,51 +1,74 @@
-# 📚 Docs for individual plugins 
+# 🔌 Plugins
 
 >[!NOTE]
 > Please check this [Plugins debugging guide](https://github.com/jokob-sk/NetAlertX/blob/main/docs/DEBUG_PLUGINS.md) and the corresponding Plugin documentation in the below table if you are facing issues.  
 
-## 🔌 Plugins & 📚 Docs 
 
-| Required | CurrentScan | Unique Prefix | Data source        |   Type         | Link + Docs                                                          | 
+## Plugin types
-|----------|-------------|---------------|--------------------|----------------|---------------------------------------------------------------------|
+
-|          |             | APPRISE       | Script             | 💬 publisher   | 📚[_publisher_apprise](/front/plugins/_publisher_apprise/)          |
+If you want to discover or import devices into the application enable some of the `🔍 dev scanner` plugins. The next step is to pick a notification plugin, or `💬 publisher` plugin, to get notified about network changes. If you don't see a publisher you'd like to use, look at the  [📚_publisher_apprise](/front/plugins/_publisher_apprise/) plugin which is a proxy for over 80 notification services. 
-|          |    Yes      | ARPSCAN       | Script             | 🔍dev scanner  | 📚[arp_scan](/front/plugins/arp_scan/)                              |
+
-|          |             | CSVBCKP       | Script             | ⚙ system       | 📚[csv_backup](/front/plugins/csv_backup/)                          |
+### Enabling plugins
-|  Yes*    |             | DBCLNP        | Script             | ⚙ system       | 📚[db_cleanup](/front/plugins/db_cleanup/)                          |
+
-|          |             | DDNS          | Script             | ⚙ system       | 📚[ddns_update](/front/plugins/ddns_update/)                        |
+Plugins can be enabled via Settings, and can be disabled as needed. 
-|          |    Yes      | DHCPLSS       | Script             | 🔍dev scanner  | 📚[dhcp_leases](/front/plugins/dhcp_leases/)                        |
+
-|          |             | DHCPSRVS      | Script             | ♻ other        | 📚[dhcp_servers](/front/plugins/dhcp_servers/)                      |
+1. Research which plugin you'd like to use below and then load the required plugins in Settings via the `LOADED_PLUGINS` setting.
-|          |    Yes      | INTRNT        | Script             | 🔍dev scanner  | 📚[internet_ip](/front/plugins/internet_ip/)                        |
+1. Save the changes and review the Settings of the newly loaded plugins. 
-|          |             | INTRSPD       | Script             | ♻ other        | 📚[internet_speedtest](/front/plugins/internet_speedtest/)          |
+1. Change the `<prefix>_RUN` Setting to the recommended or custom value as per the documentation of the given setting  
-|          |             | MAINT         | Script             | ⚙ system       | 📚[maintenance](/front/plugins/maintenance/)                        |
+    - If using `schedule` on a `🔍 dev scanner` plugin, make sure the schedules are the same across all `🔍 dev scanner` plugins
-|          |             | MQTT          | Script             | 💬 publisher   | 📚[_publisher_mqtt](/front/plugins/_publisher_mqtt/)                |
+
-|  Yes     |             | NEWDEV        | Template           | ⚙ system       | 📚[newdev_template](/front/plugins/newdev_template/)                |
+### Disabling, Unloading and Ignoring plugins
-|          |             | NMAP          | Script             | ♻ other        | 📚[nmap_scan](/front/plugins/nmap_scan/)                            |
+
-|          |    Yes      | NMAPDEV       | Script             | 🔍dev scanner  | 📚[nmap_dev_scan](/front/plugins/nmap_dev_scan/)                    |
+1. Change the `<prefix>_RUN` Setting to `disabled` if you want to disable the plugin, but keep the settings
-|          |             | NSLOOKUP      | Script             | ♻ other        | 📚[nslookup_scan](/front/plugins/nslookup_scan/)                    |
+1. If you want to speed up the application, you can unload the plugin by unselecting it in the `LOADED_PLUGINS` setting.
-|  Yes     |             | NTFPRCS       | Template           | ⚙ system       | 📚[notification_processing](/front/plugins/notification_processing/)|
+    - Careful, once you save the Settings Unloaded plugin settings will be lost (old `app.conf` files are kept in the `/config` folder) 
-|          |             | NTFY          | Script             | 💬 publisher   | 📚[_publisher_ntfy](/front/plugins/_publisher_ntfy/)                |
+1. You can completely ignore plugins by placing a `ignore_plugin` file into the plugin directory. Ignored plugins won't show up in the `LOADED_PLUGINS` setting.
-|          |             | PHOLUS        | Script             | ♻ other        | 📚[pholus_scan](/front/plugins/pholus_scan/)                        |
+
-|          |    Yes      | PIHOLE        | External SQLite DB | 🔍dev scanner  | 📚[pihole_scan](/front/plugins/pihole_scan/)                        |
+
-|          |             | PUSHSAFER     | Script             | 💬 publisher   | 📚[_publisher_pushsafer](/front/plugins/_publisher_pushsafer/)      |
+## Available Plugins
-|          |             | PUSHOVER      | Script             | 💬 publisher   | 📚[_pushover_pushsafer](/front/plugins/_publisher_pushover/)        |
+
-|  Yes     |             | SETPWD        | Template           | ⚙ system       | 📚[set_password](/front/plugins/set_password/)                      |
+Device-detecting plugins insert values into the `CurrentScan` database table.  The plugins that are not required are safe to ignore, however, it makes sense to have a least some device-detecting plugins enabled, such as `ARPSCAN` or `NMAPDEV`. 
-|          |             | SMTP          | Script             | 💬 publisher   | 📚[_publisher_email](/front/plugins/_publisher_email/)              |
+
-|          |    Yes      | SNMPDSC       | Script             | 🔍dev scanner  | 📚[snmp_discovery](/front/plugins/snmp_discovery/)                  |
+| ID            | Type           | Description                  | Required | Data source        | Detailed docs                                                       |
-|          |    Yes**    | UNDIS         | Script             | ♻ other        | 📚[undiscoverables](/front/plugins/undiscoverables/)                |
+|---------------|----------------|------------------------------|----------|--------------------|---------------------------------------------------------------------|
-|          |    Yes      | UNFIMP        | Script             | 🔍dev scanner  | 📚[unifi_import](/front/plugins/unifi_import/)                      |
+| `APPRISE`       | 💬 publisher   | Apprise publisher plugin     |          | Script             | [📚_publisher_apprise](/front/plugins/_publisher_apprise/)          |
-|          |             | VNDRPDT       | Script             | ⚙ system       | 📚[vendor_update](/front/plugins/vendor_update/)                    |
+| `ARPSCAN`       | 🔍 dev scanner | ARP scan plugin              |          | Script             | [📚arp_scan](/front/plugins/arp_scan/)                              |
-|          |             | WEBHOOK       | Script             | 💬 publisher   | 📚[_publisher_webhook](/front/plugins/_publisher_webhook/)          |
+| `CSVBCKP`       | ⚙ system       | CSV backup plugin            |          | Script             | [📚csv_backup](/front/plugins/csv_backup/)                          |
-|          |             | WEBMON        | Script             | ♻ other        | 📚[website_monitor](/front/plugins/website_monitor/)                |
+| `DBCLNP`        | ⚙ system       | Database cleanup plugin      |  Yes*    | Script             | [📚db_cleanup](/front/plugins/db_cleanup/)                          |
-|  N/A     |             | N/A           | SQL query          |                 | N/A, but the External SQLite DB plugins work similarly              |
+| `DDNS`          | ⚙ system       | DDNS update plugin           |          | Script             | [📚ddns_update](/front/plugins/ddns_update/)                        |
+| `DHCPLSS`       | 🔍 dev scanner | DHCP leases plugin           |          | Script             | [📚dhcp_leases](/front/plugins/dhcp_leases/)                        |
+| `DHCPSRVS`      | ♻ other        | DHCP servers plugin          |          | Script             | [📚dhcp_servers](/front/plugins/dhcp_servers/)                      |
+| `INTRNT`        | 🔍 dev scanner | Internet IP scanner          |          | Script             | [📚internet_ip](/front/plugins/internet_ip/)                        |
+| `INTRSPD`       | ♻ other        | Internet speed test plugin   |          | Script             | [📚internet_speedtest](/front/plugins/internet_speedtest/)          |
+| `MAINT`         | ⚙ system       | Maintenance plugin           |          | Script             | [📚maintenance](/front/plugins/maintenance/)                        |
+| `MQTT`          | 💬 publisher   | MQTT publisher plugin        |          | Script             | [📚_publisher_mqtt](/front/plugins/_publisher_mqtt/)                |
+| `NEWDEV`        | ⚙ system       | New device template          |  Yes     | Template           | [📚newdev_template](/front/plugins/newdev_template/)                |
+| `NMAP`          | ♻ other        | Nmap scan plugin             |          | Script             | [📚nmap_scan](/front/plugins/nmap_scan/)                            |
+| `NMAPDEV`       | 🔍 dev scanner | Nmap device scan plugin      |          | Script             | [📚nmap_dev_scan](/front/plugins/nmap_dev_scan/)                    |
+| `NSLOOKUP`      | ♻ other        | NSLookup scan plugin         |          | Script             | [📚nslookup_scan](/front/plugins/nslookup_scan/)                    |
+| `NTFPRCS`       | ⚙ system       | Notification processing      |  Yes     | Template           | [📚notification_processing](/front/plugins/notification_processing/)|
+| `NTFY`          | 💬 publisher   | NTFY publisher plugin        |          | Script             | [📚_publisher_ntfy](/front/plugins/_publisher_ntfy/)                |
+| `PHOLUS`        | ♻ other        | Pholus scan plugin           |          | Script             | [📚pholus_scan](/front/plugins/pholus_scan/)                        |
+| `PIHOLE`        | 🔍 dev scanner | Pi-hole scan plugin          |          | SQLite DB          | [📚pihole_scan](/front/plugins/pihole_scan/)                        |
+| `PUSHSAFER`     | 💬 publisher   | Pushsafer publisher plugin   |          | Script             | [📚_publisher_pushsafer](/front/plugins/_publisher_pushsafer/)      |
+| `PUSHOVER`      | 💬 publisher   | Pushover publisher plugin    |          | Script             | [📚_publisher_pushover](/front/plugins/_publisher_pushover/)        |
+| `SETPWD`        | ⚙ system       | Set password template        |  Yes     | Template           | [📚set_password](/front/plugins/set_password/)                      |
+| `SMTP`          | 💬 publisher   | Email publisher plugin       |          | Script             | [📚_publisher_email](/front/plugins/_publisher_email/)              |
+| `SNMPDSC`       | 🔍 dev scanner | SNMP discovery plugin        |          | Script             | [📚snmp_discovery](/front/plugins/snmp_discovery/)                  |
+| `UNDIS`         | ♻ other        | Undiscoverables scan plugin  |          | Script             | [📚undiscoverables](/front/plugins/undiscoverables/)                |
+| `UNFIMP`        | 🔍 dev scanner | UniFi import plugin          |          | Script             | [📚unifi_import](/front/plugins/unifi_import/)                      |
+| `VNDRPDT`       | ⚙ system       | Vendor update plugin         |          | Script             | [📚vendor_update](/front/plugins/vendor_update/)                    |
+| `WEBHOOK`       | 💬 publisher   | Webhook publisher plugin     |          | Script             | [📚_publisher_webhook](/front/plugins/_publisher_webhook/)          |
+| `WEBMON`        | ♻ other        | Website monitor plugin       |          | Script             | [📚website_monitor](/front/plugins/website_monitor/)                |
+
 
 
 > \* The database cleanup plugin (`DBCLNP`) is not _required_ but the app will become unusable after a while if not executed.
 >
 > \** The Undiscoverables plugin (`UNDIS`) inserts only user-specified dummy devices.
 
-> [!NOTE] 
-> You soft-disable plugins via Settings or completely ignore plugins by placing a `ignore_plugin` file into the plugin directory. The difference is that ignored plugins don't show up anywhere in the UI (Settings, Device details, Plugins pages). The app skips ignored plugins completely. Device-detecting plugins insert values into the `CurrentScan` database table.  The plugins that are not required are safe to ignore, however it makes sense to have a least some device-detecting plugins (that insert entries into the `CurrentScan` table) enabled, such as `ARPSCAN` or `PIHOLE`. You can also load/unload Plugins with the `LOADED_PLUGINS` setting.
-
 > It's recommended to use the same schedule interval for all plugins responsible for discovering new devices.
 
+## Developing custom plugins
+
 If you want to develop a custom plugin, please read this [Plugin development guide](/docs/PLUGINS_DEV.md).

front/plugins/db_cleanup/script.py

@@ -32,7 +32,7 @@ def main():
     parser.add_argument('pluginskeephistory', action="store", help="TBC")
     parser.add_argument('hourstokeepnewdevice', action="store", help="TBC")
     parser.add_argument('daystokeepevents', action="store", help="TBC")
-    parser.add_argument('pholuskeepdays', action="store", help="TBC")
+    parser.add_argument('pholuskeepdays', action="store", help="TBC") # unused
     
     values = parser.parse_args()
 

front/plugins/internet_ip/config.json

@@ -258,6 +258,28 @@
         }
       ]
     },
+    {
+      "function": "RETRIES",
+      "type": "integer",
+      "default_value": 3,
+      "options": [],
+      "localized": [
+        "name",
+        "description"
+      ],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Retries"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "Number of retries before Internet is marked as unreachable. Exponential backoff strategy is used between tries (1s - 1st retry, 2s - 2nd retry, etc)."
+        }
+      ]
+    },
     {
       "function": "WATCH",
       "type": "text.multiselect",
@@ -434,6 +456,21 @@
           "string" : "Response"
         }
       ]
+    },    
+    {
+      "column": "Watched_Value3",
+      "css_classes": "col-sm-2",
+      "show": true,
+      "type": "label",            
+      "default_value":"",
+      "options": [],
+      "localized": ["name"],
+      "name":[
+        {
+          "language_code": "en_us",
+          "string" : "Retries needed"
+        }
+      ]
     },
     {
       "column": "Dummy",

front/plugins/internet_ip/script.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 
 import os
+import time
 import pathlib
 import argparse
 import sys
@@ -29,6 +30,8 @@ RESULT_FILE = os.path.join(CUR_PATH, 'last_result.log')
 
 pluginName = 'INTRNT'
 
+no_internet_ip = '0.0.0.0'
+
 def main():
 
     mylog('verbose', [f'[{pluginName}] In script'])     
@@ -45,13 +48,20 @@ def main():
 
     mylog('verbose', [f'[{pluginName}] INTRNT_DIG_GET_IP_ARG: ', DIG_GET_IP_ARG])     
 
-    # Decode the base64-encoded value to get the actual value in ASCII format.
+    # perform the new IP lookup N times specified by the INTRNT_TRIES setting
-    # DIG_GET_IP_ARG = base64.b64decode(DIG_GET_IP_ARG).decode('ascii')
+    new_internet_IP = ""
-    
+    INTRNT_RETRIES  = get_setting_value("INTRNT_RETRIES")
-    # mylog('verbose', [f'[{pluginName}] DIG_GET_IP_ARG resolved: {DIG_GET_IP_ARG} ']) 
+    tries_needed    = 0
 
-    # perform the new IP lookup
+    for i in range(INTRNT_RETRIES + 1):
-    new_internet_IP, cmd_output = check_internet_IP( PREV_IP, DIG_GET_IP_ARG)   
+
+        new_internet_IP, cmd_output = check_internet_IP( PREV_IP, DIG_GET_IP_ARG)   
+
+        if new_internet_IP == no_internet_ip:
+            time.sleep(1*i) # Exponential backoff strategy
+        else:
+            tries_needed = i
+            break
 
     plugin_objects = Plugin_Objects(RESULT_FILE)    
     
@@ -60,7 +70,7 @@ def main():
         secondaryId = new_internet_IP,  # IP Address 
         watched1    = f'Previous IP: {PREV_IP}',
         watched2    = cmd_output.replace('\n',''),
-        watched3    = '',  
+        watched3    = tries_needed,  
         watched4    = '',
         extra       = f'Previous IP: {PREV_IP}', 
         foreignKey  = 'Internet')
@@ -84,7 +94,7 @@ def check_internet_IP ( PREV_IP, DIG_GET_IP_ARG ):
     mylog('verbose', [f'[{pluginName}]  Current internet_IP : {internet_IP}'])        
     
     # Check previously stored IP    
-    previous_IP = '0.0.0.0'
+    previous_IP = no_internet_ip
 
     if  PREV_IP is not None and len(PREV_IP) > 0 :
         previous_IP = PREV_IP
@@ -116,7 +126,7 @@ def get_internet_IP (DIG_GET_IP_ARG):
 
     # Handle invalid response
     if IP == '':
-        IP = '0.0.0.0'
+        IP = no_internet_ip
 
     return IP, cmd_output
 

front/settings.php

@@ -592,8 +592,10 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
 
     }
 
-
+    // init finished
+    
     setupSmoothScrolling()
+    hideSpinner()
 
   }
 
@@ -840,6 +842,8 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
   // -----------------------------------------------------------------------------
   function handleLoadingDialog()
   {
+
+    // check if config file has been updated
     $.get('api/app_state.json?nocache=' + Date.now(), function(appState) {   
 
       fileModificationTime = <?php echo filemtime($confPath)*1000;?>;  
@@ -865,7 +869,8 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
         // check if the app is initialized and hide the spinner
         if(isAppInitialized())
         {          
-          hideSpinner()
+          // init page
+          getData()
           
           // reload page if outdated information might be displayed
           if(secondsSincePageLoad() > 3)
@@ -892,8 +897,6 @@ while ($row = $result -> fetchArray (SQLITE3_ASSOC)) {
   }
   
 
-  getData()
-
 </script>
 
 <script defer>