vh/NetAlertX
1
0
Fork 0
mirror of https://github.com/jokob-sk/NetAlertX.git synced 2025-12-07 09:36:05 -08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into feature/freebox

Browse Source
This commit is contained in:
jokob-sk
2024-12-02 12:04:33 +11:00
committed by GitHub
parent 12f2d12d52 826bd8f524
commit ea16302f1f
53 changed files with 3523 additions and 1942 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
front/plugins/README.md
View File

@@ -23,42 +23,45 @@ NetAlertX supports additional plugins to extend its functionality, each with its
Device-detecting plugins insert values into the `CurrentScan` database table. The plugins that are not required are safe to ignore, however, it makes sense to have at least some device-detecting plugins enabled, such as `ARPSCAN` or `NMAPDEV`.
| ID | Type | Description | Features | Required | Data source | Detailed docs |
| ----------- | ----- | --------------------------------------------------- | -------- | -------- | ----------- | ------------------------------------------------------------------ |
| `APPRISE` | ▶️ | Apprise notification proxy | | | Script | [_publisher_apprise](/front/plugins/_publisher_apprise/) |
| `ARPSCAN` | 🔍 | ARP-scan on current network | | | Script | [arp_scan](/front/plugins/arp_scan/) |
| `AVAHISCAN` | ♻ | Avahi (mDNS-based) name resolution | | | Script | [avahi_scan](/front/plugins/avahi_scan/) |
| `CSVBCKP` | ⚙ | CSV devices backup | | | Script | [csv_backup](/front/plugins/csv_backup/) |
| `DBCLNP` | ⚙ | Database cleanup | | Yes* | Script | [db_cleanup](/front/plugins/db_cleanup/) |
| `DDNS` | ⚙ | DDNS update | | | Script | [ddns_update](/front/plugins/ddns_update/) |
| `DHCPLSS` | 🔍/📥 | Import devices from DHCP leases | | | Script | [dhcp_leases](/front/plugins/dhcp_leases/) |
| `DHCPSRVS` | ♻ | DHCP servers | | | Script | [dhcp_servers](/front/plugins/dhcp_servers/) |
| `INTRNT` | 🔍 | Internet IP scanner | | | Script | [internet_ip](/front/plugins/internet_ip/) |
| `INTRSPD` | ♻ | Internet speed test | | | Script | [internet_speedtest](/front/plugins/internet_speedtest/) |
| `MAINT` | ⚙ | Maintenance of logs, etc. | | | Script | [maintenance](/front/plugins/maintenance/) |
| `MQTT` | ▶️ | MQTT for synching to Home Assistant | | | Script | [_publisher_mqtt](/front/plugins/_publisher_mqtt/) |
| `NBTSCAN` | ♻ | Nbtscan (NetBIOS-based) name resolution | | | Script | [nbtscan_scan](/front/plugins/nbtscan_scan/) |
| `NEWDEV` | ⚙ | New device template | | Yes | Template | [newdev_template](/front/plugins/newdev_template/) |
| `NMAP` | ♻ | Nmap port scanning & discovery | | | Script | [nmap_scan](/front/plugins/nmap_scan/) |
| `NMAPDEV` | 🔍 | Nmap dev scan on current network | | | Script | [nmap_dev_scan](/front/plugins/nmap_dev_scan/) |
| `NSLOOKUP` | ♻ | NSLookup (DNS-based) name resolution | | | Script | [nslookup_scan](/front/plugins/nslookup_scan/) |
| `NTFPRCS` | ⚙ | Notification processing | | Yes | Template | [notification_processing](/front/plugins/notification_processing/) |
| `NTFY` | ▶️ | NTFY notifications | | | Script | [_publisher_ntfy](/front/plugins/_publisher_ntfy/) |
| `OMDSDN` | 📥 | OMADA TP-Link import | 🖧 🔄 | | Script | [omada_sdn_imp](/front/plugins/omada_sdn_imp/) |
| `PIHOLE` | 🔍/📥 | Pi-hole device import & sync | | | SQLite DB | [pihole_scan](/front/plugins/pihole_scan/) |
| `PUSHSAFER` | ▶️ | Pushsafer notifications | | | Script | [_publisher_pushsafer](/front/plugins/_publisher_pushsafer/) |
| `PUSHOVER` | ▶️ | Pushover notifications | | | Script | [_publisher_pushover](/front/plugins/_publisher_pushover/) |
| `SETPWD` | ⚙ | Set password | | Yes | Template | [set_password](/front/plugins/set_password/) |
| `SMTP` | ▶️ | Email notifications | | | Script | [_publisher_email](/front/plugins/_publisher_email/) |
| `SNMPDSC` | 🔍/📥 | SNMP device import & sync | | | Script | [snmp_discovery](/front/plugins/snmp_discovery/) |
| `SYNC` | 🔍/⚙/📥 | Sync & import from NetAlertX instances | 🖧 🔄 | | Script | [sync](/front/plugins/sync/) |
| `TELEGRAM` | ▶️ | Telegram notifications | | | Script | [_publisher_telegram](/front/plugins/_publisher_telegram/) |
| `UNDIS` | 🔍/📥 | Create dummy devices | | | Script | [undiscoverables](/front/plugins/undiscoverables/) |
| `UNFIMP` | 🔍/📥 | UniFi device import & sync | 🖧 | | Script | [unifi_import](/front/plugins/unifi_import/) |
| `VNDRPDT` | ⚙ | Vendor database update | | | Script | [vendor_update](/front/plugins/vendor_update/) |
| `WEBHOOK` | ▶️ | Webhook notifications | | | Script | [_publisher_webhook](/front/plugins/_publisher_webhook/) |
| `WEBMON` | ♻ | Website down monitoring | | | Script | [website_monitor](/front/plugins/website_monitor/) |
| `FREEBOX` | 🔍/♻ | Pull data and names from a Freebox/Iliadbox gateway | | | Script | [freebox](/front/plugins/freebox/) |
| ID | Type | Description | Features | Required | Data source | Detailed docs |
|---------------|---------|--------------------------------------------|----------|----------|--------------------|---------------------------------------------------------------|
| `APPRISE` | ▶️ | Apprise notification proxy | | | Script | [_publisher_apprise](/front/plugins/_publisher_apprise/) |
| `ARPSCAN` | 🔍 | ARP-scan on current network | | | Script | [arp_scan](/front/plugins/arp_scan/) |
| `AVAHISCAN` | ♻ | Avahi (mDNS-based) name resolution | | | Script | [avahi_scan](/front/plugins/avahi_scan/) |
| `CSVBCKP` | ⚙ | CSV devices backup | | | Script | [csv_backup](/front/plugins/csv_backup/) |
| `DBCLNP` | ⚙ | Database cleanup | | Yes* | Script | [db_cleanup](/front/plugins/db_cleanup/) |
| `DDNS` | ⚙ | DDNS update | | | Script | [ddns_update](/front/plugins/ddns_update/) |
| `DHCPLSS` | 🔍/📥 | Import devices from DHCP leases | | | Script | [dhcp_leases](/front/plugins/dhcp_leases/) |
| `DHCPSRVS` | ♻ | DHCP servers | | | Script | [dhcp_servers](/front/plugins/dhcp_servers/) |
| `FREEBOX` | 🔍/♻ | Pull data and names from Freebox/Iliadbox | | | Script | [freebox](/front/plugins/freebox/) |
| `ICMP` | 🔍 | ICMP (ping) status checker | | | Script | [icmp_scan](/front/plugins/icmp_scan/) |
| `INTRNT` | 🔍 | Internet IP scanner | | | Script | [internet_ip](/front/plugins/internet_ip/) |
| `INTRSPD` | ♻ | Internet speed test | | | Script | [internet_speedtest](/front/plugins/internet_speedtest/) |
| `IPNEIGH` | 🔍 | Scan ARP (IPv4) and NDP (IPv6) tables | | | Script | [ipneigh](/front/plugins/ipneigh/) |
| `MAINT` | ⚙ | Maintenance of logs, etc. | | | Script | [maintenance](/front/plugins/maintenance/) |
| `MQTT` | ▶️ | MQTT for synching to Home Assistant | | | Script | [_publisher_mqtt](/front/plugins/_publisher_mqtt/) |
| `NBTSCAN` | ♻ | Nbtscan (NetBIOS-based) name resolution | | | Script | [nbtscan_scan](/front/plugins/nbtscan_scan/) |
| `NEWDEV` | ⚙ | New device template | | Yes | Template | [newdev_template](/front/plugins/newdev_template/) |
| `NMAP` | ♻ | Nmap port scanning & discovery | | | Script | [nmap_scan](/front/plugins/nmap_scan/) |
| `NMAPDEV` | 🔍 | Nmap dev scan on current network | | | Script | [nmap_dev_scan](/front/plugins/nmap_dev_scan/) |
| `NSLOOKUP` | ♻ | NSLookup (DNS-based) name resolution | | | Script | [nslookup_scan](/front/plugins/nslookup_scan/) |
| `NTFPRCS` | ⚙ | Notification processing | | Yes | Template | [notification_processing](/front/plugins/notification_processing/)|
| `NTFY` | ▶️ | NTFY notifications | | | Script | [_publisher_ntfy](/front/plugins/_publisher_ntfy/) |
| `OMDSDN` | 📥 | OMADA TP-Link import | 🖧 🔄 | | Script | [omada_sdn_imp](/front/plugins/omada_sdn_imp/) |
| `PIHOLE` | 🔍/📥 | Pi-hole device import & sync | | | SQLite DB | [pihole_scan](/front/plugins/pihole_scan/) |
| `PUSHSAFER` | ▶️ | Pushsafer notifications | | | Script | [_publisher_pushsafer](/front/plugins/_publisher_pushsafer/) |
| `PUSHOVER` | ▶️ | Pushover notifications | | | Script | [_publisher_pushover](/front/plugins/_publisher_pushover/) |
| `SETPWD` | ⚙ | Set password | | Yes | Template | [set_password](/front/plugins/set_password/) |
| `SMTP` | ▶️ | Email notifications | | | Script | [_publisher_email](/front/plugins/_publisher_email/) |
| `SNMPDSC` | 🔍/📥 | SNMP device import & sync | | | Script | [snmp_discovery](/front/plugins/snmp_discovery/) |
| `SYNC` | 🔍/⚙/📥| Sync & import from NetAlertX instances | 🖧 🔄 | | Script | [sync](/front/plugins/sync/) |
| `TELEGRAM` | ▶️ | Telegram notifications | | | Script | [_publisher_telegram](/front/plugins/_publisher_telegram/) |
| `UNDIS` | 🔍/📥 | Create dummy devices | | | Script | [undiscoverables](/front/plugins/undiscoverables/) |
| `UNFIMP` | 🔍/📥 | UniFi device import & sync | 🖧 | | Script | [unifi_import](/front/plugins/unifi_import/) |
| `VNDRPDT` | ⚙ | Vendor database update | | | Script | [vendor_update](/front/plugins/vendor_update/) |
| `WEBHOOK` | ▶️ | Webhook notifications | | | Script | [_publisher_webhook](/front/plugins/_publisher_webhook/) |
| `WEBMON` | ♻ | Website down monitoring | | | Script | [website_monitor](/front/plugins/website_monitor/) |
> \* The database cleanup plugin (`DBCLNP`) is not _required_ but the app will become unusable after a while if not executed.
>

7
front/plugins/icmp_scan/README.md Executable file
View File

@@ -0,0 +1,7 @@
## Overview
Plugin for pinging existing devices via the [ping](https://linux.die.net/man/8/ping) network utility. The devices have to be accessible from the container. You can use this plugin with other suplementing plugins as described in the [subnets docs](https://github.com/jokob-sk/NetAlertX/blob/main/docs/SUBNETS.md).
### Usage
- Check the Settings page for details.

399
front/plugins/icmp_scan/config.json Executable file
View File

@@ -0,0 +1,399 @@
{
"code_name": "icmp_scan",
"unique_prefix": "ICMP",
"plugin_type": "other",
"execution_order" : "Layer_4",
"enabled": true,
"data_source": "script",
"show_ui": true,
"data_filters": [
{
"compare_column": "Object_PrimaryID",
"compare_operator": "==",
"compare_field_id": "txtMacFilter",
"compare_js_template": "'{value}'.toString()",
"compare_use_quotes": true
}
],
"localized": ["display_name", "description", "icon"],
"display_name": [
{
"language_code": "en_us",
"string": "ICMP (Status check)"
}
],
"icon": [
{
"language_code": "en_us",
"string": "<i class=\"fa-solid fa-search\"></i>"
}
],
"description": [
{
"language_code": "en_us",
"string": "A plugin to check the status of the device."
}
],
"params": [
{
"name": "ips",
"type": "sql",
"value": "SELECT devLastIP from DEVICES order by devMac",
"timeoutMultiplier": true
}
],
"settings": [
{
"function": "RUN",
"events": ["run"],
"type": {
"dataType": "string",
"elements": [
{ "elementType": "select", "elementOptions": [], "transformers": [] }
]
},
"default_value": "disabled",
"options": [
"disabled",
"on_new_device",
"once",
"schedule",
"always_after_scan"
],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "When to run"
}
],
"description": [
{
"language_code": "en_us",
"string": "Enable a regular scan of your devices with PING to determine their status. If you select <code>schedule</code> the interval from below is applied, for which the recommendation is to <a href=\"https://github.com/jokob-sk/NetAlertX/blob/main/docs/NOTIFICATIONS.md\" target=\"_blank\">align all scan schedules</a> otherwise false down reports will be generated."
}
]
},
{
"function": "CMD",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [{ "readonly": "true" }],
"transformers": []
}
]
},
"default_value": "python3 /app/front/plugins/icmp_scan/icmp.py",
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Command"
}
],
"description": [
{
"language_code": "en_us",
"string": "Command to run. This can not be changed"
}
]
},
{
"function": "ARGS",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [],
"transformers": []
}
]
},
"default_value": "-i 0.5 -c 3 -W 4 -w 5",
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Command arguments"
}
],
"description": [
{
"language_code": "en_us",
"string": "Arguments passed to the <code>ping</code> command. Please be careful modifying these."
}
]
},
{
"function": "RUN_SCHD",
"type": {
"dataType": "string",
"elements": [
{ "elementType": "input", "elementOptions": [], "transformers": [] }
]
},
"default_value": "*/5 * * * *",
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Schedule"
}
],
"description": [
{
"language_code": "en_us",
"string": "Only enabled if you select <code>schedule</code> in the <a href=\"#ICMP_RUN\"><code>ICMP_RUN</code> setting</a>. Make sure you enter the schedule in the correct cron-like format (e.g. validate at <a href=\"https://crontab.guru/\" target=\"_blank\">crontab.guru</a>). For example entering <code>0 4 * * *</code> will run the scan after 4 am in the <a onclick=\"toggleAllSettings()\" href=\"#TIMEZONE\"><code>TIMEZONE</code> you set above</a>. Will be run NEXT time the time passes."
}
]
},
{
"function": "RUN_TIMEOUT",
"type": {
"dataType": "integer",
"elements": [
{
"elementType": "input",
"elementOptions": [{ "type": "number" }],
"transformers": []
}
]
},
"default_value": 10,
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Run timeout"
}
],
"description": [
{
"language_code": "en_us",
"string": "Maximum time in seconds to wait for the script to finish. If this time is exceeded the script is aborted."
}
]
}
],
"database_column_definitions": [
{
"column": "Index",
"css_classes": "col-sm-2",
"show": true,
"type": "none",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Index"
}
]
},
{
"column": "Object_PrimaryID",
"mapped_to_column": "cur_MAC",
"css_classes": "col-sm-2",
"show": true,
"type": "device_name_mac",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "MAC"
}
]
},
{
"column": "Object_SecondaryID",
"mapped_to_column": "cur_IP",
"css_classes": "col-sm-2",
"show": true,
"type": "device_ip",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "IP"
}
]
},
{
"column": "Watched_Value1",
"mapped_to_column": "cur_Name",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Name"
}
]
},
{
"column": "Watched_Value2",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Output"
}
]
},
{
"column": "Watched_Value3",
"css_classes": "col-sm-2",
"show": false,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "N/A"
}
]
},
{
"column": "Watched_Value4",
"css_classes": "col-sm-2",
"show": false,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "N/A"
}
]
},
{
"column": "Dummy",
"mapped_to_column": "cur_ScanMethod",
"mapped_to_column_data": {
"value": "ICMP"
},
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Scan method"
}
]
},
{
"column": "DateTimeCreated",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Created"
}
]
},
{
"column": "DateTimeChanged",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Changed"
}
]
},
{
"column": "Status",
"css_classes": "col-sm-1",
"show": true,
"type": "replace",
"default_value": "",
"options": [
{
"equals": "watched-not-changed",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-square-check'></i><div></div>"
},
{
"equals": "watched-changed",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-triangle-exclamation'></i></div>"
},
{
"equals": "new",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-circle-plus'></i></div>"
},
{
"equals": "missing-in-last-scan",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-question'></i></div>"
}
],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Status"
}
]
}
]
}

156
front/plugins/icmp_scan/icmp.py Executable file
View File

@@ -0,0 +1,156 @@
#!/usr/bin/env python
# test script by running:
# tbc
import os
import pathlib
import argparse
import subprocess
import sys
import hashlib
import csv
import sqlite3
import re
from io import StringIO
from datetime import datetime
# Register NetAlertX directories
INSTALL_PATH="/app"
sys.path.extend([f"{INSTALL_PATH}/front/plugins", f"{INSTALL_PATH}/server"])
from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64
from logger import mylog, append_line_to_file
from helper import timeNowTZ, get_setting_value
from const import logPath, applicationPath, fullDbPath
from database import DB
from device import Device_obj
import conf
from pytz import timezone
# Make sure the TIMEZONE for logging is correct
conf.tz = timezone(get_setting_value('TIMEZONE'))
CUR_PATH = str(pathlib.Path(__file__).parent.resolve())
LOG_FILE = os.path.join(CUR_PATH, 'script.log')
RESULT_FILE = os.path.join(CUR_PATH, 'last_result.log')
pluginName = 'ICMP'
def main():
mylog('verbose', [f'[{pluginName}] In script'])
timeout = get_setting_value('ICMP_RUN_TIMEOUT')
args = get_setting_value('ICMP_ARGS')
# Create a database connection
db = DB() # instance of class DB
db.open()
# Initialize the Plugin obj output file
plugin_objects = Plugin_Objects(RESULT_FILE)
# Create a Device_obj instance
device_handler = Device_obj(db)
# Retrieve devices
all_devices = device_handler.getAll()
mylog('verbose', [f'[{pluginName}] Devices to PING: {len(all_devices)}'])
for device in all_devices:
is_online, output = execute_scan(device['devLastIP'], timeout, args)
mylog('verbose', [f'[{pluginName}] ip: "{device['devLastIP']}" is_online: "{is_online}"'])
if is_online:
plugin_objects.add_object(
# "MAC", "IP", "Name", "Output"
primaryId = device['devMac'],
secondaryId = device['devLastIP'],
watched1 = device['devName'],
watched2 = output.replace('\n',''),
watched3 = '',
watched4 = '',
extra = '',
foreignKey = device['devMac'])
plugin_objects.write_result_file()
mylog('verbose', [f'[{pluginName}] Script finished'])
return 0
#===============================================================================
# Execute scan
#===============================================================================
def execute_scan (ip, timeout, args):
"""
Execute the ICMP command on IP.
"""
icmp_args = ['ping'] + args.split() + [ip]
# Execute command
output = ""
try:
# try runnning a subprocess with a forced (timeout) in case the subprocess hangs
output = subprocess.check_output (icmp_args, universal_newlines=True, stderr=subprocess.STDOUT, timeout=(timeout), text=True)
mylog('verbose', [f'[{pluginName}] DEBUG OUTPUT : {output}'])
# Parse output using case-insensitive regular expressions
#Synology-NAS:/# ping -i 0.5 -c 3 -W 8 -w 9 192.168.1.82
# PING 192.168.1.82 (192.168.1.82): 56 data bytes
# 64 bytes from 192.168.1.82: seq=0 ttl=64 time=0.080 ms
# 64 bytes from 192.168.1.82: seq=1 ttl=64 time=0.081 ms
# 64 bytes from 192.168.1.82: seq=2 ttl=64 time=0.089 ms
# --- 192.168.1.82 ping statistics ---
# 3 packets transmitted, 3 packets received, 0% packet loss
# round-trip min/avg/max = 0.080/0.083/0.089 ms
# Synology-NAS:/# ping -i 0.5 -c 3 -W 8 -w 9 192.168.1.82a
# ping: bad address '192.168.1.82a'
# Synology-NAS:/# ping -i 0.5 -c 3 -W 8 -w 9 192.168.1.92
# PING 192.168.1.92 (192.168.1.92): 56 data bytes
# --- 192.168.1.92 ping statistics ---
# 3 packets transmitted, 0 packets received, 100% packet loss
# TODO: parse output and return True if online, False if Offline (100% packet loss, bad address)
is_online = True
# Check for 0% packet loss in the output
if re.search(r"0% packet loss", output, re.IGNORECASE):
is_online = True
elif re.search(r"bad address", output, re.IGNORECASE):
is_online = False
elif re.search(r"100% packet loss", output, re.IGNORECASE):
is_online = False
return is_online, output
except subprocess.CalledProcessError as e:
# An error occurred, handle it
mylog('verbose', [f'[{pluginName}] ⚠ ERROR - check logs'])
mylog('verbose', [f'[{pluginName}]', e.output])
return False, output
except subprocess.TimeoutExpired as timeErr:
mylog('verbose', [f'[{pluginName}] TIMEOUT - the process forcefully terminated as timeout reached'])
return False, output
return False, output
#===============================================================================
# BEGIN
#===============================================================================
if __name__ == '__main__':
main()

27
front/plugins/ipneigh/README.md Executable file
View File

@@ -0,0 +1,27 @@
## Overview
This plugin reads from the ARP and NDP tables using the `ip neigh` command.
This differs from the `ARPSCAN` plugin because
* It does *not* send arp requests, it just reads the table
* It supports IPv6
* It sends an IPv6 multicast ping to solicit IPv6 neighbour discovery
### Quick setup guide
To set up the plugin correctly, make sure to add in the plugin settings the name of the interfaces you want to scan. This plugin doesn't use the global `SCAN_SUBNET` setting, this is because by design it is not aware of subnets, it just looks at all the IPs reachable from an interface.
### Usage
- Head to **Settings** > **IP Neigh** to adjust the settings
- Interfaces are extracted from the `SCAN_SUBNETS` setting (make sure you add interfaces in the prescribed format, e.g. `192.168.1.0/24 --interface=eth1`)
### Notes
- `ARPSCAN` does a better job at discovering IPv4 devices because it explicitly sends arp requests
- IPv6 devices will often have multiple addresses, but the ping answer will contain only one. This means that in general this plugin will not discover every address but only those who answer
### Other info
- Author : [KayJay7](https://github.com/KayJay7)
- Date : 31-Nov-2024 - version 1.0

402
front/plugins/ipneigh/config.json Executable file
View File

@@ -0,0 +1,402 @@
{
"code_name": "ipneigh",
"unique_prefix": "IPNEIGH",
"plugin_type": "device_scanner",
"execution_order": "Layer_0",
"enabled": true,
"data_source": "script",
"mapped_to_table": "CurrentScan",
"data_filters": [
{
"compare_column": "Object_PrimaryID",
"compare_operator": "==",
"compare_field_id": "txtMacFilter",
"compare_js_template": "'{value}'.toString()",
"compare_use_quotes": true
}
],
"show_ui": true,
"localized": [
"display_name",
"description",
"icon"
],
"display_name": [
{
"language_code": "en_us",
"string": "IP Neigh"
}
],
"description": [
{
"language_code": "en_us",
"string": "Plugin to scan the ip tables"
}
],
"icon": [
{
"language_code": "en_us",
"string": "<i class=\"fa fa-search\"></i>"
}
],
"params": [],
"settings": [
{
"function": "RUN",
"events": [
"run"
],
"type": {
"dataType": "string",
"elements": [
{
"elementType": "select",
"elementOptions": [],
"transformers": []
}
]
},
"default_value": "disabled",
"options": [
"disabled",
"once",
"schedule",
"always_after_scan",
"on_new_device",
"on_notification"
],
"localized": [
"name",
"description"
],
"name": [
{
"language_code": "en_us",
"string": "When to run"
}
],
"description": [
{
"language_code": "en_us",
"string": "When the plugin should run. Good options are <code>always_after_scan</code>, <code>on_new_device</code>, <code>on_notification</code>"
}
]
},
{
"function": "RUN_SCHD",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [],
"transformers": []
}
]
},
"default_value": "*/5 * * * *",
"options": [],
"localized": [
"name",
"description"
],
"name": [
{
"language_code": "en_us",
"string": "Schedule"
}
],
"description": [
{
"language_code": "en_us",
"string": "Only enabled if you select <code>schedule</code> in the <a href=\"#SYNC_RUN\"><code>SYNC_RUN</code> setting</a>. Make sure you enter the schedule in the correct cron-like format (e.g. validate at <a href=\"https://crontab.guru/\" target=\"_blank\">crontab.guru</a>). For example entering <code>0 4 * * *</code> will run the scan after 4 am in the <a onclick=\"toggleAllSettings()\" href=\"#TIMEZONE\"><code>TIMEZONE</code> you set above</a>. Will be run NEXT time the time passes."
}
]
},
{
"function": "CMD",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [
{
"readonly": "true"
}
],
"transformers": []
}
]
},
"default_value": "python3 /app/front/plugins/ipneigh/ipneigh.py",
"options": [],
"localized": [
"name",
"description"
],
"name": [
{
"language_code": "en_us",
"string": "Command"
}
],
"description": [
{
"language_code": "en_us",
"string": "Command to run. This can not be changed"
}
]
},
{
"function": "RUN_TIMEOUT",
"type": {
"dataType": "integer",
"elements": [
{
"elementType": "input",
"elementOptions": [
{
"type": "number"
}
],
"transformers": []
}
]
},
"default_value": 30,
"options": [],
"localized": [
"name",
"description"
],
"name": [
{
"language_code": "en_us",
"string": "Run timeout"
}
],
"description": [
{
"language_code": "en_us",
"string": "Maximum time in seconds to wait for the script to finish. If this time is exceeded the script is aborted."
}
]
}
],
"database_column_definitions": [
{
"column": "Index",
"css_classes": "col-sm-2",
"show": true,
"type": "none",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Index"
}
]
},
{
"column": "Object_PrimaryID",
"mapped_to_column": "cur_MAC",
"css_classes": "col-sm-2",
"show": true,
"type": "device_name_mac",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "MAC"
}
]
},
{
"column": "Object_SecondaryID",
"mapped_to_column": "cur_IP",
"css_classes": "col-sm-2",
"show": true,
"type": "device_ip",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "IP"
}
]
},
{
"column": "Watched_Value1",
"mapped_to_column": "cur_Name",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Name"
}
]
},
{
"column": "Watched_Value2",
"mapped_to_column": "cur_Vendor",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Vendor"
}
]
},
{
"column": "Watched_Value3",
"mapped_to_column": "cur_Type",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Device Type"
}
]
},
{
"column": "Watched_Value4",
"css_classes": "col-sm-2",
"show": false,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "N/A"
}
]
},
{
"column": "Dummy",
"mapped_to_column": "cur_ScanMethod",
"mapped_to_column_data": {
"value": "IPNEIGH"
},
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Scan method"
}
]
},
{
"column": "DateTimeCreated",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Created"
}
]
},
{
"column": "DateTimeChanged",
"css_classes": "col-sm-2",
"show": true,
"type": "label",
"default_value": "",
"options": [],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Changed"
}
]
},
{
"column": "Status",
"css_classes": "col-sm-1",
"show": true,
"type": "replace",
"default_value": "",
"options": [
{
"equals": "watched-not-changed",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-square-check'></i><div></div>"
},
{
"equals": "watched-changed",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-triangle-exclamation'></i></div>"
},
{
"equals": "new",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-circle-plus'></i></div>"
},
{
"equals": "missing-in-last-scan",
"replacement": "<div style='text-align:center'><i class='fa-solid fa-question'></i></div>"
}
],
"localized": [
"name"
],
"name": [
{
"language_code": "en_us",
"string": "Status"
}
]
}
]
}

146
front/plugins/ipneigh/ipneigh.py Executable file
View File

@@ -0,0 +1,146 @@
#!/usr/bin/env python
import os
import pathlib
import sys
import json
import sqlite3
import subprocess
from datetime import datetime
from pytz import timezone
from functools import reduce
# Define the installation path and extend the system path for plugin imports
INSTALL_PATH = "/app"
sys.path.extend([f"{INSTALL_PATH}/front/plugins", f"{INSTALL_PATH}/server"])
from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64, handleEmpty
from plugin_utils import get_plugins_configs
from logger import mylog
from const import pluginsPath, fullDbPath
from helper import timeNowTZ, get_setting_value
from notification import write_notification
import conf
# Make sure the TIMEZONE for logging is correct
conf.tz = timezone(get_setting_value('TIMEZONE'))
# Define the current path and log file paths
CUR_PATH = str(pathlib.Path(__file__).parent.resolve())
LOG_FILE = os.path.join(CUR_PATH, 'script.log')
RESULT_FILE = os.path.join(CUR_PATH, 'last_result.log')
# Initialize the Plugin obj output file
plugin_objects = Plugin_Objects(RESULT_FILE)
pluginName = 'IPNEIGH'
def main():
mylog('verbose', [f'[{pluginName}] In script'])
# Retrieve configuration settings
SCAN_SUBNETS = get_setting_value('SCAN_SUBNETS')
mylog('verbose', [f'[{pluginName}] SCAN_SUBNETS value: {SCAN_SUBNETS}'])
# Extract interfaces from SCAN_SUBNETS
interfaces = ','.join(
entry.split('--interface=')[-1].strip() for entry in SCAN_SUBNETS if '--interface=' in entry
)
mylog('verbose', [f'[{pluginName}] Interfaces value: "{interfaces}"'])
# retrieve data
raw_neighbors = get_neighbors(interfaces)
neighbors = parse_neighbors(raw_neighbors)
# Process the data into native application tables
if len(neighbors) > 0:
for device in neighbors:
plugin_objects.add_object(
primaryId = device['mac'],
secondaryId = device['ip'],
watched4 = device['last_seen'],
# The following are always unknown
watched1 = device['hostname'], # don't use these --> handleEmpty(device['hostname']),
watched2 = device['vendor'], # handleEmpty(device['vendor']),
watched3 = device['device_type'], # handleEmpty(device['device_type']),
extra = '',
foreignKey = "" #device['mac']
# helpVal1 = "Something1", # Optional Helper values to be passed for mapping into the app
# helpVal2 = "Something1", # If you need to use even only 1, add the remaining ones too
# helpVal3 = "Something1", # and set them to 'null'. Check the the docs for details:
# helpVal4 = "Something1", # https://github.com/jokob-sk/NetAlertX/blob/main/docs/PLUGINS_DEV.md
)
mylog('verbose', [f'[{pluginName}] New entries: "{len(neighbors)}"'])
# log result
plugin_objects.write_result_file()
return 0
def parse_neighbors(raw_neighbors: list[str]):
neighbors = []
for line in raw_neighbors:
if "lladdr" in line and "REACHABLE" in line:
# Known data
fields = line.split()
if not is_multicast(fields[0]):
# mylog('verbose', [f'[{pluginName}] adding ip {fields[0]}"'])
neighbor = {}
neighbor['ip'] = fields[0]
neighbor['mac'] = fields[2]
neighbor['last_seen'] = datetime.now()
# Unknown data
neighbor['hostname'] = '(unknown)'
neighbor['vendor'] = '(unknown)'
neighbor['device_type'] = '(unknown)'
neighbors.append(neighbor)
return neighbors
def is_multicast(ip):
prefixes = ['ff', '224', '231', '232', '233', '234', '238', '239']
return reduce(lambda acc, prefix: acc or ip.startswith(prefix), prefixes, False)
# retrieve data
def get_neighbors(interfaces):
results = []
for interface in interfaces.split(","):
try:
# Ping all IPv6 devices in multicast to trigger NDP
mylog('verbose', [f'[{pluginName}] Pinging on interface: "{interface}"'])
command = f"ping ff02::1%{interface} -c 2".split()
subprocess.run(command)
mylog('verbose', [f'[{pluginName}] Pinging completed: "{interface}"'])
# Check the neighbourhood tables
mylog('verbose', [f'[{pluginName}] Scanning interface: "{interface}"'])
command = f"ip neighbor show nud all dev {interface}".split()
output = subprocess.check_output(command, universal_newlines=True)
results += output.split("\n")
mylog('verbose', [f'[{pluginName}] Scanning interface succeded: "{interface}"'])
except subprocess.CalledProcessError as e:
# An error occurred, handle it
mylog('verbose', [f'[{pluginName}] Scanning interface failed: "{interface}"'])
error_type = type(e).__name__ # Capture the error type
return results
if __name__ == '__main__':
main()

134
front/plugins/newdev_template/config.json
View File

@@ -265,6 +265,34 @@
}
]
},
{
"function": "replace_preset_icon",
"type": {
"dataType": "boolean",
"elements": [
{
"elementType": "input",
"elementOptions": [{ "type": "checkbox" }],
"transformers": []
}
]
},
"default_value": false,
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Overwrite Preset Icon"
}
],
"description": [
{
"language_code": "en_us",
"string": "If checked, the application replaces the preset icon in <code>NEWDEV_devIcon</code> with a pre-assigned vendor or device icon if found. If this is checked, avoid manually assigning this icon to devices, as it may be replaced."
}
]
},
{
"function": "devMac",
"type": {
@@ -284,7 +312,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device MAC"
"string": "MAC"
}
],
"description": [
@@ -313,18 +341,19 @@
"name": [
{
"language_code": "en_us",
"string": "Device Name"
"string": "Name"
}
],
"description": [
{
"language_code": "en_us",
"string": "The name of the device. Uneditable as internal functionality is dependent on specific new device names."
"string": "The name of the device. If the value is set to <code>(unknown)</code> or <code>(name not found)</code> the application will try to discover the name of the device."
}
]
},
{
"function": "devOwner",
"events": ["add_option"],
"type": {
"dataType": "string",
"elements": [
@@ -345,7 +374,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Owner"
"string": "Owner"
}
],
"description": [
@@ -357,6 +386,7 @@
},
{
"function": "devType",
"events": ["add_option"],
"type": {
"dataType": "string",
"elements": [
@@ -382,7 +412,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Type"
"string": "Type"
}
],
"description": [
@@ -399,7 +429,7 @@
"elements": [
{
"elementType": "input",
"elementOptions": [{ "readonly": "true" }],
"elementOptions": [],
"transformers": []
}
]
@@ -411,13 +441,13 @@
"name": [
{
"language_code": "en_us",
"string": "Device Vendor"
"string": "Vendor"
}
],
"description": [
{
"language_code": "en_us",
"string": "The vendor of the device. Uneditable - Autodetected."
"string": "The vendor of the device. If set to empty or <code>(unknown)</code> teh app will try to auto-detect it."
}
]
},
@@ -451,6 +481,7 @@
},
{
"function": "devGroup",
"events": ["add_option"],
"type": {
"dataType": "string",
"elements": [
@@ -471,7 +502,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Group"
"string": "Group"
}
],
"description": [
@@ -486,7 +517,7 @@
"type": {
"dataType": "string",
"elements": [
{ "elementType": "input", "elementOptions": [], "transformers": [] }
{ "elementType": "textarea", "elementOptions": [], "transformers": [] }
]
},
"default_value": "",
@@ -495,7 +526,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Comments"
"string": "Comments"
}
],
"description": [
@@ -570,7 +601,7 @@
"elements": [
{
"elementType": "input",
"elementOptions": [{ "readonly": "true" }],
"elementOptions": [],
"transformers": []
}
]
@@ -638,13 +669,13 @@
"name": [
{
"language_code": "en_us",
"string": "Scan Cycle"
"string": "Scan device"
}
],
"description": [
{
"language_code": "en_us",
"string": "The default value of the <code>Scan device</code> dropdown. Enable if newly discovered devices should be scanned."
"string": "Select if the device should be scanned."
}
]
},
@@ -722,13 +753,13 @@
"name": [
{
"language_code": "en_us",
"string": "Alert Device Down"
"string": "Alert Down"
}
],
"description": [
{
"language_code": "en_us",
"string": "Indicates whether an alert should be triggered when the device goes down. The default value of the <code>Alert Down</code> checkbox."
"string": "Indicates whether an alert should be triggered when the device goes down. The device has to be down for longet than the time set in the <b>Alert Down After</b> <code>NTFPRCS_alert_down_time</code> setting."
}
]
},
@@ -759,7 +790,7 @@
"description": [
{
"language_code": "en_us",
"string": "The default value of the <code>Skip repeated notifications for</code> dropdown. Enter number of <b>hours</b> for which repeated notifications should be ignored for. If you enter <code>0</code> then you get notified on all events."
"string": "Enter number of <b>hours</b> for which repeated notifications should be ignored for. If you select <code>0</code> then you get notified on all events."
}
]
},
@@ -850,6 +881,7 @@
},
{
"function": "devLocation",
"events": ["add_option"],
"type": {
"dataType": "string",
"elements": [
@@ -870,7 +902,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Location"
"string": "Location"
}
],
"description": [
@@ -910,6 +942,7 @@
},
{
"function": "devParentMAC",
"events": ["go_to_node"],
"type": {
"dataType": "string",
"elements": [
@@ -934,7 +967,7 @@
"name": [
{
"language_code": "en_us",
"string": "Network Node MAC Address"
"string": "Network Node"
}
],
"description": [
@@ -951,7 +984,7 @@
"elements": [
{
"elementType": "input",
"elementOptions": [{ "readonly": "true" }],
"elementOptions": [],
"transformers": []
}
]
@@ -968,12 +1001,69 @@
"description": [
{
"language_code": "en_us",
"string": "The port number of the network node. Uneditable."
"string": "The port number of the network node."
}
]
},
{
"function": "devSSID",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [],
"transformers": []
}
]
},
"default_value": "",
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "SSID"
}
],
"description": [
{
"language_code": "en_us",
"string": "The network SSID."
}
]
},
{
"function": "devSite",
"type": {
"dataType": "string",
"elements": [
{
"elementType": "input",
"elementOptions": [],
"transformers": []
}
]
},
"default_value": "",
"options": [],
"localized": ["name", "description"],
"name": [
{
"language_code": "en_us",
"string": "Network Site"
}
],
"description": [
{
"language_code": "en_us",
"string": "The network site."
}
]
},
{
"function": "devIcon",
"events": ["copy_icons", "add_icon"],
"type": {
"dataType": "string",
"elements": [
@@ -1013,7 +1103,7 @@
"name": [
{
"language_code": "en_us",
"string": "Device Icon"
"string": "Icon"
}
],
"description": [

627
front/plugins/omada_sdn_imp/omada_sdn.py
View File

@@ -1,6 +1,6 @@
#!/usr/bin/env python
__author__ = "ffsb"
__version__ = "0.1" #initial
__version__ = "0.1" # initial
__version__ = "0.2" # added logic to retry omada api call once as it seems to sometimes fail for some reasons, and error handling logic...
__version__ = "0.3" # split devices API calls to allow multithreading but had to stop due to concurency issues.
__version__ = "0.6" # found issue with multithreading - my omada calls redirect stdout which gets clubbered by normal stdout... not sure how to fix for now...
@@ -8,11 +8,12 @@ __version__ = "0.7" # avoid updating omada sdn client name when it is the MAC,
__version__ = "1.0" # fixed the timzone mylog issue by resetting the tz value at the begining of the script... I suspect it doesn't inherit the tz from the main.
__version__ = "1.1" # added logic to handle gracefully a failure of omada devices so it won't try to populate uplinks on non-existent switches and AP.
__version__ = "1.2" # finally got multiprocessing to work to parse devices AND to update names! yeah!
__version__ = "1.3" # fix detection of the default gateway IP address that would pick loopback address instead of the actual gateway.
# query OMADA SDN to populate NetAlertX witch omada switches, access points, clients.
# try to identify and populate their connections by switch/accesspoints and ports/SSID
# try to differentiate root bridges from accessory
# try to differentiate root bridges from accessory
#
@@ -29,12 +30,13 @@ import importlib.util
import time
import io
import re
#import concurrent.futures
# import concurrent.futures
import subprocess
import multiprocessing
#import netifaces
# import netifaces
# Define the installation path and extend the system path for plugin imports
INSTALL_PATH = "/app"
@@ -44,85 +46,99 @@ from plugin_helper import Plugin_Object, Plugin_Objects, decodeBase64
from plugin_utils import get_plugins_configs
from logger import mylog
from const import pluginsPath, fullDbPath
from helper import timeNowTZ, get_setting_value
from helper import timeNowTZ, get_setting_value
from notification import write_notification
from pytz import timezone
import conf
conf.tz = timezone(get_setting_value('TIMEZONE'))
conf.tz = timezone(get_setting_value("TIMEZONE"))
PARALLELISM = 4
# Define the current path and log file paths
CUR_PATH = str(pathlib.Path(__file__).parent.resolve())
LOG_FILE = os.path.join(CUR_PATH, 'script.log')
RESULT_FILE = os.path.join(CUR_PATH, 'last_result.log')
OMADA_API_RETURN_FILE = os.path.join(CUR_PATH, 'omada_api_return')
LOG_FILE = os.path.join(CUR_PATH, "script.log")
RESULT_FILE = os.path.join(CUR_PATH, "last_result.log")
OMADA_API_RETURN_FILE = os.path.join(CUR_PATH, "omada_api_return")
# Initialize the Plugin obj output file
plugin_objects = Plugin_Objects(RESULT_FILE)
#
# sample target output:
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID, 5 TYPE
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', '17', '40-AE-30-A5-A7-50, 'Switch']"
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID, 5 TYPE
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', '17', '40-AE-30-A5-A7-50, 'Switch']"
# Constants for array indices
MAC, IP, NAME, SWITCH_AP, PORT_SSID, TYPE = range(6)
# sample omada devices input format:
#
#
# 0.MAC 1.IP 2.type 3.status 4.name 5.model
#40-AE-30-A5-A7-50 192.168.0.11 ap CONNECTED office_Access_point EAP773(US) v1.0
#B0-95-75-46-0C-39 192.168.0.4 switch CONNECTED pantry12 T1600G-52PS v4.0
dMAC, dIP, dTYPE, dSTATUS, dNAME, dMODEL = range(6)
# 40-AE-30-A5-A7-50 192.168.0.11 ap CONNECTED office_Access_point EAP773(US) v1.0
# B0-95-75-46-0C-39 192.168.0.4 switch CONNECTED pantry12 T1600G-52PS v4.0
dMAC, dIP, dTYPE, dSTATUS, dNAME, dMODEL = range(6)
# sample omada clients input format:
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID,
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', 'myssid_name2', '(office_Access_point)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-01', '192.168.0.153', 'frontyard_ESP_29E753', 'pantry12', '(48)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-02', '192.168.0.1', 'bastion', 'office24', '(23)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-03', '192.168.0.226', 'brick', 'myssid_name3', '(office_Access_point)']"
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID,
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', 'myssid_name2', '(office_Access_point)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-01', '192.168.0.153', 'frontyard_ESP_29E753', 'pantry12', '(48)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-02', '192.168.0.1', 'bastion', 'office24', '(23)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-03', '192.168.0.226', 'brick', 'myssid_name3', '(office_Access_point)']"
cMAC, cIP, cNAME, cSWITCH_AP, cPORT_SSID = range(5)
OMDLOGLEVEL = 'debug'
pluginName = 'OMDSDN'
OMDLOGLEVEL = "debug"
pluginName = "OMDSDN"
#
# translate MAC address from standard ieee model to ietf draft
# AA-BB-CC-DD-EE-FF to aa:bb:cc:dd:ee:ff
# tplink adheres to ieee, Nax adheres to ietf
def ieee2ietf_mac_formater(inputmac):
return(inputmac.lower().replace('-',':'))
return inputmac.lower().replace("-", ":")
def ietf2ieee_mac_formater(inputmac):
return(inputmac.upper().replace(':','-'))
if not inputmac or not isinstance(inputmac, str):
mylog(
"minimal",
[
f"[{pluginName}] ietf2ieee_mac_formater ERROR: inputmac is not a string: {inputmac}"
],
)
return None
return inputmac.upper().replace(":", "-")
def get_mac_from_IP(target_IP):
from scapy.all import ARP, Ether, srp
try:
arp_request = ARP(pdst=target_IP)
ether = Ether(dst="ff:ff:ff:ff:ff:ff")
packet = ether/arp_request
packet = ether / arp_request
result = srp(packet, timeout=3, verbose=0)[0]
if result:
return result[0][1].hwsrc
else:
return None
except Exception as e:
mylog('minimal', [f'[{pluginName}] get_mac_from_IP ERROR:{e}'])
mylog("minimal", [f"[{pluginName}] get_mac_from_IP ERROR:{e}"])
return None
#
# wrapper to call the omada python library's own wrapper
# it returns the output as a multiline python string
# wrapper to call the omada python library's own wrapper
# it returns the output as a multiline python string
#
def callomada(myargs):
arguments=" ".join(myargs)
mylog('verbose', [f'[{pluginName}] callomada:{arguments}'])
from tplink_omada_client.cli import main as omada
arguments = " ".join(myargs)
mylog("verbose", [f"[{pluginName}] callomada:{arguments}"])
from tplink_omada_client.cli import main as omada
from contextlib import redirect_stdout
omada_output = ''
omada_output = ""
retries = 2
while omada_output == '' and retries > 1:
while omada_output == "" and retries > 1:
retries = retries - 1
try:
mf = io.StringIO()
@@ -130,47 +146,73 @@ def callomada(myargs):
bar = omada(myargs)
omada_output = mf.getvalue()
except Exception as e:
mylog('minimal', [f'[{pluginName}] ERROR WHILE CALLING callomada:{arguments}\n {mf}'])
omada_output= ''
return(omada_output)
mylog(
"minimal",
[f"[{pluginName}] ERROR WHILE CALLING callomada:{arguments}\n {mf}"],
)
omada_output = ""
return omada_output
#
# extract all the mac addresses from a multilines text...
# return a list of MAC as 'string'
# return a list of MAC as 'string'
#
def extract_mac_addresses(text):
mac_pattern = r"([0-9A-Fa-f]{2}[:-][0-9A-Fa-f]{2}[:-][0-9A-Fa-f]{2}[:-][0-9A-Fa-f]{2}[:-][0-9A-Fa-f]{2}[:-][0-9A-Fa-f]{2})"
mac_addresses = re.findall(mac_pattern, text)
return ["".join(parts) for parts in mac_addresses]
def find_default_gateway_ip ():
#import netifaces
#gw = netifaces.gateways()
#return(gw['default'][netifaces.AF_INET][0])
def find_default_gateway_ip():
# Get the routing table
from scapy.all import conf, Route, sr1, IP, ICMP
default_route = conf.route.route("0.0.0.0")
return default_route[2] if default_route[2] else None
routing_table = conf.route.routes
for route in routing_table:
# Each route is a tuple: (destination, netmask, gateway, iface, output_ip, metric)
destination, netmask, gateway, iface, output_ip, metric = route
# Look for the default route (destination and netmask are both 0.0.0.0)
if destination == 0 and netmask == 0 and gateway != "0.0.0.0":
mylog("verbose", [f"[DEBUG] Default Gateway IP: {gateway}"])
return gateway # Found the default gateway
return None # Default gateway not found
#return('192.168.0.1')
def add_uplink (uplink_mac, switch_mac, device_data_bymac, sadevices_linksbymac,port_byswitchmac_byclientmac):
#mylog(OMDLOGLEVEL, [f'[{pluginName}] trying to add uplink="{uplink_mac}" to switch="{switch_mac}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}] before adding:"{device_data_bymac[switch_mac]}"'])
if device_data_bymac[switch_mac][SWITCH_AP] == 'null':
def add_uplink(
uplink_mac,
switch_mac,
device_data_bymac,
sadevices_linksbymac,
port_byswitchmac_byclientmac,
):
# mylog(OMDLOGLEVEL, [f'[{pluginName}] trying to add uplink="{uplink_mac}" to switch="{switch_mac}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}] before adding:"{device_data_bymac[switch_mac]}"'])
if device_data_bymac[switch_mac][SWITCH_AP] == "null":
device_data_bymac[switch_mac][SWITCH_AP] = uplink_mac
if device_data_bymac[switch_mac][TYPE] == 'Switch' and device_data_bymac[uplink_mac][TYPE] == 'Switch':
port_to_uplink = port_byswitchmac_byclientmac[switch_mac][uplink_mac]
#find_port_of_uplink_switch(switch_mac, uplink_mac)
if (
device_data_bymac[switch_mac][TYPE] == "Switch"
and device_data_bymac[uplink_mac][TYPE] == "Switch"
):
port_to_uplink = port_byswitchmac_byclientmac[switch_mac][uplink_mac]
# find_port_of_uplink_switch(switch_mac, uplink_mac)
else:
port_to_uplink=device_data_bymac[uplink_mac][PORT_SSID]
port_to_uplink = device_data_bymac[uplink_mac][PORT_SSID]
device_data_bymac[switch_mac][PORT_SSID] = port_to_uplink
# mylog(OMDLOGLEVEL, [f'[{pluginName}] after adding:"{device_data_bymac[switch_mac]}"'])
for link in sadevices_linksbymac[switch_mac]:
if device_data_bymac[link][SWITCH_AP] == 'null' and device_data_bymac[switch_mac][TYPE] == 'Switch':
add_uplink(switch_mac, link, device_data_bymac, sadevices_linksbymac,port_byswitchmac_byclientmac)
# mylog(OMDLOGLEVEL, [f'[{pluginName}] after adding:"{device_data_bymac[switch_mac]}"'])
for link in sadevices_linksbymac[switch_mac]:
if (
device_data_bymac[link][SWITCH_AP] == "null"
and device_data_bymac[switch_mac][TYPE] == "Switch"
):
add_uplink(
switch_mac,
link,
device_data_bymac,
sadevices_linksbymac,
port_byswitchmac_byclientmac,
)
# ----------------------------------------------
@@ -178,51 +220,69 @@ def add_uplink (uplink_mac, switch_mac, device_data_bymac, sadevices_linksbymac,
def main():
start_time = time.time()
mylog('verbose', [f'[{pluginName}] starting execution'])
mylog("verbose", [f"[{pluginName}] starting execution"])
from database import DB
from device import Device_obj
db = DB() # instance of class DB
db.open()
# Create a Device_obj instance
# Create a Device_obj instance
device_handler = Device_obj(db)
# Retrieve configuration settings
# these should be self-explanatory
omada_sites = []
omada_username = get_setting_value('OMDSDN_username')
omada_password = get_setting_value('OMDSDN_password')
omada_sites = get_setting_value('OMDSDN_sites')
omada_sites = []
omada_username = get_setting_value("OMDSDN_username")
omada_password = get_setting_value("OMDSDN_password")
omada_sites = get_setting_value("OMDSDN_sites")
omada_site = omada_sites[0]
omada_url = get_setting_value('OMDSDN_url')
omada_login = callomada(['-t','myomada','target','--url',omada_url,'--user',omada_username,
'--password',omada_password,'--site',omada_site,'--set-default'])
mylog('verbose', [f'[{pluginName}] login to omada result is: {omada_login}'])
clients_list = callomada(['-t','myomada','clients'])
mylog('verbose', [f'[{pluginName}] clients found:"{clients_list.count("\n")}"\n{clients_list}'])
omada_url = get_setting_value("OMDSDN_url")
switches_and_aps = callomada(['-t','myomada','devices'])
mylog('verbose', [f'[{pluginName}] omada devices (switches, access points) found:"{switches_and_aps.count("\n")}" \n {switches_and_aps}'])
omada_login = callomada(
[
"-t",
"myomada",
"target",
"--url",
omada_url,
"--user",
omada_username,
"--password",
omada_password,
"--site",
omada_site,
"--set-default",
]
)
mylog("verbose", [f"[{pluginName}] login to omada result is: {omada_login}"])
#some_setting = get_setting_value('OMDSDN_url')
#mylog(OMDLOGLEVEL, [f'[{pluginName}] some_setting value {some_setting}'])
mylog(OMDLOGLEVEL, [f'[{pluginName}] ffsb'])
clients_list = callomada(["-t", "myomada", "clients"])
mylog(
"verbose",
[f'[{pluginName}] clients found:"{clients_list.count("\n")}"\n{clients_list}'],
)
switches_and_aps = callomada(["-t", "myomada", "devices"])
mylog(
"verbose",
[
f'[{pluginName}] omada devices (switches, access points) found:"{switches_and_aps.count("\n")}" \n {switches_and_aps}'
],
)
# some_setting = get_setting_value('OMDSDN_url')
# mylog(OMDLOGLEVEL, [f'[{pluginName}] some_setting value {some_setting}'])
mylog(OMDLOGLEVEL, [f"[{pluginName}] ffsb"])
# retrieve data
device_data = get_device_data(clients_list, switches_and_aps, device_handler)
# Process the data into native application tables
mylog('verbose', [f'[{pluginName}] New entries to create: "{len(device_data)}"'])
mylog("verbose", [f'[{pluginName}] New entries to create: "{len(device_data)}"'])
if len(device_data) > 0:
# insert devices into the lats_result.log
# make sure the below mapping is mapped in config.json, for example:
#"database_column_definitions": [
# insert devices into the lats_result.log
# make sure the below mapping is mapped in config.json, for example:
# "database_column_definitions": [
# {
# "column": "Object_PrimaryID", <--------- the value I save into primaryId
# "mapped_to_column": "cur_MAC", <--------- gets unserted into the CurrentScan DB table column cur_MAC
@@ -230,91 +290,115 @@ def main():
# figure a way to run my udpate script delayed
for device in device_data:
mylog(OMDLOGLEVEL, [f'[{pluginName}] main parsing device: "{device}"'])
myport = device[PORT_SSID] if device[PORT_SSID].isdigit() else ''
myssid = device[PORT_SSID] if not device[PORT_SSID].isdigit() else ''
ParentNetworkNode = ieee2ietf_mac_formater(device[SWITCH_AP]) if device[SWITCH_AP] != 'Internet' else 'Internet'
mymac = ieee2ietf_mac_formater(device[MAC])
plugin_objects.add_object(
primaryId = mymac, # MAC
secondaryId = device[IP], # IP
watched1 = device[NAME], # NAME/HOSTNAME
watched2 = ParentNetworkNode, # PARENT NETWORK NODE MAC
watched3 = myport, # PORT
watched4 = myssid, # SSID
extra = device[TYPE],
#omada_site, # SITENAME (cur_NetworkSite) or VENDOR (cur_Vendor) (PICK one and adjust config.json -> "column": "Extra")
foreignKey = device[MAC].lower().replace('-',':')) # usually MAC
mylog('verbose', [f'[{pluginName}] New entries: "{mymac:<18}, {device[IP]:<16}, {device[NAME]:<63}, {ParentNetworkNode:<18}, {myport:<4}, {myssid:<32}, {device[TYPE]}"'])
mylog('verbose', [f'[{pluginName}] New entries: "{len(device_data)}"'])
mylog(OMDLOGLEVEL, [f'[{pluginName}] main parsing device: "{device}"'])
myport = device[PORT_SSID] if device[PORT_SSID].isdigit() else ""
myssid = device[PORT_SSID] if not device[PORT_SSID].isdigit() else ""
ParentNetworkNode = (
ieee2ietf_mac_formater(device[SWITCH_AP])
if device[SWITCH_AP] != "Internet"
else "Internet"
)
mymac = ieee2ietf_mac_formater(device[MAC])
plugin_objects.add_object(
primaryId=mymac, # MAC
secondaryId=device[IP], # IP
watched1=device[NAME], # NAME/HOSTNAME
watched2=ParentNetworkNode, # PARENT NETWORK NODE MAC
watched3=myport, # PORT
watched4=myssid, # SSID
extra=device[TYPE],
# omada_site, # SITENAME (cur_NetworkSite) or VENDOR (cur_Vendor) (PICK one and adjust config.json -> "column": "Extra")
foreignKey=device[MAC].lower().replace("-", ":"),
) # usually MAC
mylog(
"verbose",
[
f'[{pluginName}] New entries: "{mymac:<18}, {device[IP]:<16}, {device[NAME]:<63}, {ParentNetworkNode:<18}, {myport:<4}, {myssid:<32}, {device[TYPE]}"'
],
)
mylog("verbose", [f'[{pluginName}] New entries: "{len(device_data)}"'])
# log result
plugin_objects.write_result_file()
#mylog(OMDLOGLEVEL, [f'[{pluginName}] TEST name from MAC: {device_handler.getValueWithMac('devName','00:e2:59:00:a0:8e')}'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}] TEST MAC from IP: {get_mac_from_IP('192.168.0.1')} also {ietf2ieee_mac_formater(get_mac_from_IP('192.168.0.1'))}'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}] TEST name from MAC: {device_handler.getValueWithMac('devName','00:e2:59:00:a0:8e')}'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}] TEST MAC from IP: {get_mac_from_IP('192.168.0.1')} also {ietf2ieee_mac_formater(get_mac_from_IP('192.168.0.1'))}'])
end_time = time.time()
mylog('verbose', [f'[{pluginName}] execution completed in {end_time - start_time:.2f} seconds'])
mylog(
"verbose",
[f"[{pluginName}] execution completed in {end_time - start_time:.2f} seconds"],
)
return 0
def get_omada_devices_details(msadevice_data):
mthisswitch = msadevice_data[dMAC]
mtype = msadevice_data[dTYPE]
mswitch_detail = ''
mswitch_dump = ''
if mtype == 'ap':
mswitch_detail = callomada(['access-point', mthisswitch])
elif mtype == 'switch':
mswitch_detail = callomada(['switch', mthisswitch])
mswitch_dump = callomada(['-t','myomada','switch','-d',mthisswitch])
mswitch_detail = ""
mswitch_dump = ""
if mtype == "ap":
mswitch_detail = callomada(["access-point", mthisswitch])
elif mtype == "switch":
mswitch_detail = callomada(["switch", mthisswitch])
mswitch_dump = callomada(["-t", "myomada", "switch", "-d", mthisswitch])
else:
mswitch_detail = ''
nswitch_dump = ''
mswitch_detail = ""
nswitch_dump = ""
return mswitch_detail, mswitch_dump
def get_omada_devices_details_parallel(msadevice_data):
mthisswitch = msadevice_data[dMAC]
mtype = msadevice_data[dTYPE]
mswitch_detail = ''
mswitch_dump = ''
if mtype == 'ap':
mswitch_detail = subprocess.run('omada access-point '+mthisswitch, capture_output=True, text=True, shell=True).stdout
elif mtype == 'switch':
mswitch_detail = subprocess.run('omada switch '+mthisswitch, capture_output=True, text=True, shell=True).stdout
mswitch_dump = subprocess.run('omada access-point '+mthisswitch, capture_output=True, text=True, shell=True).stdout
mswitch_detail = ""
mswitch_dump = ""
if mtype == "ap":
mswitch_detail = subprocess.run(
"omada access-point " + mthisswitch,
capture_output=True,
text=True,
shell=True,
).stdout
elif mtype == "switch":
mswitch_detail = subprocess.run(
"omada switch " + mthisswitch, capture_output=True, text=True, shell=True
).stdout
mswitch_dump = subprocess.run(
"omada access-point " + mthisswitch,
capture_output=True,
text=True,
shell=True,
).stdout
else:
mswitch_detail = ''
mswitch_dump = ''
mswitch_detail = ""
mswitch_dump = ""
return mthisswitch, mswitch_detail, mswitch_dump
# ----------------------------------------------
# retrieve data
def get_device_data(omada_clients_output,switches_and_aps,device_handler):
def get_device_data(omada_clients_output, switches_and_aps, device_handler):
# sample omada devices input format:
# 0.MAC 1.IP 2.type 3.status 4.name 5.model
#40-AE-30-A5-A7-50 192.168.0.11 ap CONNECTED office_Access_point EAP773(US) v1.0
#B0-95-75-46-0C-39 192.168.0.4 switch CONNECTED pantry12 T1600G-52PS v4.0
# 40-AE-30-A5-A7-50 192.168.0.11 ap CONNECTED office_Access_point EAP773(US) v1.0
# B0-95-75-46-0C-39 192.168.0.4 switch CONNECTED pantry12 T1600G-52PS v4.0
#
# sample target output:
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID, 5 TYPE
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', '17', '40-AE-30-A5-A7-50, 'Switch']"
#constants
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID, 5 TYPE
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', '17', '40-AE-30-A5-A7-50, 'Switch']"
# constants
sadevices_macbyname = {}
sadevices_macbymac = {}
sadevices_linksbymac = {}
port_byswitchmac_byclientmac = {}
device_data_bymac = {}
device_data_mac_byip = {}
omada_force_overwrite = get_setting_value('OMDSDN_force_overwrite')
omada_force_overwrite = get_setting_value("OMDSDN_force_overwrite")
switch_details = {}
switch_dumps = {}
'''
"""
command = 'which omada'
def run_command(command, index):
result = subprocess.run(command, capture_output=True, text=True, shell=True)
@@ -322,108 +406,125 @@ def get_device_data(omada_clients_output,switches_and_aps,device_handler):
myindex, command_output= run_command(command, 2)
mylog('verbose', [f'[{pluginName}] command={command} index={myindex} results={command_output}'])
'''
"""
sadevices = switches_and_aps.splitlines()
mylog(OMDLOGLEVEL, [f'[{pluginName}] switches_and_aps rows: "{len(sadevices)}"'])
with multiprocessing.Pool(processes = PARALLELISM) as mypool:
oresults = mypool.map(get_omada_devices_details_parallel, [sadevice.split() for sadevice in sadevices])
with multiprocessing.Pool(processes=PARALLELISM) as mypool:
oresults = mypool.map(
get_omada_devices_details_parallel,
[sadevice.split() for sadevice in sadevices],
)
for thisswitch, details, dump in oresults:
switch_details[thisswitch] = details
switch_dumps[thisswitch] = dump
mylog(OMDLOGLEVEL, [f'[{pluginName}] switch={thisswitch} details={details}'])
'''
mylog(OMDLOGLEVEL, [f"[{pluginName}] switch={thisswitch} details={details}"])
"""
for sadevice in sadevices:
sadevice_data = sadevice.split()
thisswitch = sadevice_data[dMAC]
thistype = sadevice_data[dTYPE]
switch_details[thisswitch], switch_dumps[thisswitch] = get_omada_devices_details(sadevice_data)
'''
mylog('verbose', [f'[{pluginName}] switches details collected "{len(switch_details)}"'])
mylog('verbose', [f'[{pluginName}] dump details collected "{len(switch_details)}"'])
"""
mylog(
"verbose",
[f'[{pluginName}] switches details collected "{len(switch_details)}"'],
)
mylog("verbose", [f'[{pluginName}] dump details collected "{len(switch_details)}"'])
for sadevice in sadevices:
sadevice_data = sadevice.split()
thisswitch = sadevice_data[dMAC]
sadevices_macbyname[sadevice_data[4]] = thisswitch
if sadevice_data[dTYPE] == 'ap':
sadevice_type = 'AP'
#sadevice_details = callomada(['access-point', thisswitch])
if sadevice_data[dTYPE] == "ap":
sadevice_type = "AP"
# sadevice_details = callomada(['access-point', thisswitch])
sadevice_details = switch_details[thisswitch]
if sadevice_details == '':
if sadevice_details == "":
sadevice_links = [thisswitch]
else:
sadevice_links = extract_mac_addresses(sadevice_details)
sadevices_linksbymac[thisswitch] = sadevice_links[1:]
#mylog(OMDLOGLEVEL, [f'[{pluginName}]adding switch details: "{sadevice_details}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}]links are: "{sadevice_links}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}]linksbymac are: "{sadevices_linksbymac[thisswitch]}"'])
elif sadevice_data[dTYPE] == 'switch':
sadevice_type = 'Switch'
#sadevice_details=callomada(['switch', thisswitch])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]adding switch details: "{sadevice_details}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]links are: "{sadevice_links}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]linksbymac are: "{sadevices_linksbymac[thisswitch]}"'])
elif sadevice_data[dTYPE] == "switch":
sadevice_type = "Switch"
# sadevice_details=callomada(['switch', thisswitch])
sadevice_details = switch_details[thisswitch]
if sadevice_details == '':
if sadevice_details == "":
sadevice_links = [thisswitch]
else:
sadevice_links=extract_mac_addresses(sadevice_details)
sadevice_links = extract_mac_addresses(sadevice_details)
sadevices_linksbymac[thisswitch] = sadevice_links[1:]
# recovering the list of switches connected to sadevice switch and on which port...
#switchdump = callomada(['-t','myomada','switch','-d',thisswitch])
# switchdump = callomada(['-t','myomada','switch','-d',thisswitch])
switchdump = switch_dumps[thisswitch]
mylog(OMDLOGLEVEL, [f'[{pluginName}] switchdump: {switchdump}'])
port_byswitchmac_byclientmac[thisswitch] = {}
mylog(OMDLOGLEVEL, [f"[{pluginName}] switchdump: {switchdump}"])
port_byswitchmac_byclientmac[thisswitch] = {}
for link in sadevices_linksbymac[thisswitch]:
port_pattern = r"(?:{[^}]*\"port\"\: )([0-9]+)(?=[^}]*"+re.escape(link)+r")"
myport = re.findall(port_pattern, switchdump,re.DOTALL)
#mylog(OMDLOGLEVEL, [f'[{pluginName}] switchdump: link={link} myport:{myport}'])
port_byswitchmac_byclientmac[thisswitch][link] = myport[0] if myport else ''
#mylog(OMDLOGLEVEL, [f'[{pluginName}]links are: "{sadevice_links}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}]linksbymac are: "{sadevices_linksbymac[thisswitch]}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}]ports of each links are: "{port_byswitchmac_byclientmac[thisswitch]}"'])
#mylog(OMDLOGLEVEL, [f'[{pluginName}]adding switch details: "{sadevice_details}"'])
port_pattern = (
r"(?:{[^}]*\"port\"\: )([0-9]+)(?=[^}]*" + re.escape(link) + r")"
)
myport = re.findall(port_pattern, switchdump, re.DOTALL)
# mylog(OMDLOGLEVEL, [f'[{pluginName}] switchdump: link={link} myport:{myport}'])
port_byswitchmac_byclientmac[thisswitch][link] = (
myport[0] if myport else ""
)
# mylog(OMDLOGLEVEL, [f'[{pluginName}]links are: "{sadevice_links}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]linksbymac are: "{sadevices_linksbymac[thisswitch]}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]ports of each links are: "{port_byswitchmac_byclientmac[thisswitch]}"'])
# mylog(OMDLOGLEVEL, [f'[{pluginName}]adding switch details: "{sadevice_details}"'])
else:
sadevice_type = 'null'
sadevice_details='null'
device_data_bymac[thisswitch] = [thisswitch, sadevice_data[dIP], sadevice_data[dNAME], 'null', 'null',sadevice_type]
sadevice_type = "null"
sadevice_details = "null"
device_data_bymac[thisswitch] = [
thisswitch,
sadevice_data[dIP],
sadevice_data[dNAME],
"null",
"null",
sadevice_type,
]
device_data_mac_byip[sadevice_data[dIP]] = thisswitch
foo=[thisswitch, sadevice_data[1], sadevice_data[4], 'null', 'null']
foo = [thisswitch, sadevice_data[1], sadevice_data[4], "null", "null"]
mylog(OMDLOGLEVEL, [f'[{pluginName}]adding switch: "{foo}"'])
# sadevices_macbymac[thisswitch] = thisswitch
mylog(OMDLOGLEVEL, [f'[{pluginName}] switch_macbyname: "{sadevices_macbyname}"'])
mylog(OMDLOGLEVEL, [f'[{pluginName}] switches: "{device_data_bymac}"'])
# do some processing, call exteranl APIs, and return a device list
# ...
# sample omada clients input format:
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID,
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', 'myssid_name2', '(office_Access_point)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-01', '192.168.0.153', 'frontyard_ESP_29E753', 'pantry12', '(48)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-02', '192.168.0.1', 'bastion', 'office24', '(23)']"
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-03', '192.168.0.226', 'brick', 'myssid_name3', '(office_Access_point)']"
# 0 MAC, 1 IP, 2 Name, 3 switch/AP, 4 port/SSID,
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', '1A-2B-3C-4D-5E-6F', 'myssid_name2', '(office_Access_point)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-01', '192.168.0.153', 'frontyard_ESP_29E753', 'pantry12', '(48)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-02', '192.168.0.1', 'bastion', 'office24', '(23)']"
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-03', '192.168.0.226', 'brick', 'myssid_name3', '(office_Access_point)']"
# sample target output:
# 0 MAC, 1 IP, 2 Name, 3 MAC of switch/AP, 4 port/SSID, 5 TYPE
#17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', 'brick', 'office_Access_point','myssid_name2', , 'Switch']"
# 0 MAC, 1 IP, 2 Name, 3 MAC of switch/AP, 4 port/SSID, 5 TYPE
# 17:27:10 [<unique_prefix>] token: "['1A-2B-3C-4D-5E-6F', '192.168.0.217', 'brick', 'office_Access_point','myssid_name2', , 'Switch']"
odevices = omada_clients_output.splitlines()
mylog(OMDLOGLEVEL, [f'[{pluginName}] omada_clients_outputs rows: "{len(odevices)}"'])
mylog(
OMDLOGLEVEL, [f'[{pluginName}] omada_clients_outputs rows: "{len(odevices)}"']
)
omada_clients_to_rename = []
for odevice in odevices:
odevice_data = odevice.split()
odevice_data_reordered = [ MAC, IP, NAME, SWITCH_AP, PORT_SSID, TYPE]
odevice_data_reordered[MAC]=odevice_data[cMAC]
odevice_data_reordered[IP]=odevice_data[cIP]
real_naxname = device_handler.getValueWithMac('devName',ieee2ietf_mac_formater(odevice_data[cMAC]))
odevice_data_reordered = [MAC, IP, NAME, SWITCH_AP, PORT_SSID, TYPE]
odevice_data_reordered[MAC] = odevice_data[cMAC]
odevice_data_reordered[IP] = odevice_data[cIP]
real_naxname = device_handler.getValueWithMac(
"devName", ieee2ietf_mac_formater(odevice_data[cMAC])
)
#
# if the name stored in Nax for a device is empty or the MAC addres or has some parenthhesis or is the same as in omada
@@ -432,86 +533,122 @@ def get_device_data(omada_clients_output,switches_and_aps,device_handler):
naxname = real_naxname
if real_naxname != None:
if '(' in real_naxname:
# removing parenthesis and domains from the name
naxname = real_naxname.split('(')[0]
if naxname != None and '.' in naxname:
naxname = naxname.split('.')[0]
if naxname in ( None, 'null', '' ):
naxname = odevice_data[cNAME] if odevice_data[cNAME] != '' else odevice_data[cMAC]
if "(" in real_naxname:
# removing parenthesis and domains from the name
naxname = real_naxname.split("(")[0]
if naxname != None and "." in naxname:
naxname = naxname.split(".")[0]
if naxname in (None, "null", ""):
naxname = (
odevice_data[cNAME] if odevice_data[cNAME] != "" else odevice_data[cMAC]
)
naxname = naxname.strip()
mylog('debug', [f'[{pluginName}] TEST name from MAC: {naxname}'])
if odevice_data[cNAME] in ('null', ''):
mylog('verbose', [f'[{pluginName}] updating omada server because odevice_data is: {odevice_data[cNAME]} and naxname is: "{naxname}"'])
omada_clients_to_rename.append(['set-client-name',odevice_data[cMAC], naxname])
#callomada(['set-client-name', odevice_data[cMAC], naxname])
mylog("debug", [f"[{pluginName}] TEST name from MAC: {naxname}"])
if odevice_data[cNAME] in ("null", ""):
mylog(
"verbose",
[
f'[{pluginName}] updating omada server because odevice_data is: {odevice_data[cNAME]} and naxname is: "{naxname}"'
],
)
omada_clients_to_rename.append(
["set-client-name", odevice_data[cMAC], naxname]
)
# callomada(['set-client-name', odevice_data[cMAC], naxname])
odevice_data_reordered[NAME] = naxname
elif odevice_data[cNAME] == odevice_data[cMAC] and ieee2ietf_mac_formater(naxname) != ieee2ietf_mac_formater(odevice_data[cNAME]) :
mylog('verbose', [f'[{pluginName}] updating omada server because odevice_data is: "{odevice_data[cNAME]} and naxname is: "{naxname}"'])
omada_clients_to_rename.append(['set-client-name',odevice_data[cMAC], naxname])
#callomada(['set-client-name', odevice_data[cMAC], naxname])
elif odevice_data[cNAME] == odevice_data[cMAC] and ieee2ietf_mac_formater(
naxname
) != ieee2ietf_mac_formater(odevice_data[cNAME]):
mylog(
"verbose",
[
f'[{pluginName}] updating omada server because odevice_data is: "{odevice_data[cNAME]} and naxname is: "{naxname}"'
],
)
omada_clients_to_rename.append(
["set-client-name", odevice_data[cMAC], naxname]
)
# callomada(['set-client-name', odevice_data[cMAC], naxname])
odevice_data_reordered[NAME] = naxname
else:
if omada_force_overwrite and naxname != odevice_data[cNAME] :
mylog('verbose', [f'[{pluginName}] updating omada server because odevice_data is: "{odevice_data[cNAME]} and naxname is: "{naxname}"'])
omada_clients_to_rename.append(['set-client-name',odevice_data[cMAC], naxname])
#callomada(['set-client-name', odevice_data[cMAC], naxname])
if omada_force_overwrite and naxname != odevice_data[cNAME]:
mylog(
"verbose",
[
f'[{pluginName}] updating omada server because odevice_data is: "{odevice_data[cNAME]} and naxname is: "{naxname}"'
],
)
omada_clients_to_rename.append(
["set-client-name", odevice_data[cMAC], naxname]
)
# callomada(['set-client-name', odevice_data[cMAC], naxname])
odevice_data_reordered[NAME] = naxname
mightbeport = odevice_data[cPORT_SSID].lstrip('(')
mightbeport = mightbeport.rstrip(')')
mightbeport = odevice_data[cPORT_SSID].lstrip("(")
mightbeport = mightbeport.rstrip(")")
if mightbeport.isdigit():
odevice_data_reordered[SWITCH_AP] = odevice_data[cSWITCH_AP]
odevice_data_reordered[PORT_SSID] = mightbeport
else:
odevice_data_reordered[SWITCH_AP] = mightbeport
odevice_data_reordered[PORT_SSID] = odevice_data[cSWITCH_AP]
# replacing the switch name with its MAC...
try:
mightbemac = sadevices_macbyname[odevice_data_reordered[SWITCH_AP]]
odevice_data_reordered[SWITCH_AP] = mightbemac
except KeyError:
mylog(OMDLOGLEVEL, [f'[{pluginName}] could not find the mac adddress for: "{odevice_data_reordered[SWITCH_AP]}"'])
mylog(
OMDLOGLEVEL,
[
f'[{pluginName}] could not find the mac adddress for: "{odevice_data_reordered[SWITCH_AP]}"'
],
)
# adding the type
odevice_data_reordered[TYPE] = 'null'
device_data_bymac[odevice_data_reordered[MAC]] = odevice_data_reordered
odevice_data_reordered[TYPE] = "null"
device_data_bymac[odevice_data_reordered[MAC]] = odevice_data_reordered
device_data_mac_byip[odevice_data_reordered[IP]] = odevice_data_reordered[MAC]
mylog(OMDLOGLEVEL, [f'[{pluginName}] tokens: "{odevice_data}"'])
mylog(OMDLOGLEVEL, [f'[{pluginName}] tokens_reordered: "{odevice_data_reordered}"'])
mylog(
OMDLOGLEVEL,
[f'[{pluginName}] tokens_reordered: "{odevice_data_reordered}"'],
)
# RENAMING
#for omada_client_to_rename in omada_clients_to_rename:
# for omada_client_to_rename in omada_clients_to_rename:
# mylog('verbose', [f'[{pluginName}] calling omada: "{omada_client_to_rename}"'])
#callomada(omada_client_to_rename)
# callomada(omada_client_to_rename)
# populating the uplinks nodes of the omada switches and access points manually
# populating the uplinks nodes of the omada switches and access points manually
# since OMADA SDN makes is unreliable if the gateway is not their own tplink hardware...
#
with multiprocessing.Pool(processes = PARALLELISM) as mypool2:
#
with multiprocessing.Pool(processes=PARALLELISM) as mypool2:
oresults = mypool2.map(callomada, omada_clients_to_rename)
mylog(OMDLOGLEVEL, [f'[{pluginName}] results are: "{oresults}"'])
# step1 let's find the the default router
#
default_router_ip = find_default_gateway_ip()
#
default_router_ip = find_default_gateway_ip()
default_router_mac = ietf2ieee_mac_formater(get_mac_from_IP(default_router_ip))
device_data_bymac[default_router_mac][TYPE] = 'Firewall'
device_data_bymac[default_router_mac][TYPE] = "Firewall"
# step2 let's find the first switch and set the default router parent to internet
first_switch=device_data_bymac[default_router_mac][SWITCH_AP]
device_data_bymac[default_router_mac][SWITCH_AP] = 'Internet'
first_switch = device_data_bymac[default_router_mac][SWITCH_AP]
device_data_bymac[default_router_mac][SWITCH_AP] = "Internet"
# step3 let's set the switch connected to the default gateway uplink to the default gateway and hardcode port to 1 for now:
#device_data_bymac[first_switch][SWITCH_AP]=default_router_mac
#device_data_bymac[first_switch][SWITCH_AP][PORT_SSID] = '1'
# device_data_bymac[first_switch][SWITCH_AP]=default_router_mac
# device_data_bymac[first_switch][SWITCH_AP][PORT_SSID] = '1'
# step4, let's go recursively through switches other links to mark update their uplinks
# and pray it ends one day...
#
# and pray it ends one day...
#
if len(sadevices) > 0:
add_uplink(default_router_mac,first_switch, device_data_bymac,sadevices_linksbymac,port_byswitchmac_byclientmac)
add_uplink(
default_router_mac,
first_switch,
device_data_bymac,
sadevices_linksbymac,
port_byswitchmac_byclientmac,
)
return device_data_bymac.values()
if __name__ == '__main__':
if __name__ == "__main__":
main()

7
front/plugins/sync/sync.py
View File

@@ -236,8 +236,9 @@ def main():
# Prepare the insert statement
if new_devices:
columns = ', '.join(k for k in new_devices[0].keys() if k != 'rowid')
placeholders = ', '.join('?' for k in new_devices[0] if k != 'rowid')
# creating insert statement, removing 'rowid', 'devStatus' as handled on the target and devStatus is resolved on the fly
columns = ', '.join(k for k in new_devices[0].keys() if k not in ['rowid', 'devStatus'])
placeholders = ', '.join('?' for k in new_devices[0] if k not in ['rowid', 'devStatus'])
sql = f'INSERT INTO Devices ({columns}) VALUES ({placeholders})'
# Extract values for the new devices
@@ -254,8 +255,6 @@ def main():
mylog('verbose', [message])
write_notification(message, 'info', timeNowTZ())
# Commit and close the connection
conn.commit()