release version 1.4 (#21)

* chore: init working on history functions
rename classes

* refac: remove raw-result option. I cannot find a use ( I have found it later on )

* chore: add basic history directory checking

* chore: further expansion on the code. Now its christmas

* chore: rework some code and make a logging/history feature workable

---------

Co-authored-by: DaanSelen <dselen@systemec.nl>

.github/dependabot.yml

@@ -0,0 +1,11 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
+
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "weekly"

.gitignore

@@ -1,5 +1,6 @@
 *.conf
 venv
+books
 .vscode
 
 # Byte-compiled / optimized / DLL files

README.md

@@ -1,119 +1,282 @@
 # Meshbook
 
-A way to programmatically manage MeshCentral-managed machines, a bit like Ansible does.<br>
+[![CodeQL Advanced](https://github.com/DaanSelen/meshbook/actions/workflows/codeql.yaml/badge.svg)](https://github.com/DaanSelen/meshbook/actions/workflows/codeql.yaml)
-What problem does it solve? Well, what I wanted to be able to do is to automate system updates through [MeshCentral](https://github.com/ylianst/meshcentral).<br>
-And many people will be comfortable with YAML configurations! It's almost like JSON, but different!<br>
 
-# Quick-start:
+> \[!NOTE]
+> 💬 If you experience issues or have suggestions, [submit an issue](https://github.com/DaanSelen/meshbook/issues) — I'll respond ASAP!
 
-The quickest way to start is to grab a template from the templates folder in this repository.<br>
+---
-Make sure to correctly pass the MeshCentral websocket API as `wss://<MeshCentral-Host>/control.ashx`.<br>
-And make sure to fill in the credentails of an account which has remote commands permissions.<br>
-Then make a yaml with a target and some commands! See below examples as a guideline. And do not forget to look at the bottom's notice.<br>
-To install, follow the following commands:<br>
 
-```shell
+Meshbook is a tool to **programmatically manage MeshCentral-managed machines**, inspired by tools like [Ansible](https://github.com/ansible/ansible).
+
+## What problem does it solve?
+
+Meshbook is designed to:
+
+* Automate system updates or commands across multiple systems via [MeshCentral](https://github.com/Ylianst/MeshCentral), even behind third-party-managed firewalls.
+* Allow configuration using simple and readable **YAML files** (like Ansible playbooks).
+* Simplify the use of **group-based** or **tag-based** device targeting.
+
+---
+
+## 🏁 Quick Start
+
+### ✅ Prerequisites
+
+* Python 3.7+
+* Git
+* Access to a MeshCentral instance and credentials with:
+
+  * `Remote Commands`
+  * `Details`
+  * `Agent Console` permissions
+
+A service account with access to the relevant device groups is recommended.
+
+---
+
+### 🔧 Installation
+
+#### Linux
+
+```bash
+git clone https://github.com/daanselen/meshbook
 cd ./meshbook
 python3 -m venv ./venv
 source ./venv/bin/activate
-pip3 install -r requirements.txt
+pip install -r requirements.txt
-```
+cp ./templates/meshcentral.conf.template ./meshcentral.conf
-Then you can use meshbook, for example:
-```shell
-python3 meshbook.py -pb examples/ping.yaml
 ```
 
-# Example:
+#### Windows (PowerShell)
 
-For the example, I used the following yaml file:
+```powershell
+git clone https://github.com/daanselen/meshbook
+cd .\meshbook
+python -m venv .\venv
+.\venv\Scripts\activate
+pip install -r .\requirements.txt
+cp .\templates\meshcentral.conf.template .\meshcentral.conf
+```
+
+> 📌 Rename `meshcentral.conf.template` to `meshcentral.conf` and fill in your actual connection details.
+> The URL must start with `wss://<MeshCentral-Host>`.
+
+---
+
+## 🚀 Running Meshbook
+
+Once installed and configured, run a playbook like this:
+
+### Linux:
+
+```bash
+python3 meshbook.py -pb ./examples/echo_example.yaml
+```
+
+### Windows:
+
+```powershell
+.\venv\Scripts\python.exe .\meshbook.py -pb .\examples\echo_example.yaml
+```
+
+Use `--help` to explore available command-line options:
+
+```bash
+python3 meshbook.py --help
+```
+
+---
+
+## 🛠️ Creating Configurations
+
+Meshbook configurations are written in YAML. Below is an overview of supported fields.
+
+### ▶️ Group Targeting (Primary*)
 
 ```yaml
 ---
-name: Ping a single Point
+name: My Configuration
-company: Temp-Agents
+group: "Dev Machines"
+powershell: true
+variables:
+  - name: message
+    value: "Hello from Meshbook"
 tasks:
-  - name: Ping Cloudflare
+  - name: Echo a message
-    command: "ping 1.1.1.1 -c 4"
+    command: 'echo "{{ message }}"'
 ```
 
-The above group: `Temp-Agents` has four devices, of which one is offline.<br>
+* `group`: MeshCentral group (aka "mesh"). Quotation marks required for multi-word names.
-You can expand the command chain as follows:<br>
+* `powershell`: Set `true` for PowerShell commands on Windows clients.
+
+### ▶️ Device Targeting (Secondary*)
+
+You can also target a **specific device** rather than a group. See [`apt_update_example.yaml`](./examples/linux/apt_update_example.yaml) for reference.
+
+### ▶️ Variables
+
+Variables are replaced by Meshbook before execution. Syntax:
+
+```yaml
+variables:
+  - name: example_var
+    value: "Example value"
+
+tasks:
+  - name: Use the variable
+    command: 'echo "{{ example_var }}"'
+```
+
+* Primary and Secondary mark the order in which will take prescendence
+
+### ▶️ Tasks
+
+Define multiple tasks:
+
+```yaml
+tasks:
+  - name: Show OS info
+    command: "cat /etc/os-release"
+```
+
+Each task must include:
+
+* `name`: Description for human readability.
+* `command`: The actual shell or PowerShell command.
+
+---
+
+## 🪟 Windows Client Notes
+
+* Keep your `os_categories.json` up to date for proper OS filtering.
+* Ensure Windows commands are compatible (use `powershell: true` if needed).
+* Examples are available in [`examples/windows`](./examples/windows).
+
+---
+
+## 🔎 OS & Tag Filtering
+
+### Filter by OS
+
+You can limit commands to specific OS types:
+
+```yaml
+target_os: "Linux"  # As defined in os_categories.json
+```
+
+See [docs/operating\_system\_filtering.md](./docs/operating_system_filtering.md) for details.
+
+### Filter by Tag
+
+You can also filter using MeshCentral tags:
+
+```yaml
+target_tag: "Production"
+```
+
+> ⚠️ Tag values are **case-sensitive**.
+
+---
+
+## 📋 Example Playbook
 
 ```yaml
 ---
-name: Ping Multiple Points
+name: Echo OS Info
-company: Temp-Agents
+group: "Dev"
+target_os: "Linux"
+variables:
+  - name: file
+    value: "/etc/os-release"
 tasks:
-  - name: Ping Cloudflare
+  - name: Show contents of os-release
-    command: "ping 1.1.1.1 -c 4"
+    command: "echo $(cat {{ file }})"
-
-  - name: Ping Google
-    command: "ping 8.8.8.8 -c 4"
 ```
 
-The following response it received when executing the first yaml of the above files.
+Sample output:
 
-```shell
+```json
-python3 meshbook.py -pb examples/ping.yaml -s
 {
-    "Batch 1": [
+  "Task 1": {
-        {
+    "task_name": "Show contents of os-release",
-            "action": "msg",
+    "data": [
-            "type": "runcommands",
+      {
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=6.88 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=6.50 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=59 time=6.46 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=59 time=6.51 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 6.460/6.588/6.879/0.169 ms\n",
+        "command": "echo $(cat /etc/os-release)",
-            "responseid": "meshctrl",
+        "result": [
-            "nodeid": "MSI"
+          "NAME=\"Ubuntu\"",
-        },
+          "VERSION=\"22.04.4 LTS (Jammy Jellyfish)\""
-        {
+        ],
-            "action": "msg",
+        "complete": true,
-            "type": "runcommands",
+        "device_name": "dev-host1"
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=6.22 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=6.07 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=5.97 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=5.90 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms\nrtt min/avg/max/mdev = 5.904/6.038/6.216/0.117 ms\n",
+      }
-            "responseid": "meshctrl",
-            "nodeid": "server"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=6.83 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=6.64 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=59 time=6.65 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=59 time=6.53 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 6.534/6.664/6.834/0.108 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "raspberrypi5"
-        }
-    ],
-    "Batch 2": [
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.\n64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=5.69 ms\n64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=5.22 ms\n64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=5.19 ms\n64 bytes from 8.8.8.8: icmp_seq=4 ttl=118 time=5.16 ms\n\n--- 8.8.8.8 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms\nrtt min/avg/max/mdev = 5.161/5.315/5.694/0.219 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "MSI"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.\n64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=5.65 ms\n64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=5.28 ms\n64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=5.25 ms\n64 bytes from 8.8.8.8: icmp_seq=4 ttl=118 time=5.25 ms\n\n--- 8.8.8.8 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms\nrtt min/avg/max/mdev = 5.246/5.357/5.648/0.168 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "raspberrypi5"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.\n64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=4.94 ms\n64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=4.68 ms\n64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=4.79 ms\n64 bytes from 8.8.8.8: icmp_seq=4 ttl=118 time=4.77 ms\n\n--- 8.8.8.8 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 4.678/4.792/4.940/0.094 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "server"
-        }
     ]
+  }
 }
 ```
 
-# Important Notice:
+---
 
-If you want to use this, make sure to use `NON-BLOCKING` commands. MeshCentral does not work if you send it commands that wait.<br>
+## ⚠️ Blocking Commands Warning
-A couple examples of `BLOCKING COMMANDS` which will never get back to the main MeshCentral server:
 
-```shell
+Avoid using commands that **block indefinitely** — MeshCentral requires **non-blocking** execution.
-apt upgrade # without -y.
 
+🚫 Examples of bad (blocking) commands:
+
+```bash
+apt upgrade       # Without -y
 sleep infinity
-
+ping 1.1.1.1      # Without -c
-ping 1.1.1.1 # without a -c flag (because it pings forever).
 ```
+
+✅ Use instead:
+
+```bash
+apt upgrade -y
+ping 1.1.1.1 -c 1
+```
+
+---
+
+## 🧪 Check Python Environment
+
+Sometimes the wrong Python interpreter or environment is used. To verify:
+
+```bash
+python3 -m pip list
+pip3 list
+```
+
+The lists should match. If not, make sure the correct environment is activated.
+
+---
+
+## 📂 Project Structure (excerpt)
+
+```bash
+meshbook/
+├── books/
+│   ├── apt-update.yaml
+│   └── rdp.yaml
+├── examples/
+│   ├── linux/
+│   │   ├── apt_update_example.yaml
+│   │   └── ...
+│   └── windows/
+│       ├── get_sys_info.yaml
+│       └── ...
+├── modules/
+│   ├── executor.py
+│   └── utilities.py
+├── meshbook.py
+├── os_categories.json
+├── requirements.txt
+├── templates/
+│   └── config.conf.template
+```
+
+---
+
+## 📄 License
+
+This project is licensed under the terms of the GPL3 License. See [LICENSE](./LICENSE).

SECURITY.md

@@ -0,0 +1,11 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| >= 1.0   | :white_check_mark: |
+
+## Reporting a Vulnerability
+
+If you encounter a vulnerability, report this by sending a email to dselen@nerthus.nl or via the GitHub issues.

docs/operating_system_filtering.md

@@ -0,0 +1,65 @@
+# **Understanding the OS Filtering Mechanism**
+
+## **Overview**
+This function filters devices based on their **reachability** and an optional **OS category filter**. It supports:
+
+- **Broad OS categories** (e.g., `"Linux"` includes all OS versions under `"Linux"`)
+- **Specific OS categories** (e.g., `"Debian"` only includes OS versions under `"Linux" -> "Debian"`)
+- **Single category selection** (Only `target_os="Linux"` OR `target_os="Debian"` is used, never both at once)
+
+---
+
+## **How It Works (Simplified)**
+
+### **1. OS Category Expansion**
+The function first expands the `target_os` category by retrieving all valid OS names under it.
+
+#### **Example OS Category Structure:**
+```json
+{
+    "Linux": {
+        "Debian": [
+            "Debian GNU/Linux 12 (bookworm)",
+            "Debian GNU/Linux 11 (bullseye)"
+        ],
+        "Ubuntu": [
+            "Ubuntu 24.04.1 LTS"
+        ]
+    }
+}
+```
+
+#### **Expanding Different `target_os` Values:**
+
+| `target_os`   | Expanded OS Versions |
+|--------------|---------------------------------------------------|
+| `"Linux"`   | `{ "Debian GNU/Linux 12 (bookworm)", "Debian GNU/Linux 11 (bullseye)", "Ubuntu 24.04.1 LTS" }` |
+| `"Debian"`  | `{ "Debian GNU/Linux 12 (bookworm)", "Debian GNU/Linux 11 (bullseye)" }` |
+
+---
+
+### **2. Device Filtering**
+Once the function has the allowed OS versions, it checks each device:
+
+#### **Example Device List:**
+```json
+[
+    {"device_id": "A1", "device_os": "Debian GNU/Linux 12 (bookworm)", "reachable": true},
+    {"device_id": "A2", "device_os": "Ubuntu 24.04.1 LTS", "reachable": true},
+    {"device_id": "A3", "device_os": "Windows 11", "reachable": true},
+    {"device_id": "A4", "device_os": "Debian GNU/Linux 11 (bullseye)", "reachable": false}
+]
+```
+
+#### **Filtering Behavior:**
+| Device ID | Device OS                         | Reachable | Matches `target_os="Linux"` | Matches `target_os="Debian"` |
+|-----------|----------------------------------|-----------|-------------------------------|-------------------------------|
+| A1        | Debian GNU/Linux 12 (bookworm)  | ✅        | ✅                             | ✅                             |
+| A2        | Ubuntu 24.04.1 LTS              | ✅        | ✅                             | ❌                             |
+| A3        | Windows 11                      | ✅        | ❌                             | ❌                             |
+| A4        | Debian GNU/Linux 11 (bullseye)  | ❌        | ❌ (Unreachable)               | ❌ (Unreachable)               |
+
+#### **Final Output:**
+- If `target_os="Linux"`: `["A1", "A2"]`
+- If `target_os="Debian"`: `["A1"]`
+- If `target_os=None` or `target_os` is undefined: `["A1", "A2", "A3"]`

examples/all_variable_usage_example.yaml

@@ -0,0 +1,19 @@
+---
+name: Ping Multiple Points
+group: "Kubernetes"
+#target_os: "Debian"
+variables:
+  - name: host1
+    value: "1.1.1.1"
+  - name: host2
+    value: "9.9.9.9"
+  - name: command1
+    value: "ping"
+  - name: cmd_arguments
+    value: "-c 4"
+tasks:
+  - name: Ping host1
+    command: "{{ command1 }} {{ host1 }} {{ cmd_arguments }}"
+
+  - name: Ping host2
+    command: "{{ command1 }} {{ host2 }} {{ cmd_arguments }}"

examples/linux/aggregate_example.yaml

@@ -0,0 +1,32 @@
+---
+name: Echo some text in the terminal of the device
+group: "Development"
+target_os: "Linux" # <----
+variables:
+  - name: package_manager
+    value: "apt"
+  - name: google_dns
+    value: "8.8.8.8"
+  - name: "quad9_dns"
+    value: "9.9.9.9"
+tasks:
+  - name: refresh the cache
+    command: "{{ package_manager }} update"
+
+  - name: display available upgrades
+    command: "{{ package_manager }} list --upgradable"
+
+  - name: apply upgrades
+    command: "{{ package_manager }} upgrade -y"
+
+  - name: cleanup remaining packages
+    command: "{{ package_manager }} autoremove -y"
+
+  - name: run autoclean
+    command: "{{ package_manager }} autoclean -y"
+
+  - name: Ping Google DNS
+    command: "ping {{ google_dns }} -c 4"
+
+  - name: Ping Quad9 DNS
+    command: "ping {{ quad9_dns }} -c 4"

examples/linux/apt_update_example.yaml

@@ -0,0 +1,13 @@
+---
+name: Refresh the apt cache
+device: "<Device-Name>"
+#target_os: "Linux"
+variables:
+  - name: package_manager
+    value: "apt"
+tasks:
+  - name: refresh the {{ package_manager }} cache
+    command: "{{ package_manager }} update"
+
+  - name: display available upgrades with {{ package_manager }}
+    command: "{{ package_manager }} list --upgradable"

examples/linux/apt_upgrade_example.yaml

@@ -0,0 +1,19 @@
+---
+name: Refresh the apt cache
+group: "Dev"
+#target_os: "Linux"
+variables:
+  - name: package_manager
+    value: "apt"
+tasks:
+  - name: refresh the cache
+    command: "{{ package_manager }} update"
+
+  - name: display available upgrades
+    command: "{{ package_manager }} list --upgradable"
+
+  - name: apply upgrades
+    command: "{{ package_manager }} upgrade -y"
+
+  - name: cleanup remaining packages
+    command: "{{ package_manager }} autoremove -y"

examples/linux/disk_info.yaml

@@ -0,0 +1,8 @@
+---
+name: Use DF to get drive information in JSON.
+group: Systemec Development
+target_os: "Linux"
+tasks:
+  - name: Get disk-info with df returning JSON.
+    command: >
+      df -Th -x overlay -x tmpfs -x devtmpfs | awk 'NR>1 {printf "%s{\"size\":\"%s\",\"used\":\"%s\",\"available\":\"%s\",\"mount_point\":\"%s\",\"type\":\"%s\"}", (NR==2?"[":","), $3, $4, $5, $7, $2} END {print "]"}'

examples/linux/echo_example.yaml

@@ -0,0 +1,10 @@
+---
+name: Echo a string to the terminal through the meshbook example.
+group: "Development"
+target_os: "Linux"
+variables:
+  - name: file
+    value: "/etc/os-release"
+tasks:
+  - name: Echo!
+    command: "echo $(cat {{ file }})"

examples/windows/get_sys_info.yaml

@@ -0,0 +1,11 @@
+---
+name: Echo a string to the terminal through the meshbook example.
+group: "Endpoint"
+target_os: "Windows"
+powershell: True
+#variables:
+#  - name: file
+#    value: "/etc/os-release"
+tasks:
+  - name: Echo!
+    command: "Get-ComputerInfo | Select-Object CsName, OsName, OsArchitecture, OsLastBootUpTime | Write-Output"

examples/windows/get_update_info.yaml

@@ -0,0 +1,8 @@
+---
+name: Echo a string to the terminal through the meshbook example.
+group: "Endpoint"
+target_os: "Windows"
+powershell: True
+tasks:
+  - name: Get some update information
+    command: "Get-HotFix | Select-Object PSComputerName, HotFixID, InstalledOn"

examples/windows/multiline_example.yaml

@@ -0,0 +1,16 @@
+---
+name: Echo a string to the terminal through the meshbook example.
+group: "Endpoint"
+target_os: "Windows"
+powershell: True
+#variables:
+#  - name: file
+#    value: "/etc/os-release"
+tasks:
+  - name: Echo!
+    command: >
+      $systemInfo = Get-ComputerInfo | Select-Object CsName, OsName, OsArchitecture, OsLastBootUpTime;
+      $systemInfo | Format-Table -AutoSize;
+      Write-Output "I like monkeys.";
+      Get-SystemLanguage | Write-Output;
+      Get-ComputerInfo | Format-List;

legacy/meshbook-legacy.py

@@ -49,29 +49,27 @@ class MeshbookUtilities:
 
         return ids
 
-    @staticmethod
-    def load_config(conffile: str = None, segment: str = 'meshcentral-service') -> ConfigParser:
-        """Load and return the configuration from a file."""
-        conffile = conffile or './api.conf'
-        if not os.path.exists(conffile):
-            raise ScriptEndTrigger(f'Missing config file {conffile}. Provide an alternative path.')
 
-        try:
-            my_config = ConfigParser()
-            my_config.read(conffile)
-        except Exception as err:
-            raise ScriptEndTrigger(f'Error reading config file {conffile}: {err}')
-        
-        if segment not in my_config:
-            raise ScriptEndTrigger(f'Segment "{segment}" not found in config file {conffile}.')
-
-        return my_config[segment]
 
     @staticmethod
     def read_yaml(file_path: str) -> dict:
         """Read a YAML file and return its content as a dictionary."""
         with open(file_path, 'r') as file:
             return yaml.safe_load(file)
+        
+    @staticmethod
+    def replace_placeholders(playbook) -> dict:
+        # Convert 'variables' to a dictionary for quick lookup
+        variables = {var["name"]: var["value"] for var in playbook.get("variables", [])}
+
+        # Traverse 'tasks' to replace placeholders
+        for task in playbook.get("tasks", []):
+            command = task.get("command", "")
+            for var_name, var_value in variables.items():
+                placeholder = f"{{{{ {var_name} }}}}"  # Create the placeholder string like "{{ host1 }}"
+                command = command.replace(placeholder, var_value)  # Update the command string
+            task["command"] = command  # Save the updated command string
+        return playbook
 
     @staticmethod
     def translate_nodeids(batches_dict, global_list) -> dict:
@@ -220,6 +218,7 @@ class MeshbookProcessor:
         """Processes messages received from the WebSocket."""
         global response_counter
         temp_responses_list = []
+        
         while True:
             message = await python_client.received_response_queue.get()
             action_type = message.get('action')
@@ -250,13 +249,12 @@ class MeshcallerActions:
     """Processes playbook actions."""
     
     @staticmethod
-    async def process_arguments(python_client: MeshbookWebsocket, playbook_path: str):
+    async def process_arguments(python_client: MeshbookWebsocket, playbook_yaml: dict):
         """Executes tasks defined in the playbook."""
         global response_counter, expected_responses, target_ids
 
         await basic_ready_state.wait()  # Wait for the basic data to be ready
 
-        playbook_yaml = MeshbookUtilities.read_yaml(playbook_path)
         target_ids = MeshbookUtilities.get_target_ids(
             company=playbook_yaml.get('company'),
             device=playbook_yaml.get('device')
@@ -308,9 +306,10 @@ class MeshcallerActions:
 
 async def main():
     parser = argparse.ArgumentParser(description="Process command-line arguments")
+    parser.add_argument("-pb", "--playbook", type=str, help="Path to the playbook file.", required=True)
+
     parser.add_argument("--conf", type=str, help="Path for the API configuration file (default: ./api.conf).")
     parser.add_argument("--nojson", action="store_true", help="Makes the program not output the JSON response data.")
-    parser.add_argument("-pb", "--playbook", type=str, help="Path to the playbook file.", required=True)
     parser.add_argument("-s", "--silent", action="store_true", help="Suppress terminal output.")
     parser.add_argument("-i", "--information", action="store_true", help="Add the calculations and other informational data to the output.")
 
@@ -328,7 +327,11 @@ async def main():
             credentials['password']
         ))
         processor_task = asyncio.create_task(processor.receive_processor(python_client))
-        await MeshcallerActions.process_arguments(python_client, args.playbook)
+
+        playbook_yaml = MeshbookUtilities.read_yaml(args.playbook)
+        translated_playbook = MeshbookUtilities.replace_placeholders(playbook_yaml)
+        await MeshcallerActions.process_arguments(python_client, translated_playbook)
+
         await asyncio.gather(websocket_task, processor_task)
         
     except ScriptEndTrigger as e:

meshbook.py

@@ -0,0 +1,278 @@
+#!/bin/python3
+
+# Public Python libraries
+import argparse
+import asyncio
+from colorama import just_fix_windows_console
+import pyotp
+import json
+import meshctrl
+
+# Local Python libraries/modules
+from modules.console import Console
+from modules.executor import Executor
+from modules.history import History
+from modules.utilities import Transform, Utilities
+
+meshbook_version = "1.3.2"
+grace_period = 3 # Grace period will last for x (by default 3) second(s).
+
+def define_cmdargs() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="Process command-line arguments")
+
+    parser.add_argument("-mb", "--meshbook", type=str, help="Path to the meshbook yaml file.")
+
+    parser.add_argument("--historydir", type=str, help="Define a custom history log directory (default: ./history).", default="./history")
+    parser.add_argument("--nohistory", action="store_true", help="Disable the logging of the history into a local log (text) file inside './history'.")
+    parser.add_argument("--flushhistory", action="store_true", help="Clear old history logs before running the Meshbook.")
+
+    parser.add_argument("-oc", "--oscategories", type=str, help="Path to the Operating System categories JSON file.", default="./os_categories.json")
+    parser.add_argument("--conf", type=str, help="Path for the API configuration file (default: ./config.conf).", default="./api.conf")
+    parser.add_argument("--nograce", action="store_true", help="Disable the grace 3 seconds before running the meshbook.")
+
+    parser.add_argument("-g", "--group", type=str, help="Specify a manual override for the group.", default="")
+    parser.add_argument("-d", "--device", type=str, help="Specify a manual override for a device.", default="")
+    parser.add_argument("-i", "--indent", action="store_true", help="Use an JSON indentation of 4 when this flag is passed.", default=False)
+    parser.add_argument("-s", "--silent", action="store_true", help="Suppress terminal output.", default=False)
+    parser.add_argument("--shlex", action="store_true", help="Shlex the lines. (SHell LEXical Analysis)", default=False)
+
+    parser.add_argument("--version", action="store_true", help="Show the Meshbook version.")
+
+    return parser
+
+async def init_connection(credentials: dict) -> meshctrl.Session:
+    '''
+    Use the libmeshctrl library to initiate a Secure Websocket (wss) connection to the MeshCentral instance.
+    '''
+
+    if "totp_secret" in credentials:
+        totp = pyotp.TOTP(credentials["totp_secret"])
+        otp = totp.now()
+
+        session = meshctrl.Session(
+            credentials['hostname'],
+            user=credentials['username'],
+            password=credentials['password'],
+            token=otp
+        )
+    else:
+        session = meshctrl.Session(
+            credentials['hostname'],
+            user=credentials['username'],
+            password=credentials['password']
+        )
+    await session.initialized.wait()
+    return session
+
+async def main():
+    local_categories_file = "./os_categories.json"
+
+    just_fix_windows_console()
+    '''
+    Main function where the program starts. Place from which all comands originate (eventually).
+    '''
+
+    # Define the cmd arguments
+    parser = define_cmdargs()
+    args = parser.parse_args()
+
+    if args.version:
+        Console.print_text(args.silent,
+                           Console.text_color.reset + "MeshBook Version: " + Console.text_color.yellow + str(meshbook_version))
+        return
+
+    if not args.meshbook:
+        parser.print_help()
+        return
+
+    try:
+        with open(local_categories_file, "r") as file:
+            os_categories = json.load(file)
+
+        if not Utilities.path_exist(args.meshbook) or Utilities.path_type(args.meshbook) != "File":
+            Console.print_text(args.silent,
+                               Console.text_color.red + "The given meshbook path is either not present on the filesystem or not a file.")
+            return
+
+        credentials, meshbook = await asyncio.gather(
+            (Utilities.load_config(args)),
+            (Utilities.compile_book(args.meshbook))
+        )
+
+        if args.group != "":
+            meshbook["group"] = args.group
+            if "device" in meshbook:
+              del meshbook["device"]
+        elif args.device != "":
+            meshbook["device"] = args.device
+            if "group" in meshbook:
+              del meshbook["group"]
+
+        '''
+        The following section mainly displays used variables and first steps of the program to the Console.
+        '''
+
+        # INIT ARGUMENTS PRINTING
+        Console.print_line(args.silent)
+        Console.print_text(args.silent,
+                           "meshbook: " + Console.text_color.yellow + args.meshbook + Console.text_color.reset + ".")
+        Console.print_text(args.silent,
+                           "Operating System Categorisation file: " + Console.text_color.yellow + args.oscategories + Console.text_color.reset + ".")
+        Console.print_text(args.silent,
+                           "Configuration file: " + Console.text_color.yellow + args.conf + Console.text_color.reset + ".")
+
+        # TARGET OS PRINTING
+        if "target_os" in meshbook:
+            Console.print_text(args.silent,
+                               "Target Operating System category given: " + Console.text_color.yellow + meshbook["target_os"] + Console.text_color.reset + ".")
+        else:
+            Console.print_text(args.silent,
+                               "Target Operating System category given: " + Console.text_color.yellow + "All" + Console.text_color.reset + ".")
+
+        # Should Meshbook ignore categorisation?
+        if "ignore_categorisation" in meshbook:
+            Console.print_text(args.silent,
+                                "Ignore the OS Categorisation file: " + Console.text_color.yellow + str(meshbook["ignore_categorisation"]) + Console.text_color.reset + ".")
+            if meshbook["ignore_categorisation"]:
+                Console.print_text(args.silent,
+                                Console.text_color.red + "!!!!\n" +
+                                Console.text_color.yellow + 
+                                "Ignore categorisation is True.\nThis means that the program checks if the target Operating System is somewhere in the reported device Operating System." + 
+                                Console.text_color.red + "\n!!!!")
+        else:
+            Console.print_text(args.silent,
+                                "Ignore the OS Categorisation file: " + Console.text_color.yellow + "False" + Console.text_color.reset + ".")
+
+        # TARGET TAG PRINTING
+        if "target_tag" in meshbook:
+            Console.print_text(args.silent,
+                               "Target Device tag given: " + Console.text_color.yellow + meshbook["target_tag"] + Console.text_color.reset + ".")
+        else:
+            Console.print_text(args.silent,
+                               "Target Device tag given: " + Console.text_color.yellow + "All" + Console.text_color.reset + ".")
+
+        # TARGET PRINTING
+        if "device" in meshbook:
+            Console.print_text(args.silent,
+                               "Target device: " + Console.text_color.yellow + str(meshbook["device"]) + Console.text_color.reset + ".")
+        elif "devices" in meshbook:
+            Console.print_text(args.silent,
+                               "Target devices: " + Console.text_color.yellow + str(meshbook["devices"]) + Console.text_color.reset + ".")
+        elif "group" in meshbook:
+            Console.print_text(args.silent,
+                               "Target group: " + Console.text_color.yellow + str(meshbook["group"]) + Console.text_color.reset + ".")
+        elif "groups" in meshbook:
+            Console.print_text(args.silent,
+                               "Target groups: " + Console.text_color.yellow + str(meshbook["groups"]) + Console.text_color.reset + ".")
+
+        # RUNNING PARAMETERS PRINTING
+        Console.print_text(args.silent, "Grace: " + Console.text_color.yellow + str((not args.nograce))) # Negation of bool for correct explanation
+        Console.print_text(args.silent, "Silent: " + Console.text_color.yellow + "False") # Can be pre-defined because if silent flag was passed then none of this would be printed.
+
+        session = await init_connection(credentials)
+
+        # PROCESS PRINTING aka what its doing in the moment...
+        Console.print_line(args.silent)
+        Console.print_text(args.silent,
+                           Console.text_color.italic + "Trying to load the MeshCentral account credential file...")
+        Console.print_text(args.silent,
+                           Console.text_color.italic + "Trying to load the meshbook yaml file and compile it into something workable...")
+        Console.print_text(args.silent,
+                           Console.text_color.italic + "Trying to load the Operating System categorisation JSON file...")
+        Console.print_text(args.silent,
+                           Console.text_color.italic + "Connecting to MeshCentral and establish a session using variables from previous credential file.")
+        Console.print_text(args.silent,
+                           Console.text_color.italic + "Generating group list with nodes and reference the targets from that.")
+
+        '''
+        End of the main information displaying section.
+        '''
+
+        group_list = await Transform.compile_group_list(session)
+        compiled_device_list = await Utilities.gather_targets(args, meshbook, group_list, os_categories)
+
+        # Check if we have reachable targets on the MeshCentral host
+        if "target_list" not in compiled_device_list or len(compiled_device_list["target_list"]) == 0:
+            Console.print_text(args.silent,
+                               Console.text_color.red + "No targets found or targets unreachable, quitting.")
+
+            Console.print_line(args.silent)
+            return
+
+        Console.print_line(args.silent)
+
+        match meshbook:
+            case {"group": candidate_target_name}:
+                target_name = candidate_target_name
+
+            case {"groups": candidate_target_name}:
+                target_name = str(candidate_target_name)
+
+            case {"device": candidate_target_name}:
+                target_name = candidate_target_name
+
+            case {"devices": candidate_target_name}:
+                target_name = str(candidate_target_name)
+
+            case _:
+                target_name = ""
+
+        # Initialize the history / logging functions class (whatever you want to name it)
+        history = History(args.silent, args.historydir, args.flushhistory)
+
+        # Conclude history initlialization
+        Console.print_line(args.silent)
+
+        # From here on the actual exection happens
+        Console.print_text(args.silent,
+                            Console.text_color.yellow + "Executing meshbook on the target(s): " + Console.text_color.green + target_name + Console.text_color.yellow + ".")
+
+        if not args.nograce:
+            Console.print_text(args.silent,
+                                Console.text_color.yellow + "Initiating grace-period...")
+
+            for x in range(grace_period):
+                Console.print_text(args.silent,
+                                    Console.text_color.yellow + "{}...".format(x+1)) # Countdown!
+                await asyncio.sleep(1)
+
+        Console.print_line(args.silent)
+        complete_log = await Executor.execute_meshbook(args.silent,
+                                                    args.shlex,
+                                                    session,
+                                                    compiled_device_list,
+                                                    meshbook,
+                                                    group_list)
+        Console.print_line(args.silent)
+
+        indent = None
+        if args.indent: indent = 4
+
+        formatted_history = json.dumps(complete_log,indent=indent)
+
+        Console.print_text(args.silent, formatted_history, 9)
+
+        # Pass the output of the whole program to the history class
+        if args.nohistory:
+            Console.print_text(args.silent, "Not writing to file.")
+        else:
+            Console.print_text(args.silent, "Writing to file...")
+            history.write_history(formatted_history)
+
+        await session.close()
+
+    except OSError as message:
+        Console.print_text(args.silent,
+                           Console.text_color.red + f'{message}')
+
+    except asyncio.CancelledError:
+        Console.print_text(args.silent,
+                           Console.text_color.red + "Received SIGINT, Aborting - (Tasks may still be running on targets).")
+        await session.close()
+        raise
+
+if __name__ == "__main__":
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        Console.print_text(False, Console.text_color.red + "Cancelled execution.")

meshbook/examples/apt_upgrade.yaml

@@ -1,16 +0,0 @@
----
-name: Refresh the apt cache
-company: Temp-Agents
-#device: MSI
-tasks:
-  - name: refresh the cache
-    command: "apt update"
-
-  - name: display available upgrades
-    command: "apt list --upgradable"
-
-  - name: apply upgrades
-    command: "apt upgrade -y"
-
-  - name: cleanup remaining packages
-    command: "apt autoremove -y"

meshbook/examples/ping.yaml

@@ -1,9 +0,0 @@
----
-name: Ping Multiple Points
-company: Temp-Agents
-tasks:
-  - name: Ping Cloudflare
-    command: "ping 1.1.1.1 -c 4"
-
-  - name: Ping Google
-    command: "ping 8.8.8.8 -c 4"

meshbook/examples/refresh_aptcache.yaml

@@ -1,9 +0,0 @@
----
-name: Refresh the apt cache
-company: Temp-Agents
-tasks:
-  - name: refresh the cache
-    command: "apt update"
-
-  - name: display available upgrades
-    command: "apt list --upgradable"

meshbook/requirements.txt

@@ -1,4 +0,0 @@
-asyncio==3.4.3
-configparser==7.1.0
-pyyaml==6.0.2
-websockets==14.1

modules/console.py

@@ -0,0 +1,54 @@
+# Public Python libraries
+import argparse
+from datetime import datetime
+
+class Console:
+    class text_color:
+        black = "\033[30m"
+        red = "\033[31m"
+        green = "\033[32m"
+        yellow = "\033[33m"
+        blue = "\033[34m"
+        magenta = "\033[35m"
+        cyan = "\033[36m"
+        white = "\033[37m"
+        italic = "\x1B[3m"
+        reset = "\x1B[0m"
+
+    @staticmethod
+    def print_text(silent: bool, message: str, prefix_select: int = 0) -> None:
+        '''
+        Helper function for terminal output, with a couple variables for the silent flag. Also clears terminal color each time.
+
+        int tag_select legend:
+        0 / default = timestamp
+        1 = info
+        2 = warn
+        3 = err
+        4 = fatal
+        9 = nothing
+        '''
+        match prefix_select:
+            case 1:
+                tag_prefix = "[INFO] "
+            case 2:
+                tag_prefix = "[WARN] "
+            case 3:
+                tag_prefix = "[ERROR] "
+            case 4:
+                tag_prefix = "[FATAL] "
+            case 9:
+                tag_prefix = ""
+            case _:
+                tag_prefix = f"{datetime.now().strftime('%Y-%m-%d %H:%M:%S')} "
+
+        if not silent:
+            print(tag_prefix + message + Console.text_color.reset)
+
+    @staticmethod
+    def print_line(silent: bool, special: bool = False) -> None:
+        if not silent:
+            if special:
+                print("-=-" * 40)
+            else:
+                print(("-" * 40))

modules/executor.py

@@ -0,0 +1,55 @@
+# Public Python libraries
+import argparse
+import json
+import meshctrl
+from time import sleep
+
+# Local Python libraries/modules
+from modules.console import Console
+from modules.utilities import Transform
+
+intertask_delay = 1
+
+class Executor:
+    @staticmethod
+    async def execute_meshbook(silent: bool, enable_shlex: bool, session: meshctrl.Session, compiled_device_list: dict, meshbook: dict, group_list: dict) -> dict:
+        '''
+        Actual function that handles meshbook execution, also responsible for formatting the resulting JSON.
+        '''
+
+        complete_log = {}
+        targets = compiled_device_list["target_list"]
+        offline = compiled_device_list["offline_list"]
+        round = 1
+
+        for task in meshbook["tasks"]:
+            Console.print_text(silent,
+                               Console.text_color.green + str(round) + ". Running: " + task["name"])
+
+            if "powershell" in meshbook and meshbook["powershell"]:
+                response = await session.run_command(nodeids=targets, command=task["command"],powershell=True,ignore_output=False,timeout=1800)
+            else:
+                response = await session.run_command(nodeids=targets, command=task["command"],powershell=False,ignore_output=False,timeout=1800)
+
+            task_batch = []
+            for device in response:
+                device_result = response[device]["result"]
+                response[device]["result"] = device_result.replace("Run commands completed.", "")
+                response[device]["device_id"] = device
+                response[device]["device_name"] = await Transform.translate_nodeid_to_name(device, group_list)
+                task_batch.append(response[device])
+
+            complete_log["task_" + str(round)] = {
+                "task_name": task["name"],
+                "data": task_batch
+            }
+            round += 1
+            sleep(intertask_delay) # Sleep for x amount of time.
+
+        for index, device in enumerate(offline): # Replace Device_id with actual human readable name
+            device_name = await Transform.translate_nodeid_to_name(device, group_list)
+            offline[index] = device_name
+        complete_log["Offline"] = offline
+
+        # Return the result 
+        return Transform.process_shell_response(enable_shlex, complete_log)

modules/history.py

@@ -0,0 +1,48 @@
+import os
+from datetime import datetime
+
+from modules.console import Console
+
+class History():
+    def __init__(self, silent: bool, history_directory: str, flush_history: bool) -> None:
+        '''
+        Init function to declare some stuff and make sure we are good to go, mostly the directory.
+        '''
+        self.silent = silent
+        self.history_directory = history_directory
+
+        if not os.path.exists(history_directory):
+            Console.print_text(silent, "Directory absent, trying to create it now...")
+
+            try:
+                os.mkdir(history_directory)
+
+            except PermissionError:
+                Console.print_text(silent, Console.text_color.red + f"Failed to create directory, permission error.")
+                return
+        
+        history_items = os.listdir(history_directory)
+        if len(history_items) == 1:
+            Console.print_text(silent, f"There is {len(history_items)} history item.")
+        else:
+            Console.print_text(silent, f"There are {len(history_items)} history items.")
+
+        if flush_history:
+            self.remove_history(history_items)
+
+    def remove_history(self, history_items: list[str]) -> None:
+        if not os.access(self.history_directory, os.W_OK):
+            Console.print_text(self.silent, Console.text_color.red + "Unable to flush history logs, no write access.")
+            return
+
+        for item in history_items:
+            stitched_path = f"{self.history_directory}/{item}"
+
+            Console.print_text(self.silent, f"Removing: {item}.")
+            os.remove(stitched_path)
+
+    def write_history(self, history: dict) -> bool:
+        stitched_file = f"{self.history_directory}/meshbook_run_{datetime.now().strftime('%Y_%m_%d_%H_%M_%S')}.log"
+
+        with open(stitched_file, "x") as f:
+            f.write(history)

modules/utilities.py

@@ -0,0 +1,371 @@
+# Public Python libraries
+import argparse
+from configparser import ConfigParser
+import meshctrl
+import os
+import shlex
+import yaml
+
+'''
+Creation and compilation of the MeshCentral nodes list (list of all nodes available to the user in the configuration) is handled in the following section.
+'''
+
+class Utilities:
+    @staticmethod
+    async def load_config(args: argparse.Namespace,
+                          segment: str = 'meshcentral-account') -> dict:
+        '''
+        Function that loads the segment from the config.conf (by default) file and returns the it in a dict.
+        '''
+
+        conf_file = args.conf
+        if not os.path.exists(conf_file):
+            print(f'Missing config file {conf_file}. Provide an alternative path.')
+            os._exit(1)
+
+        config = ConfigParser()
+        try:
+            config.read(conf_file)
+        except Exception as err:
+            print(f"Error reading configuration file '{conf_file}': {err}")
+            os._exit(1)
+
+        if segment not in config:
+            print(f'Segment "{segment}" not found in config file {conf_file}.')
+            os._exit(1)
+
+        return dict(config[segment])
+
+    @staticmethod
+    async def compile_book(meshbook_file: str) -> dict:
+        '''
+        Simple function that opens the file and replaces placeholders through the next function. After that just return it.
+        '''
+
+        with open(meshbook_file, 'r') as f:
+            meshbook = f.read()
+        meshbook = await Transform.replace_placeholders(yaml.safe_load(meshbook))
+        return meshbook
+
+    @staticmethod
+    async def gather_targets(args: argparse.Namespace,
+                            meshbook: dict,
+                            group_list: dict[str, list[dict]],
+                            os_categories: dict) -> dict:
+        """
+        Finds target devices based on meshbook criteria (device, devices, group or groups).
+        """
+
+        group_list = {k.lower(): v for k, v in group_list.items()}  # Normalize keys
+        target_list = []
+        offline_list = []
+
+        target_os = meshbook.get("target_os")
+        target_tag = meshbook.get("target_tag")
+        ignore_categorisation = meshbook.get("ignore_categorisation", False)
+
+        async def add_processed_devices(processed):
+            """Helper to update target and offline lists."""
+            if processed:
+                target_list.extend(processed.get("valid_devices", []))
+                offline_list.extend(processed.get("offline_devices", []))
+
+        async def process_device_helper(device):
+            processed = await Utilities.process_device(
+                device,
+                group_list,
+                os_categories,
+                target_os,
+                ignore_categorisation,
+                target_tag
+            )
+            await add_processed_devices(processed)
+
+        async def process_group_helper(group):
+            processed = await Utilities.filter_targets(
+                group, os_categories, target_os, ignore_categorisation, target_tag
+            )
+            await add_processed_devices(processed)
+
+        '''
+        Groups receive the first priority, then device targets.
+        '''
+        match meshbook:
+            case {"group": pseudo_target}:
+                if isinstance(pseudo_target, str):
+                    pseudo_target = pseudo_target.lower()
+
+                    if pseudo_target in group_list:
+                        await process_group_helper(group_list[pseudo_target])
+
+                    elif pseudo_target not in group_list:
+                        console.nice_print(
+                            args,
+                            console.text_color.yellow + "Targeted group not found on the MeshCentral server."
+                        )
+                elif isinstance(pseudo_target, list):
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "Please use groups (Notice the plural with 'S') for multiple groups."
+                    )
+                else:
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'group' key is being used, but an unknown data type was found, please check your values."
+                    )
+
+            case {"groups": pseudo_target}:
+                if isinstance(pseudo_target, list):
+                    for sub_group in pseudo_target:
+                        sub_group = sub_group.lower()
+                        if sub_group in group_list:
+                            await process_group_helper(group_list[sub_group])
+                elif isinstance(pseudo_target, str) and pseudo_target.lower() == "all":
+                    for group in group_list.values():
+                        await process_group_helper(group)
+                elif isinstance(pseudo_target, str):
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'groups' key is being used, but only one string is given. Did you mean 'group'?"
+                    )
+                else:
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'groups' key is being used, but an unknown data type was found, please check your values."
+                    )
+
+            case {"device": pseudo_target}:
+                if isinstance(pseudo_target, str):
+                    await process_device_helper(pseudo_target)
+                elif isinstance(pseudo_target, list):
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "Please use devices (Notice the plural with 'S') for multiple devices."
+                    )
+                else:
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'device' key is being used, but an unknown data type was found, please check your values."
+                    )
+
+            case {"devices": pseudo_target}:
+                if isinstance(pseudo_target, list):
+                    for sub_device in pseudo_target:
+                        await process_device_helper(sub_device)
+                elif isinstance(pseudo_target, str):
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'devices' key is being used, but only one string is given. Did you mean 'device'?"
+                    )
+                else:
+                    console.nice_print(
+                        args,
+                        console.text_color.yellow + "The 'devices' key is being used, but an unknown data type was found, please check your values."
+                    )
+
+        return {"target_list": target_list, "offline_list": offline_list}
+
+    @staticmethod
+    def get_os_variants(target_category: str,
+                        os_map: dict) -> set:
+        '''
+        Extracts all OS names under a given category if it exists.
+        '''
+
+        for key, value in os_map.items():
+            if key == target_category:
+
+                if isinstance(value, dict):  # Expand nested categories
+                    os_set = set()
+
+                    for sub_target_cat in value:
+                        os_set.update(Utilities.get_os_variants(sub_target_cat, value))
+
+                    return os_set
+
+                elif isinstance(value, list):  # Direct OS list
+                    return set(value)
+
+        return set()
+
+    @staticmethod
+    async def filter_targets(devices: list[dict],
+                             os_categories: dict,
+                             target_os: str = "",
+                             ignore_categorisation: bool = False,
+                             target_tag: str = "") -> dict:
+        '''
+        Filters devices based on reachability and optional OS criteria, supporting nested OS categories.
+        '''
+
+        valid_devices = []
+        offline_devices = []
+        allowed_os = set()
+
+        # Identify correct OS filtering scope
+        for key in os_categories:
+            if key == target_os:
+                allowed_os = Utilities.get_os_variants(target_os, os_categories)
+                break  # Stop searching once a match is found
+
+            if isinstance(os_categories[key], dict) and target_os in os_categories[key]:
+                allowed_os = Utilities.get_os_variants(target_os, os_categories[key])
+                break  # Stop searching once a match is found
+
+        for device in devices: # Filter out unwanted or unreachable devices.
+            if target_tag and target_tag not in device["device_tags"]:
+                continue
+
+            if not ignore_categorisation:
+                if device["device_os"] not in allowed_os:
+                    continue                
+            else:
+                if target_os not in device["device_os"]:
+                    continue
+
+            if not device["reachable"]:
+                offline_devices.append(device["device_id"])
+                continue
+
+            valid_devices.append(device["device_id"])
+
+        return {
+            "valid_devices": valid_devices,
+            "offline_devices": offline_devices
+        }
+
+    @staticmethod
+    async def process_device(device: str,
+                            group_list: dict,
+                            os_categories: dict,
+                            target_os: str,
+                            ignore_categorisation: bool,
+                            target_tag: str) -> dict:
+        """
+        Processes a single device or pseudo-target against group_list,
+        filters matches by OS and tags, and adds processed devices.
+        """
+        matched_devices = []
+        pseudo_target = device.lower()
+
+        # Find devices that match the pseudo_target
+        for group in group_list:
+            for dev in group_list[group]:
+                if dev["device_name"].lower() == pseudo_target:
+                    matched_devices.append(dev)
+
+        # If matches found, filter them and add processed devices
+        if matched_devices:
+            processed = await Utilities.filter_targets(
+                matched_devices, os_categories, target_os, ignore_categorisation, target_tag
+            )
+            return processed
+
+        # No matches found
+        return {"valid_devices": [], "offline_devices": []}
+
+    @staticmethod
+    def path_exist(path: str) -> bool:
+        return os.path.exists(path)
+
+    @staticmethod
+    def path_type(path: str) -> str:
+        if os.path.isfile(path):
+            return "File"
+        if os.path.isdir(path):
+            return "Dir"
+        if os.path.islink(path):
+            return "Link"
+        return "Undefined"
+
+class Transform:
+    @staticmethod
+    def process_shell_response(enable_shlex: bool, meshbook_result: dict) -> dict:
+        for task_name, task_data in meshbook_result.items():
+            if task_name == "Offline": # Failsafe do not parse Offline section, its simple
+                continue
+
+            for node_responses in task_data["data"]:
+                task_result = node_responses["result"].splitlines()
+                
+                if enable_shlex:
+                    for index, line in enumerate(task_result):
+                        line = shlex.split(line)
+                        task_result[index] = line
+
+                clean_output = []
+                for line in task_result:
+                    if len(line) > 0:
+                        clean_output.append(line)
+
+                node_responses["result"] = clean_output
+        return meshbook_result
+
+    @staticmethod
+    async def translate_nodeid_to_name(target_id: str, group_list: dict) -> str:
+        '''
+        Simple function that looks up nodeid to the human-readable name if existent - otherwise return None.
+        '''
+
+        for group in group_list:
+            for device in group_list[group]:
+                if device["device_id"] == target_id:
+                    return device["device_name"]
+        return ""
+    
+    @staticmethod
+    async def replace_placeholders(meshbook: dict) -> dict:
+        '''
+        Replace the placeholders in both name and command fields of the tasks. According to the variables defined in the variables list.
+        '''
+
+        variables = {}
+        if "variables" in meshbook and isinstance(meshbook["variables"], list):
+            for var in meshbook["variables"]:
+                var_name = var["name"]
+                var_value = var["value"]
+                variables[var_name] = var_value
+
+        else:
+            return meshbook
+
+        for task in meshbook.get("tasks", []):
+            task_name = task.get("name")
+
+            for var_name, var_value in variables.items():
+                placeholder = f"{{{{ {var_name} }}}}"
+                task_name = task_name.replace(placeholder, var_value)
+
+            task["name"] = task_name
+
+            command = task.get("command")
+            for var_name, var_value in variables.items():
+                placeholder = f"{{{{ {var_name} }}}}"  # Create the placeholder string like "{{ host1 }}"
+                command = command.replace(placeholder, var_value)
+
+            task["command"] = command
+
+        return meshbook
+    
+    @staticmethod
+    async def compile_group_list(session: meshctrl.Session) -> dict:
+        '''
+        Function that retrieves the devices from MeshCentral and compiles it into a efficient list.
+        '''
+
+        devices_response = await session.list_devices(details=False, timeout=10)
+
+        local_device_list = {}
+        for device in devices_response:
+            if device.meshname not in local_device_list:
+                local_device_list[device.meshname] = []
+
+            local_device_list[device.meshname].append({
+                "device_id": device.nodeid,
+                "device_name": device.name,
+                "device_os": device.os_description,
+                "device_tags": device.tags,
+                "reachable": device.connected
+            })
+
+        return local_device_list

os_categories.json

@@ -0,0 +1,27 @@
+
+{
+    "Linux": {
+        "Debian": [
+            "Debian GNU/Linux 13 (trixie)",
+            "Debian GNU/Linux 12 (bookworm)",
+            "Debian GNU/Linux 11 (bullseye)",
+            "Debian GNU/Linux 10 (buster)"
+        ],
+        "Ubuntu": [
+            "Ubuntu 24.04.3 LTS",
+            "Ubuntu 22.04.5 LTS",
+            "Ubuntu 20.04.6 LTS"
+        ]
+    },
+    "MacOS": {
+        "Sequoia": [
+            "macOS 15.0.1"
+        ]
+    },
+    "Windows": {
+        "11": [
+            "Microsoft Windows 11 Home - 24H2/26100",
+            "Microsoft Windows 11 Pro - 24H2/26100"
+        ]
+    }
+}

requirements.txt

@@ -0,0 +1,4 @@
+colorama==0.4.6
+pyyaml==6.0.3
+libmeshctrl==1.3.2
+pyotp==2.9.0

templates/api.conf.template

@@ -1,4 +1,5 @@
-[meshcentral-service]
+[meshcentral-account]
-websocket_url =
+hostname =
 username = 
-password =
+password =
+totp_secret =