Removed print line for aesthetic

Made grace-period countdown more verbose.

.gitignore

@@ -2,6 +2,9 @@
 venv
 .vscode
 
+# temporary or to prevent big commits
+examples/
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

README.md

@@ -3,15 +3,15 @@
 
 # Meshbook
 
-A way to programmatically manage MeshCentral-managed machines, a bit like Ansible does.<br>
-What problem does it solve? Well, what I wanted to be able to do is to automate system updates through [MeshCentral](https://github.com/ylianst/meshcentral).<br>
+A way to programmatically manage MeshCentral-managed machines, inspired by applications like [Ansible](https://github.com/ansible/ansible).<br>
+What problem does it solve? Well, what I wanted to be able to do is to automate system updates through [MeshCentral](https://github.com/ylianst/meshcentral). And some machines are behind unmanaged or 3rd party managed firewalls.<br>
 And many people will be comfortable with YAML configurations! It's almost like JSON, but different!<br>
 
 # Quick-start:
 
 The quickest way to start is to grab a template from the templates folder in this repository.<br>
-Make sure to correctly pass the MeshCentral websocket API as `wss://<MeshCentral-Host>/control.ashx`.<br>
-And make sure to fill in the credentails of an account which has `Remote Commands` permissions and `Device Details` permissions on the targeted devices or groups.<br>
+Make sure to correctly pass the MeshCentral websocket API as `wss://<MeshCentral-Host>`.<br>
+And make sure to fill in the credentails of an account which has `Remote Commands` permissions on the targeted devices or groups.<br>
 
 > I did this through a "Global Service" group which I added the meshbook account to!
 
@@ -25,7 +25,7 @@ git clone https://github.com/daanselen/meshbook
 cd ./meshbook
 python3 -m venv ./venv
 source ./venv/bin/activate
-pip3 install -r ./meshbook/requirements.txt
+pip3 install -r ./requirements.txt
 ```
 
 ### Windows setup:
@@ -35,22 +35,22 @@ git clone https://github.com/daanselen/meshbook
 cd ./meshbook
 python3 -m venv ./venv
 .\venv\Scripts\activate # Make sure to check the terminal prefix.
-pip3 install -r ./meshbook/requirements.txt
+pip3 install -r ./requirements.txt
 ```
 
-Now copy the configuration template from ./templates and fill it in with the correct details. The url should start with `wss://` and end in `control.ashx`.<br>
+Now copy the configuration template from ./templates and fill it in with the correct details. The url should start with `wss://`.<br>
 After this you can use meshbook, for example:
 
 ### Linux run:
 
 ```shell
-python3 .\meshbook\meshbook.py -pb .\examples\echo.yaml
+python3 .\meshbook.py -pb .\examples\echo.yaml
 ```
 
 ### Windows run:
 
 ```shell
-.\venv\Scripts\python.exe .\meshbook\meshbook.py -pb .\examples\echo.yaml
+.\venv\Scripts\python.exe .\meshbook.py -pb .\examples\echo.yaml
 ```
 
 ### How to check if everything is okay?
@@ -71,7 +71,7 @@ This paragraph explains how the program interprets certain information.
 
 ### Targeting:
 
-MeshCentral has `meshes` or `groups`, in this program they are called `companies`. Because of the way I designed this.<br>
+MeshCentral has `meshes` or `groups`, in this program they are called `group(s)`. Because of the way I designed this.<br>
 So to target for example a mesh/group in MeshCentral called: "Nerthus" do:
 
 > If your group has multiple words, then you need to use `"` to group the words.
@@ -79,7 +79,7 @@ So to target for example a mesh/group in MeshCentral called: "Nerthus" do:
 ```yaml
 ---
 name: example configuration
-company: "Nerthus"
+group: "Nerthus"
 variables:
   - name: var1
     value: "This is the first variable"
@@ -88,13 +88,13 @@ tasks:
     command: 'echo "{{ var1 }}"'
 ```
 
-It is also possible to target a single device, as seen in: [here](./examples/echo.yaml).<br>
+It is also possible to target a single device, as seen in: [here](./examples/apt_update_example.yaml).<br>
 
 ### Variables:
 
-Variables are done by replacing the placeholders just before the runtime.<br>
+Variables are done by replacing the placeholders just before the runtime (the Python program does this, not you).<br>
 So if you have var1 declared, then the value of that declaration is placed wherever it finds {{ var1 }}.<br>
-This is done to imitate popular methods. See below [from the example](./examples/variable_example.yaml).<br>
+This is done to imitate popular methods. See below [from the example](./examples/variable_usage_example.yaml).<br>
 
 ### Tasks:
 
@@ -102,119 +102,72 @@ The tasks you want to run should be contained under the `tasks:` with two fields
 The name field is for the user of meshbook, to clarify what the following command does in a summary.<br>
 The command field actually gets executed on the end-point.<br>
 
+### Granual Operating System control:
+
+I have made the program so it can have a basic filter with the Operating systems. If you have a mixed group, then you need to match the image below like this:
+
+<img src="./assets/meshcentral_os.png" alt="MeshCentral Operating System Version" width="600"/><br>
+<img src="./assets/meshbook_yaml_os.png" alt="MeshBook yaml example" width="600"/>
+
+This will filter the nodes/machines in the MeshCentral group to the ones matching this. Very basic and open for feedback. This must match the full string, not case sensitive.
+
 # Example:
 
 For the example, I used the following yaml file (you can find more in [this directory](./examples/)):
 
-The below group: `Temp-Agents` has four devices, of which one is offline.<br>
+The below group: `Dev` has three devices, of which one is offline, Meshbook checks if the device is reachable.<br>
 You can expand the command chain as follows:<br>
 
 ```yaml
 ---
-name: Ping Multiple Points
-company: Temp-Agents
+name: Echo a string to the terminal through the meshbook example.
+group: "Dev"
 variables:
-  - name: host1
-    value: "1.1.1.1"
-  - name: host2
-    value: "9.9.9.9"
-  - name: command1
-    value: "ping"
-  - name: cmd_arguments
-    value: "-c 4"
+  - name: file
+    value: "/etc/os-release"
 tasks:
-  - name: Ping host1
-    command: "{{ command1 }} {{ host1 }} {{ cmd_arguments }}"
-
-  - name: Ping host2
-    command: "{{ command1 }} {{ host2 }} {{ cmd_arguments }}"
+  - name: Echo!
+    command: "echo $(cat {{ file }})"
 ```
 
-The following response it received when executing the first yaml of the above files (with the `-s` and the `-i` parameters).
+The following response it received when executing the first yaml of the above files (without the `-s` parameters, which just outputs the below JSON).
 
 ```shell
-python3 meshbook/meshbook.py -pb examples/variable_example.yaml -si
--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=-
-Running task: {'name': 'Ping host1', 'command': 'ping 1.1.1.1 -c 4'}
--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=-
-Current Batch: 1
-Current response number: 1
-Current Calculation: 1 % 3 = 1
-Current Batch: 1
-Current response number: 2
-Current Calculation: 2 % 3 = 2
-Current Batch: 1
-Current response number: 3
-Current Calculation: 3 % 3 = 0
--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=-
-Running task: {'name': 'Ping host2', 'command': 'ping 9.9.9.9 -c 4'}
--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=-
-Current Batch: 2
-Current response number: 4
-Current Calculation: 4 % 3 = 1
-Current Batch: 2
-Current response number: 5
-Current Calculation: 5 % 3 = 2
-Current Batch: 2
-Current response number: 6
-Current Calculation: 6 % 3 = 0
--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=--=-
+python3 meshbook.py -pb examples/echo_example.yaml
+----------------------------------------
+Trying to load the MeshCentral account credential file...
+Trying to load the Playbook yaml file and compile it into something workable...
+Connecting to MeshCentral and establish a session using variables from previous credential file.
+Generating group list with nodes and reference the targets from that.
+----------------------------------------
+Executing playbook on the targets.
+1. Running: Echo!
+----------------------------------------
 {
-    "Batch 1": [
+    "Task 1": [
         {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=6.73 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=6.37 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=59 time=6.31 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=59 time=6.44 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms\nrtt min/avg/max/mdev = 6.312/6.461/6.727/0.159 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "MSI"
+            "complete": true,
+            "result": "PRETTY_NAME=\"Debian GNU/Linux 12 (bookworm)\" NAME=\"Debian GNU/Linux\" VERSION_ID=\"12\" VERSION=\"12 (bookworm)\" VERSION_CODENAME=bookworm ID=debian HOME_URL=\"https://www.debian.org/\" SUPPORT_URL=\"https://www.debian.org/support\" BUG_REPORT_URL=\"https://bugs.debian.org/\"\n",
+            "command": "echo $(cat /etc/os-release)",
+            "device_id": "<Node-Unique>",
+            "device_name": "raspberrypi5"
         },
         {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=6.18 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=6.17 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=6.17 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=6.27 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3004ms\nrtt min/avg/max/mdev = 6.170/6.200/6.274/0.042 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "raspberrypi5"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=6.33 ms\n64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=6.13 ms\n64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=5.92 ms\n64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=5.91 ms\n\n--- 1.1.1.1 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 5.908/6.072/6.334/0.173 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "server"
-        }
-    ],
-    "Batch 2": [
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.\n64 bytes from 9.9.9.9: icmp_seq=1 ttl=61 time=10.4 ms\n64 bytes from 9.9.9.9: icmp_seq=2 ttl=61 time=9.96 ms\n64 bytes from 9.9.9.9: icmp_seq=3 ttl=61 time=9.83 ms\n64 bytes from 9.9.9.9: icmp_seq=4 ttl=61 time=9.96 ms\n\n--- 9.9.9.9 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 9.830/10.036/10.396/0.214 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "MSI"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.\n64 bytes from 9.9.9.9: icmp_seq=1 ttl=60 time=10.8 ms\n64 bytes from 9.9.9.9: icmp_seq=2 ttl=60 time=10.6 ms\n64 bytes from 9.9.9.9: icmp_seq=3 ttl=60 time=10.5 ms\n64 bytes from 9.9.9.9: icmp_seq=4 ttl=60 time=10.5 ms\n\n--- 9.9.9.9 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3005ms\nrtt min/avg/max/mdev = 10.450/10.593/10.773/0.118 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "raspberrypi5"
-        },
-        {
-            "action": "msg",
-            "type": "runcommands",
-            "result": "PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.\n64 bytes from 9.9.9.9: icmp_seq=1 ttl=59 time=10.8 ms\n64 bytes from 9.9.9.9: icmp_seq=2 ttl=59 time=10.6 ms\n64 bytes from 9.9.9.9: icmp_seq=3 ttl=59 time=10.9 ms\n64 bytes from 9.9.9.9: icmp_seq=4 ttl=59 time=10.7 ms\n\n--- 9.9.9.9 ping statistics ---\n4 packets transmitted, 4 received, 0% packet loss, time 3006ms\nrtt min/avg/max/mdev = 10.600/10.750/10.898/0.117 ms\n",
-            "responseid": "meshctrl",
-            "nodeid": "server"
+            "complete": true,
+            "result": "PRETTY_NAME=\"Debian GNU/Linux 12 (bookworm)\" NAME=\"Debian GNU/Linux\" VERSION_ID=\"12\" VERSION=\"12 (bookworm)\" VERSION_CODENAME=bookworm ID=debian HOME_URL=\"https://www.debian.org/\" SUPPORT_URL=\"https://www.debian.org/support\" BUG_REPORT_URL=\"https://bugs.debian.org/\"\n",
+            "command": "echo $(cat /etc/os-release)",
+            "device_id": "<Node-Unique>",
+            "device_name": "Cubic"
         }
     ]
 }
-All tasks completed successfully: Expected 6 Received 6
 ```
-The above with `-si` is quite verbose. use `--help` to read about parameters.
+The above without `-s` is quite verbose. use `--help` to read about parameters and getting a minimal response for example.
 
 # Important Notice:
 
 If you want to use this, make sure to use `NON-BLOCKING` commands. MeshCentral does not work if you send it commands that wait.<br>
-A couple examples of `BLOCKING COMMANDS` which will never get back to the main MeshCentral server:
+A couple examples of `BLOCKING COMMANDS` which will never get back to the main MeshCentral server, and Meshbook will quit after the timeout but the agent will not come back:
 
 ```shell
 apt upgrade # without -y.

SECURITY.md

@@ -4,7 +4,7 @@
 
 | Version | Supported          |
 | ------- | ------------------ |
-| < 1.0   | :white_check_mark: |
+| >= 1.0   | :white_check_mark: |
 
 ## Reporting a Vulnerability
 

examples/aggregate_example.yaml

@@ -1,6 +1,6 @@
 ---
 name: Echo some text in the terminal of the device
-group: "Kubernetes"
+group: "Dev"
 variables:
   - name: package_manager
     value: "apt"
@@ -26,6 +26,6 @@ tasks:
 
   - name: Ping Google DNS
     command: "ping {{ google_dns }} -c 4"
- 
+
   - name: Ping Quad9 DNS
-    command: "ping {{ quad9_dns }} -c 4"
+    command: "ping {{ quad9_dns }} -c 4"

examples/apt_update_example.yaml

@@ -1,6 +1,6 @@
 ---
 name: Refresh the apt cache
-group: "Temp-Agents"
+device: Cubic
 variables:
   - name: package_manager
     value: "apt"

examples/echo.yaml

@@ -1,13 +0,0 @@
----
-name: Echo some text in the terminal of the device
-group: "Kubernetes"
-variables:
-  - name: var1
-    value: "Testing"
-  - name: var2
-    value: "with a slash!"
-tasks:
-  - name: Echo!
-    command: "echo {{ var1 }}/{{ var2 }}"
-  - name: Echo 2!
-    command: "echo womp womp"

examples/echo_example.yaml

@@ -0,0 +1,10 @@
+---
+name: Echo a string to the terminal through the meshbook example.
+group: "Dev"
+target_os: Debian GnU/Linux 12 (bookworm)
+variables:
+  - name: file
+    value: "/etc/os-release"
+tasks:
+  - name: Echo!
+    command: "echo $(cat {{ file }})"

meshbook.py

@@ -47,7 +47,7 @@ async def init_connection(credentials: dict) -> meshctrl.Session:
     await session.initialized.wait()
     return session
 
-async def translate_id_to_name(target_id: str) -> str:
+async def translate_id_to_name(target_id: str, group_list: dict) -> str:
     for group in group_list:
         for device in group_list[group]:
             if device["device_id"] == target_id:
@@ -79,7 +79,7 @@ Creation and compilation of the MeshCentral nodes list (list of all nodes availa
 
 async def compile_group_list(session: meshctrl.Session) -> dict:
     devices_response = await session.list_devices(details=False, timeout=10)
-    
+
     local_device_list = {}
     for device in devices_response:
         if device.meshname not in local_device_list:
@@ -92,10 +92,9 @@ async def compile_group_list(session: meshctrl.Session) -> dict:
             "device_tags": device.tags,
             "reachable": device.connected
         })
-
     return local_device_list
 
-async def gather_targets(playbook: dict) -> dict:
+async def gather_targets(playbook: dict, group_list: dict) -> dict:
     target_list = []
 
     if "device" in playbook and "group" not in playbook:
@@ -104,7 +103,10 @@ async def gather_targets(playbook: dict) -> dict:
         for group in group_list:
             for device in group_list[group]:
                 if device["reachable"] and pseudo_target == device["device_name"]:
-                    target_list.append(device["device_id"])
+                    if "target_os" in playbook and str(playbook["target_os"]).lower() == str(device["device_os"]).lower():
+                        target_list.append(device["device_id"])
+                    elif "target_os" not in playbook:
+                        target_list.append(device["device_id"])
 
     elif "group" in playbook and "device" not in playbook:
         pseudo_target = playbook["group"]
@@ -113,26 +115,32 @@ async def gather_targets(playbook: dict) -> dict:
             if pseudo_target == group:
                 for device in group_list[group]:
                     if device["reachable"]:
-                        target_list.append(device["device_id"])
+                        if "target_os" in playbook and str(playbook["target_os"]).lower() == str(device["device_os"]).lower():
+                            target_list.append(device["device_id"])
+                        elif "target_os" not in playbook:
+                            target_list.append(device["device_id"])
 
     return target_list
 
-async def execute_playbook(session: meshctrl.Session, targets: dict, playbook: dict):
+async def execute_playbook(session: meshctrl.Session, targets: dict, playbook: dict, group_list: dict) -> None:
     responses_list = {}
     round = 1
     for task in playbook["tasks"]:
         output_text(("\033[1m\033[92m" + str(round) + ". Running: " + task["name"] + "\033[0m"), False)
-        response = await session.run_command(nodeids=targets, command=task["command"], timeout=300)
+        response = await session.run_command(nodeids=targets, command=task["command"], ignore_output=False, timeout=900)
         
         task_batch = []
         for device in response:
             device_result = response[device]["result"]
             response[device]["result"] = device_result.replace("Run commands completed.", "")
             response[device]["device_id"] = device
-            response[device]["device_name"] = await translate_id_to_name(device)
+            response[device]["device_name"] = await translate_id_to_name(device, group_list)
             task_batch.append(response[device])
 
-        responses_list[task["name"]] = task_batch
+        responses_list["Task " + str(round)] = {
+            "task_name": task["name"],
+            "data": task_batch
+        }
         round += 1
     
     output_text(("-" * 40), False)
@@ -142,7 +150,7 @@ async def main():
     parser = argparse.ArgumentParser(description="Process command-line arguments")
     parser.add_argument("-pb", "--playbook", type=str, help="Path to the playbook file.", required=True)
     parser.add_argument("--conf", type=str, help="Path for the API configuration file (default: ./api.conf).", required=False)
-    parser.add_argument("--noout", action="store_true", help="Makes the program not output response data.", required=False)
+    parser.add_argument("--nograce", action="store_true", help="Disable the grace 3 seconds before running the playbook.", required=False)
     parser.add_argument("-s", "--silent", action="store_true", help="Suppress terminal output", required=False)
 
     global args
@@ -162,16 +170,22 @@ async def main():
         session = await init_connection(credentials)
     
         output_text(("\x1B[3mGenerating group list with nodes and reference the targets from that.\x1B[0m"), False)
-        global group_list
         group_list = await compile_group_list(session)
-        targets_list = await gather_targets(playbook)
+        targets_list = await gather_targets(playbook, group_list)
 
-        output_text(("-" * 40), False)
         if len(targets_list) == 0:
             output_text(("\033[91mNo targets found or targets unreachable, quitting.\x1B[0m"), True)
         else:
-            output_text(("\033[91mExecuting playbook on the targets.\x1B[0m"), False)
-            await execute_playbook(session, targets_list, playbook)
+            output_text(("-" * 40), False)
+            target_name = playbook["group"] if "group" in playbook else playbook["device"] # Quickly get the name.
+            output_text(("\033[91mExecuting playbook on the target(s): " + target_name + ".\x1B[0m"), False)
+            if not args.nograce:
+                output_text(("\033[91mInitiating grace-period...\x1B[0m"), False)
+                for x in range(3):
+                    output_text(("\033[91m{}...\x1B[0m".format(x+1)), False)
+                    await asyncio.sleep(1)
+            output_text(("-" * 40), False)
+            await execute_playbook(session, targets_list, playbook, group_list)
 
         await session.close()
 

meshbook/requirements.txt

@@ -1,5 +0,0 @@
-argparse
-asyncio
-configparser
-pyyaml
-libmeshctrl

requirements.txt

@@ -0,0 +1,4 @@
+asyncio==3.4.3
+configparser==7.1.0
+pyyaml==6.0.2
+libmeshctrl==1.1.0