Skill: Hypervisor Introspection (VMI) and Virtualization Escape

Supplementary Files:

• payloads.md — Attack payloads and commands organized by hypervisor family: VMware ESXi/vSphere, Hyper-V, KVM/QEMU, Xen, Proxmox, VirtualBox, VMI tooling (LibVMI/DRAKVUF/DECAF/PANDA), and hardware-assisted virtualization abuse
• test-cases.md — 12 structured test cases (TC-HI-001 through TC-HI-012) covering VMware MOB enumeration, ESXi SLP fingerprinting, QEMU QMP unauth access, KVM PCI passthrough auditing, LibVMI memory reads, DRAKVUF malware tracing, VENOM reproduction, Hyper-V VMWP enumeration, and more
• guides/hypervisor-introspection-playbook.md — Comprehensive playbook with hypervisor comparison matrix, VMI use cases, real-world incident analysis (VENOM, OpenSLP, ESXiArgs, Akira/Royal), lab setup for each hypervisor, and defensive guidance

Summary

Hypervisor Introspection skill domain covering virtualization layer operations.

Tools: VMware vSphere CLI (govc, pyvmomi, PowerCLI), ESXi Shell/DCUI/vMA, LibVMI/PyVMI, DRAKVUF, DECAF/PANDA, QEMU (QMP/HMP), virsh/libvirt, Xen xl/xm/xe, Hyper-V PowerShell, Proxmox VE qm/pct, Wireshark (VMware/VNC dissectors), Volatility (VM memory forensics), HyperPlatform/DdiMon

Domain: virtualization

MITRE ATT&CK: T1068 - Exploitation for Privilege Escalation

Description

Hypervisor introspection (VMI) and virtualization escape assessment covers the full virtualization stack: Type-1 bare-metal hypervisors (VMware ESXi/vSphere, Microsoft Hyper-V, KVM/QEMU, Xen, Proxmox VE, bhyve, AHV), Type-2 hosted hypervisors (VMware Workstation/Fusion, VirtualBox, Parallels), and the hardware-assisted virtualization primitives they rely on (Intel VT-x/EPT, AMD-V/RVI/NPT, ARM EL2). The discipline spans two complementary perspectives: offensive testing of escape paths from guest to host (CVE-driven device emulation flaws, paravirtualized backend bugs, host-passthrough misconfigurations, management plane abuse) and defensive introspection from outside the guest using Virtual Machine Introspection (VMI) tooling such as LibVMI, PyVMI, DRAKVUF, DECAF, and PANDA. The defender's perspective centers on stealthy malware analysis, kernel rootkit detection, and memory forensics performed from the hypervisor layer, where the observer cannot be tampered with by guest malware.

Mastering this skill requires deep fluency in CPU virtualization extensions (VMX root/non-root, SVM guest/host, EPT/NPT second-level address translation), hypervisor attack surfaces (QEMU device emulation, VMware VMX, Xen toolstack, Hyper-V VMWP worker processes), and the protocols used to control the management plane (vSphere REST API, MOB, QMP, libvirt, XenStore). The attacker's perspective focuses on chaining a guest-visible vulnerability into code execution in the hypervisor or host kernel (the actual "escape"), while the defender's perspective uses the hypervisor's privileged vantage point to inspect and contain malicious guests.

Differentiation

This domain is distinct from adjacent virtualization-themed skills:

• container-security covers Linux containers (Docker, containerd, podman) which share the host kernel via namespaces and cgroups. Containers are not full virtual machines — there is no separate kernel and no hardware-assisted isolation layer. Container "escape" is a namespace/cgroup boundary violation, whereas hypervisor "escape" requires breaking out of a hardware-enforced CPU virtualization container. The two skills share some conceptual overlap (both are isolation boundary failures leading to host compromise) but the attack primitives, defensive tooling, and CPU-level primitives are entirely different. A container running as root can still be introspected from the host trivially (nsenter, /proc/<pid>/root); a VM requires VMI tooling to inspect without the guest's cooperation.
• kubernetes-attack covers the orchestration layer above containers — the API server, etcd, RBAC, scheduler, and controllers. Kubernetes runs containers (typically via containerd/CRI-O on top of containerd/runc), not VMs, although projects like KubeVirt and Kata Containers bridge the two by running VMs as Kubernetes workloads. When assessing KubeVirt or Kata, this skill informs the VM-layer analysis while kubernetes-attack informs the orchestration-layer analysis; the two are complementary, not redundant.
• cloud-security covers CSP-managed IaaS/PaaS/SaaS and the cloud control plane (IAM, metadata services, object storage). Public-cloud hypervisors (AWS Nitro, Azure Hyper-V, Google GCE's KVM) are typically out of scope for direct testing because the CSP hardens and operates them; this skill focuses on customer-managed hypervisors (private vSphere, on-prem Hyper-V, KVM clusters, Proxmox VE) where the customer owns and is responsible for the hypervisor security posture.
• firmware-reverse and hardware-security cover pre-OS firmware and physical hardware, respectively. Hardware-assisted virtualization primitives (VT-x, AMD-V) are adjacent to hardware-security but the attack patterns here are about abusing virtualization features (e.g., deploying a malicious hypervisor that subverts an existing one) rather than physical glitching or JTAG abuse.

---

Use Cases

1. VMware vSphere/ESXi Attack Surface Mapping — Enumerate the vSphere REST API, Managed Object Browser (MOB), SLP services, and SSH/DCUI surfaces; fingerprint build numbers and patch levels to identify missing fixes for OpenSLP CVE-2019-5544, CVE-2020-3992, CVE-2021-21974, and related ESXi issues
2. ESXi Ransomware Posture Assessment — Evaluate a vSphere cluster's exposure to the ESXiArgs, Akira, Royal, BlackCat, and Play ransomware families that specifically target VM hosts and encrypt datastores; verify lockdown mode, role-based access control, and network segmentation of the management network
3. Hyper-V Worker Process (VMWP) Auditing — Enumerate VMWP.exe instances, identify the vmbus, vid, and hvp device drivers, audit Live Migration and Replica traffic, and assess Shielded VM and Guarded Fabric configurations for bypass paths
4. KVM/QEMU Device Emulation Vulnerability Verification — Reproduce and verify mitigations for QEMU device emulation flaws including VENOM (CVE-2015-3456 floppy), CVE-2015-7504 PCNET, CVE-2020-14364 USB XHCI, CVE-2021-3947 e1000e, CVE-2023-3354 OBjective-C (jobs overflow), and virtio-ring issues
5. Xen DomU-to-Dom0 Privilege Escalation Testing — Audit Xen security advisories (XSA-148, XSA-182, XSA-242, XSA-253, XSA-344, XSA-392, and successors) against running guest configurations; test PV vs HVM guest isolation boundaries
6. Virtual Machine Introspection for Malware Analysis — Deploy LibVMI/PyVMI, DRAKVUF, DECAF, or PANDA against a target VM to extract kernel structures, walk process lists, and trace malware behavior from outside the guest — defeating in-guest anti-analysis techniques
7. Hypervisor-based Memory Forensics — Use HyperPlatform, DdiMon, Hunter, or Moneta on Windows to deploy a thin Intel VT-x hypervisor that monitors kernel memory writes, detects rootkits, and supports Volatility-based analysis of protected guests
8. Proxmox VE Cluster Auditing — Assess Proxmox VE (KVM + LXC on Debian) management interfaces (the pveproxy web UI on 8006), Corosync cluster communication, and the boundary between LXC containers and KVM guests running on the same host
9. Nested Virtualization Lab Setup — Build reproducible labs for vulnerability reproduction: nested ESXi on Workstation/Fusion, KVM on Linux with nested=1, Hyper-V on Hyper-V, or Xen-on-Xen, so that escape PoCs can be safely tested without risking bare-metal hosts
10. Hardware-assisted Virtualization Abuse Research — Reproduce BluePill (AMD64 SVM), Vitriol/SubVirt (Intel VT-x), HyperJack, and similar proof-of-concept malicious hypervisors that install themselves as a stealth layer beneath an existing OS; evaluate EPT-based memory hiding and SMBus-based side channels
11. VM Memory Forensics with Volatility — Capture guest RAM via hypervisor-level snapshots (vmware-vmx checkpoint, QEMU pmemsave, virsh dump, Hyper-V Save-VM) and process with Volatility 3 for process tree, network connections, and malware extraction without ever touching the guest OS
12. Shielded VM and Guarded Fabric Bypass Testing — For Hyper-V Shielded VMs and guarded fabrics, assess whether the attestation flow (Host Guardian Service, TPM-based measured boot, virtualization-based security) can be subverted via administrative compromise of the fabric or via the VM worker process

---

Core Tools

Tool	Purpose	Command Example
govc	Go-based vSphere CLI for ESXi/vCenter management and enumeration	govc about -u user:pass@vc.example.com
pyvmomi	Python SDK for vSphere API; programmatic MOB traversal	python3 -c "from pyVim.connect import SmartConnect; ..."
PowerCLI	PowerShell module for VMware automation (Windows hosts)	Get-VM | Select Name,PowerState
ESXi Shell / DCUI	Direct shell on ESXi host; DCUI = Direct Console User Interface	vim-cmd vmsvc/getallvms
LibVMI / PyVMI	C library and Python bindings for VM introspection (Xen/KVM/QEMU)	pyvmi -d win7 -n pslist
DRAKVUF	Dynamic malware analysis system built on LibVMI/Xen	drakvuf -r kernel.json -d win7malware -e malware.exe
DECAF / PANDA	QEMU-based dynamic analysis platforms with taint and plugin support	panda-system-x86_64 -os windows-32-7 -mem 2G
QEMU (QMP/HMP)	Machine emulator and virtualizer with QEMU Monitor Protocol (JSON) and Human Monitor	qemu-system-x86_64 -qmp tcp:127.0.0.1:4444,server,nowait
virsh / libvirt	Generic virtualization API and CLI for KVM/QEMU/Xen/LXC	virsh list --all; virsh dump win7 win7.dump
Xen xl / xm / xe	Xen toolstack (xl is the modern LBL tool; xe for XenServer/XCP)	xl list; xl info
Hyper-V PowerShell	Hyper-V module (Get-VM, Get-VHD, Get-VMNetworkAdapter)	Get-VM | Format-Table Name,State,CPUUsage
Proxmox qm / pct	Proxmox VE CLI for KVM (qm) and LXC (pct) guests	qm list; pct list
Wireshark (VMware/VNC)	Network analyzer with dissectors for VNC, VMware VMotion, and CIFS used by Hyper-V Live Migration	tshark -i vmnet8 -Y vnc
Volatility 3	Memory forensics framework for VM RAM dump analysis	vol -f win7.dump windows.pslist
HyperPlatform / DdiMon	Intel VT-x research hypervisors for kernel memory monitoring on Windows	HyperPlatform.exe -driver:MyDriver

The 13 tools enumerated in the YAML metadata map directly to the rows above (govc/pyvmomi/PowerCLI are listed as a single "VMware vSphere CLI" family per the spec).

---

Methodology

Attack Chain

   Management Plane         Guest Kernel         Device Emulation        Hypervisor           Host / Sibling VMs
   Enumeration              Reconnaissance       Vulnerability           Compromise           Lateral Movement
   (vSphere API, MOB,       (Detect VMware       (QEMU device,           (RCE in vmx,         (vmkloadapp,
    SLP, libvirt, Xen       Tools, Hyper-V       virtio, paravirt        qemu-system,          vmktools, host
    Store, Proxmox 8006)    driver, /sys/hypervisor)  backend)              VMWP worker)         shell, datastore)
       |                       |                       |                       |                       |
       v                       v                       v                       v                       v
   Credentials               Guest-level            CVE-driven              Hypervisor             Datastore
   / Access Token             RCE Primitive          Escape Primitive        Root / Kernel          Encryption /
   (Lockdown bypass,          (Web shell,            (VENOM, XSA,            Mode                   VM Image
    SSO abuse, vMA)            kernel exploit)        Hyper-V vmbus)                                  Manipulation

Phase Details:

1. Management Plane Enumeration — Map every externally reachable management surface: vCenter on 443 (SOAP/REST), ESXi hostd on 443 and MOB on /mob, OpenSLP on 427 (UDP), SSH on 22 (if enabled), DCUI on the physical console, libvirt on 16509 (TCP) or 22 (qemu+ssh), Proxmox pveproxy on 8006, XenServer API on 443. The management plane is the most common intrusion vector for ESXi ransomware crews.
2. Guest Kernel Reconnaissance — From inside the guest, detect the hypervisor (cpuid leaf 0x40000000, /sys/hypervisor/type, VMware Tools service, Hyper-V integration services, QEMU virtio PCI device IDs). This informs escape strategy: PV drivers often expose more attack surface than emulated hardware.
3. Device Emulation Vulnerability Discovery — Map guest-visible emulated devices (PCI IDs, I/O ports, MMIO regions) to known CVEs. QEMU's floppy, PCNET, E1000, USB XHCI, NVMe, and virtio backends have all had remotely triggerable flaws reachable from inside the guest.
4. Hypervisor Compromise (Escape) — Trigger the vulnerable code path from the guest to achieve code execution in the hypervisor process (qemu-system, vmware-vmx, VMWP) or in the kernel of the host (vmkernel, Linux host, Windows). At this point the attacker has crossed the virtualization boundary.
5. Host / Sibling VM Lateral Movement — From the hypervisor host, enumerate datastores (/vmfs/volumes on ESXi), manipulate VMDK/IMG files, deploy ransomware against all suspended VMs, or pivot into sibling VMs by editing their virtual disks offline.

Defense Perspective

• ESXi Lockdown Mode — Disable direct root SSH/DCUI access to ESXi hosts; all administration flows through vCenter with explicit role assignment. Lockdown mode "Strict" further restricts even vCenter's emergency access.
• Network Segmentation of Management Plane — vCenter, ESXi vmkernel networks, Hyper-V cluster networks, Proxmox Corosync, and Live Migration/VMotion networks must be on dedicated non-routable VLANs. ESXi ransomware crews scan the internet for exposed 443/427/22 and then ransom every VM in minutes.
• vTPM and Virtual Secure Boot (vSB) — Enable vTPM and vSB on every VM whose guest OS supports it. This prevents offline tampering with VMDKs (the attacker can still delete or encrypt, but cannot modify boot chain or bitlocker-sealed secrets in a way that survives reboot).
• Hyper-V Shielded VMs / Guarded Fabric — Shielded VMs run only on attested healthy hosts (Host Guardian Service attests measured boot). Shielded VM local admin cannot log into the VM worker process, cannot mount the VHDX read-write on a different host, and cannot read the VM's BitLocker key. The bypass target is the HGS itself, not the VM.
• Xen Security Modules (XSM-FLASK) — Xen XSM with FLASK policy enforces mandatory access control between domains, limiting what a compromised Dom0 or DomU can do to other domains. Most deployments leave XSM in "dummy" mode; switch to FLASK for defense in depth.
• QEMU Sandbox and Seccomp — Run qemu-system behind a seccomp filter (-sandbox on,obsolete=deny,elevateprivileges=deny) and as an unprivileged user. Combined with libvirt's dynamic_ownership and namespace isolation, this contains many device emulation flaws.
• Patch Velocity — Hypervisor CVEs frequently have public PoCs within weeks. Subscribe to VMware VMSA, Xen XSA, and QEMU security advisory feeds; patch within the SLA appropriate to the CVSS (typically 7 days for Critical in a datacenter).
• VMI-Based Detection — Deploy DRAKVUF or a LibVMI-based sensor to high-value guests so that in-guest malware is detected by out-of-guest introspection, even if the malware subverts the guest kernel itself.

---

Practical Steps

1. VMware vSphere / ESXi Attack Surface Mapping

Use govc, pyvmomi, and the MOB to enumerate vCenter objects, host configurations, and network exposures. Confirm whether lockdown mode is enabled, whether OpenSLP is reachable on UDP 427 (the CVE-2019-5544 / CVE-2020-3992 surface), and whether SSH is disabled on production hosts. Test default credentials against the SSO administrator account and against host local users (root).

2. QEMU / KVM Device Emulation Vulnerability Verification

Build a QEMU reproduction lab with debugging symbols. Reproduce VENOM (CVE-2015-3456) against an old floppy backend, then verify that a patched QEMU rejects the malformed commands. Repeat for PCNET (CVE-2015-7504), USB XHCI (CVE-2020-14364), and the e1000e out-of-bounds (CVE-2021-3947). For each, document the QEMU command line, the guest-side PoC, and the patched version threshold.

3. Hyper-V VMWP and VMBus Enumeration

From the parent partition (host), enumerate VMWP.exe worker processes, map each to its VM via Get-VM \| Select Name,Id, and inspect the loaded modules inside VMWP. Identify the vid.sys, vmbus.sys, and hvp.sys drivers and confirm patch level against MSRC bulletins. Audit the Live Migration and Replica configuration for plaintext or weakly authenticated transport.

4. Xen XSA Verification Lab

Stand up a Debian-based Xen host with both PV and HVM guests. Test XSA-148 (PV MMIO), XSA-182 (grant table issue), and XSA-242 (PV block backend) against unpatched vs patched dom0. Document whether XSM-FLASK was enabled by default and whether the XenStore daemon (dom0) is reachable from guest qemu-dp processes.

5. Virtual Machine Introspection Deployment

Install LibVMI on a Xen or KVM host, configure the VM with a known kernel (/etc/libvmi/<domain>.conf), and use PyVMI to walk the Active Process List of a Windows guest without ever logging into the guest. Scale up to DRAKVUF for full dynamic malware analysis: trap syscalls, log network activity, and extract dropped binaries without the malware knowing it is being observed.

6. Proxmox VE Cluster Audit

Enumerate the 8006 web UI, the Corosync cluster on UDP 5404/5405, the SPICE/VNC consoles on each VM's allocated port, and the LXC container boundary (/proc/<pid>/cgroup to distinguish KVM from LXC). Verify that no production cluster has pveproxy exposed to the internet and that no host's root password is shared across nodes.

7. Hypervisor-based Defense Tooling (HyperPlatform)

On a Windows host, install HyperPlatform and a custom kernel-mode driver that registers EPT violations on the page hosting nt!PsActiveProcessHead. The driver logs every process creation in real time and survives kernel rootkit attempts to unlink the process list, because the rootkit's own CR3 manipulation is caught by the EPT hook. Use DdiMon to monitor specific kernel functions (e.g., MmCopyVirtualMemory) for credential theft.

8. VM Memory Forensics Pipeline

Snapshot the guest via virsh dump <domain> out.dump --memory-only, qemu-monitor pmemsave, ESXi snapshot consolidation, or Hyper-V Save-VM. Then run Volatility 3 against the dump: vol -f out.dump windows.pslist, windows.netscan, windows.malfind. This is the canonical "deadbox" forensics path for a VM suspected of compromise.

Detailed payloads in payloads.md, complete test checklist in test-cases.md, and the end-to-end playbook in guides/hypervisor-introspection-playbook.md.

---

Detection Methods

Hypervisor Audit

• VM escape signatures: Guest accessing host resources (host filesystem, host processes).
• Hyperjacking: Unauthorized hypervisor layer; new hypervisor on bare metal.
• Nested virtualization abuse: Unauthorized nested VMs; new VMs spawning inside other VMs.

SIEM Detection Rules

• Splunk SPL: index=vmware sourcetype=vmkernel | where event_type="vmkernel_error" | search "VMKernel heartbeat"
• vSphere / Hyper-V audit logs: Monitor for VM tools manipulation, hypervisor console access.
• LibVMI / DRAKVUF: Runtime introspection for VM malware analysis.

Defense Evasion Techniques

VM Escape Stealth

• Target less-monitored hypervisors: KVM/Xen have less mature monitoring than ESXi.
• Use guest tools channels: Abuse VMware Tools / Hyper-V Integration Services channels; appears as legitimate traffic.
• Single-shot escape: One escape attempt per VM; below anomaly threshold.
• Memory-only payloads: No disk writes on host; survives forensic snapshot.

Hyperjacking Stealth

• Legitimate-looking hypervisor: Mimic ESXi/Hyper-V API surface; appears as legit management.
• VMI (Virtual Machine Introspection) bypass: Detect VMI agents (LibVMI, DRAKVUF); hide when introspection active.
• Snapshot manipulation: Modify VM snapshot to inject malicious state; affects restored VMs.

Common Pitfalls

• Treating the management plane as trusted: ESXi ransomware crews (Akira, Royal, BlackCat, Play, ESXiArgs) almost never exploit a hypervisor CVE — they simply walk through exposed vCenter/ESXi management interfaces using stolen or default credentials. A fully patched vSphere 8 cluster with lockdown disabled, SSH enabled, and SLP reachable on 427/UDP is far more likely to be ransomed than an unpatched cluster with proper management segmentation.
• Conflating containers with VMs: Container escape (a namespace/cgroup boundary violation) is conceptually similar to VM escape but the primitives, defensive tooling, and CPU-level mechanisms are entirely different. A "container escape" via /var/run/docker.sock mount has no analog in KVM/QEMU; a "VM escape" via VENOM has no analog in Docker. Always scope the assessment to the correct isolation boundary.
• Ignoring paravirtualized drivers: Virtio-net, virtio-blk, VMware VMXNET3/PVSCSI, Xen netback/blkback, and Hyper-V netvsc/storvsc are written for performance, not defense, and have been the source of multiple high-severity escape bugs. Auditors frequently focus on emulated legacy devices (floppy, PCNET) because they are well-known, while skipping the PV drivers that are actually enabled in production.
• Assuming VMI is undetectable: VMI via LibVMI/DRAKVUF is stealthy relative to in-guest instrumentation but not invisible. Single-step traps, EPT violations, and altered TSC offsets can be detected by a sufficiently paranoid guest. For high-assurance malware analysis, use DRAKVUF's trap absorption features and avoid single-stepping where possible.
• Skipping nested virtualization in labs: Reproducing a VM escape against a bare-metal production host can brick the host, crash sibling VMs, or worse. Always reproduce in a nested virtualization environment first (ESXi-on-Workstation, KVM-on-KVM with nested=1, Hyper-V-on-Hyper-V) and only then verify on bare metal in a sacrificial lab.
• Forgetting the host kernel is part of the boundary: For KVM and Xen, the host kernel (or Dom0) is part of the trust boundary. A Linux kernel LPE in the host (e.g., CVE-2024-1086 netfilter) often beats hypervisor CVEs as the more reliable escape path, because the attacker can escape the guest to QEMU, then LPE from qemu-system (running as an unprivileged user) to host root.

Automation and Scripting