Guide for setup of a guest Libvirt/QEMU Virtual Machine (VM). Includes general overview and references, and optimizations for Windows guests and hardware-passthrough (VFIO).
Auto X.Org | Deploy VFIO | Generate Evdev | Libvirt Hooks | Power State Virtual Machine Manager
- Why?
- Host Optimizations
- Guest Optimizations
- Guest XML Layout
- Benchmarking Guest Performance
- References
- Contact
The purpose of this document is to inform a new or returning user how to optimize a Guest machine, without demanding greater research and time.
This document does not serve to replace existing knowledge-bases. If you have any unexpected questions, wish to fact-check, or want to expand your knowledge, then please visit these places!
Copy and paste what you need from here and/or any example XML files, to your Guest XML file.
TODO: add here.
TODO: add here.
Below is an incomplete layout for building a Guest machine. The lines include additional features, of which are absent when creating a Guest XML (with the virsh
CLI command or virt-manager
GUI application).
<parent_tag_name attribute_name="attribute_value">
<child_tag_name>child_tag_value</child_tag_name>
</parent_tag_name>
<domain> Tag |
Attribute | Value | Description |
---|---|---|---|
xmlns:qemu |
"http://libvirt.org/schemas/domain/qemu/1.0" |
Enable QEMU command lines and overrides. | |
type |
"kvm" |
Enable QEMU command lines and overrides. | |
<name/> 1 |
none | text | Name of the Guest. |
Note: The following formatting examples are a personal preference of the Author.
Format: purpose
_vendor
*_operating system
_architecture
_chipset
_firmware
_topology
* Optional, if Host machine contains two (2) or more video devices (GPU/VGA).
- Example systems and names:
- Modern gaming machine:
game_nvidia_win10_x64_q35_uefi_6c12t
- Older 2000s gaming machine:
retro_amd_winxp_x86_i440fx_bios_2c4t
- Retro 1990s gaming machine:
retro_3dfx_win98_x86_i440fx_bios_1c1t
- Intel MacOS workstation:
work_macos_amd_x64_q35_uefi_6c12t
- Modern gaming machine:
- Purpose of the Guest (and suggested names):
- Gaming PC:
game
- Legacy/Retro PC:
retro
- Server:
server
- Workstation PC:
work
- etc.
- Gaming PC:
- Vendor name of the Video device:
- AMD:
amd
- Intel:
intel
- NVIDIA:
nvidia
- emulated:
virtgpu
- non-mainstream or legacy:
- 3DFX:
3dfx
.
- 3DFX:
- AMD:
- Short name of the Operating System (OS):
- Apple Macintosh:
macos
- Linux:
arch
,debian
,redhat
,ubuntu
- Microsoft Windows:
win98
,winxp
,win10
- etc.
- Apple Macintosh:
- Short name of the CPU architectureref:
- AMD/Intel 32-bit:
x86
- AMD/Intel 64-bit:
x64
- ARM 32-bit:
aarch32
- ARM 64-bit:
aarch64
- etc.
- AMD/Intel 32-bit:
- Virtualized chipsetref:
- I440FX:
i440fx
- Emulated, older chipset by Intel.
- Does support legacy Guests (example: Windows NT 5 and before, XP/2000, 9x).
- PCI bus only; expansion devices will exist on a PCI topology.
- Will accept PCIe devices.
- Q35:
q35
- Virtual, newer platform.
- Native PCIe bus; expansion devices will exist on a PCIe topology.
- Does not support legacy Guests.
- I440FX:
- Firmwareref:
- BIOS:
bios
- UEFI:
uefi
- BIOS:
- Short-hand of Core topology:
4c8t
- Given the amount of physical cores (example: 4).
- Given the amount of logical threads per core (2 * 4 = 8).
To gather information of system memory, execute: free --kibi --total --wide
<memory> Tag |
Attribute | Value | Description |
---|---|---|---|
parent | a number | The total amount of memory. | |
unit |
"KiB" |
Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes). |
<currentMemory> Tag |
Attribute | Value | Description |
---|---|---|---|
parent | a number | The current amount of memory. | |
unit |
"KiB" |
Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes). |
<memoryBacking> Tag |
Attribute | Value | Description |
---|---|---|---|
<allocation/> |
mode |
immediate |
Specifies how memory allocation is performed. |
<discard/> |
none | none | TODO: define, what is this? |
<hugepages/> 1 |
none | none | Enable Huge memory pages. |
<nosharepages/> |
none | none | Prevents the Host from merging the same memory used among Guests. |
- Static allocation of Host memory pages into Guest memory pages.
- Huge: Memory page size greater than 4K bytes (2M or 1G bytes). The greater the size, the lower the Host overhead.
- Dynamic Host memory page allocation is more flexible, but will require defragmentation before use as Guest memory pages (before a Guest machine may start).
- Warning: If the specified Guest memory pages exceeds the allocated Host memory pages, then the Guest machine will fail to start.
To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"
<vcpu> Tag |
Attribute | Value | Description |
---|---|---|---|
parent | a number | Specify number of Host threads for Guest (same as <cpu> tag). |
|
placement |
"static" |
Static allocation of Guest CPU threads. |
<iothreads> Tag |
Attribute | Value | Description |
---|---|---|---|
none | a number | Specify number of Host threads to manage storage block devices (see as <iothreadpin> tag). |
<cputune> Tag |
Attribute | Value | Description |
---|---|---|---|
<vcpupin> |
vcpu |
a number | Guest CPU: the Guest thread ID number.1 |
<vcpupin> |
cpuset |
a number | Guest CPU: the Host thread ID number.2 |
<emulatorpin> |
cpuset |
a number | Guest IRQ: the Guest IRQ thread ID number.3 |
<iothreadpin> |
iothread |
a number | Guest IO: the Guest IO thread ID number. |
<iothreadpin> |
cpuset |
a number | Guest IO: the Host thread ID numbers.4 |
- Count does not exceed value as defined in
<vcpu placement>
.
- Threads should not overlap Host process threads.
- Emulator threads handle Interrupt Requests for Guest hardware emulation.
- Threads should not overlap Guest CPU threads as defined in
vcpupin cpuset
.
- IO threads handle IO processes for Guest virtual drives/disks.
- Threads should not overlap Guest CPU threads as defined in
vcpupin cpuset
.
<!-- Given a 4-core, 8-thread CPU... -->
<vcpu placement="static">4</vcpu> <!-- Statically allocate four (4) cores to Guest. -->
<iothreads>1</iothreads> <!-- Define one (1) thread to IO. -->
<cputune>
<vcpupin vcpu="0" cpuset="2"/> <!-- Guest CPU: use the third core, first thread. -->
<vcpupin vcpu="1" cpuset="6"/> <!-- Guest CPU: use the third core, second thread. -->
<vcpupin vcpu="2" cpuset="3"/> <!-- Guest CPU: use the fourth core, first thread. -->
<vcpupin vcpu="3" cpuset="7"/> <!-- Guest CPU: use the fourth core, second thread. -->
<emulatorpin cpuset="1,4"/> <!-- Guest IRQ: use the second core, two threads. -->
<iothreadpin iothread="1" cpuset="1,4"/> <!-- Guest IO: use the second core, two threads. -->
</cputune>
To gather information about your BIOS, execute:
sudo dmidecode --type bios | grep --extended-regexp --ignore-case "vendor|version|release date"
To gather information about your system, execute:
sudo dmidecode --type system | grep --extended-regexp --ignore-case "manufacturer|product name|version|serial number|sku number|family"
<!-- BIOS and System spoofing (you may copy your actual info). -->
<sysinfo type="smbios"> <!-- This line is necessary! -->
<bios>
<entry name="vendor">American Megatrends Inc.</entry> <!-- AMI is the industry standard BIOS vendor. -->
<entry name="version">version_of_bios_firmware</entry>
<entry name="date">MM/DD/YYYY</entry>
</bios>
<system>
<entry name="manufacturer">vendor_of_motherboard</entry>
<entry name="product">product_name</entry>
<entry name="version">Default string</entry>
<entry name="serial">Default string</entry>
<entry name="sku">Default string</entry>
<entry name="family">Default string</entry>
</system>
</sysinfo>
TODO: make the following inline XML into chart, describe each feature.
<hyperv mode="custom">
<relaxed state="on"/>
<vapic state="on"/>
<spinlocks state="on" retries="8191"/>
<vpindex state="on"/>
<runtime state="on"/>
<synic state="on"/>
<stimer state="on">
<direct state="on"/>
</stimer>
<reset state="on"/>
<vendor_id state="on" value="1234567890ab"/>
<frequencies state="on"/>
<reenlightenment state="on"/>
<tlbflush state="on"/>
<ipi state="on"/>
<evmcs state="on"/>
</hyperv>
<kvm>
<hidden state="on"/>
</kvm>
<vmport state="off"/>
<ioapic driver="kvm"/>
</features>
To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
TODO: make the following inline XML into chart, describe each feature.
<cpu mode="host-passthrough" check="none" migratable="on"> <!-- Spoof the CPU info, with the actual CPU info. -->
<topology sockets="1" dies="1" cores="6" threads="2"/>
<cache mode="passthrough"/>
<feature policy="disable" name="hypervisor"/>
<feature policy="disable" present="yes"/>
<timer name="tsc" present="yes" mode="native"/>
</clock>
<cpu> Tag |
Attribute | Value | Description |
---|---|---|---|
parent | mode |
"host-passthrough" |
Spoof the Guest CPU info, with the actual Host CPU info. |
check |
"none" |
TODO: add here. | |
migratable |
"on" |
TODO: add here. | |
<topology> |
sockets |
a number | The number of CPU sockets, or maxmium number of dies. |
dies |
a number | The number of CPU dies (typically one). | |
cores |
a number | The number of CPU cores. | |
threads |
a number | The number of CPU threads per core (typically a factor of two). | |
<cache> |
mode |
"passthrough" |
Passthrough the Host CPU cache. |
<feature> |
policy |
"disable" |
Disables policy. |
name |
"hypervisor" |
TODO: add here. | |
<feature> |
policy |
"disable" |
Disables policy. |
present |
"yes" |
TODO: add here. | |
<timer> |
name |
"tsc" |
TODO: add here. |
present |
"yes" |
TODO: add here. | |
mode |
"native" |
TODO: add here. |
<!-- Power Management -->
<pm>
<suspend-to-mem enabled="yes"/> <!-- Enable S3 Suspend (Sleep) -->
<suspend-to-disk enabled="yes"/> <!-- Enable S4 Suspend (Hibernation) -->
</pm>
TODO: make the following inline XML into chart, describe each feature???
<!-- Emulated, Paravirtual, Passed-through Real PCI/e, and Shared Memory devices -->
<devices>
...
</devices>
TODO: make this inline XML?
- QEMU
- RedHat
- Other legacy devices
lorem ipsum
TODO: make this inline XML?
- AMD
- Integrated
- Dedicated
- Reset bug
- Intel
- Integrated
- Dedicated
- Note: Not much research done regarding this topic. Please refer to any mentioned guides, the Reddit forum, or use an Internet search engine with the keywords
intel gpu vfio
.
- Note: Not much research done regarding this topic. Please refer to any mentioned guides, the Reddit forum, or use an Internet search engine with the keywords
- NVIDIA
- Problems:
- Boot bug (Solution: Use known-good copy of given GPU's video BIOS or VBIOS).
- TODO: add references, programs used, instructions, and XML here.
- Problems:
TODO: make this inline XML?
- Looking Glass
- Scream
- ???
TODO: make the following inline XML into chart, describe each feature.
- Evdev
<qemu:commandline>...</qemu:commandline> <!-- Add Evdev here -->
TODO: make the following inline XML into chart, describe each feature.
<qemu:override>...</qemu:override>
TODO: add here.
I440FX | Wikipedia | QEMU documentation
PCI vs PCIe | RedHat documentation
Q35 | RedHat documentation
AArch32 | Wikipedia
AArch64 | Wikipedia
x86 | Wikipedia
x64 | Wikipedia
BIOS | Wikipedia
UEFI | Wikipedia
Huge memory pages | Arch Wiki | Debian Wiki
Memory Tuning on Virtual Machines | RedHat documentation
Do you need help? Find any information to be inadequate? Notice any dead links? Please contact by raising an issue with the project itself.
- add About.
- add Documentation for:
- guest XML layout
- name syntax
- introduction
- memory
- cpu topology (1/2)
- system information spoofing
- features
- cpu topology (2/2)
- power management
- qemu command line
- qemu overrides
- host optimizations
- guest optimizations
- benchmarking performance
- guest XML layout
- add References.
- use collapsable dropdowns (https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab).
- add Guest XML file with all lines referenced in "Guest XML Layout".
- add pictures of virt-manager.
- flesh out "Guest XML Layout".
- how to setup virtio disks (virtio, virtio-scsi).