Abiquo Documentation Cookies Policy

Our Documentation website uses cookies to improve your experience. Please visit our Cookie Policy page for more information about cookies and how we use them.


Documentation

Skip to end of metadata
Go to start of metadata

Oracle VM Overview

Abiquo 2.6 offers support for Oracle VM V3.2.1, enabling customers to use the Oracle VM hypervisor and taking advantage of Oracle VM features, and improving licensing conditions for their Oracle applications. This feature enables customers to make use of hard partitioning, so they only need to buy Oracle product licenses required for the virtual CPUs used.

 

Oracle VM Hypervisor Plugin Architecture

The Oracle VM hypervisor plugin architecture is shown in the following diagram. The plugin is integrated with three Abiquo remote services and the Abiquo API.

The main architectural issues to note are that:

  • Oracle VM support is provided by an Oracle VM Hypervisor ‘plugin’, which is used by three of the Abiquo remote services and the API
  • The Abiquo Oracle VM Hypervisor Plugin connects to the Oracle VM Manager via a SOAP-based client library
  • No direct access is permitted to the Oracle VM servers - all communication is with the Oracle VM Manager.
  • The Abiquo Oracle VM Hypervisor Plugin instructs the Oracle VM Manager to import Abiquo Template repository images directly from the Abiquo NFS Repository
  • The Abiquo Oracle VM Hypervisor Plugin creates an Oracle VM Storage repository within the Abiquo NFS repository ; this repository is required to create Abiquo VM Instances.

Limitations of Oracle VM in Abiquo

LimitationDescription
VersionAbiquo only supports Oracle VM 3.2.1 or later
HypervisorOracle/Xen only. No support for Oracle/SPARC
VNCNo remote access support because of Oracle security policy
iSCSI Storage devices

Before you use a storage device with Abiquo, verify that it works well with Oracle VM in the following:

  • Initial storage device discovery
  • LUN discovery
  • Detection of new LUNs created
Note that you may need to manually review LUNs to delete LUNs from Oracle VM after they are removed from the storage device.
iSCSI LUNs1000 LUN storage array limitation
TemplatesLimited support for PVM and PVHVM-based machines. Templates in the Abiquo repository are normally fully hardware virtualized (HVM). The Abiquo Oracle VM plugin currently only supports one type of VM at a time. By default it supports HVM VMs, however, PVM or PVHVM VMs can be configured instead
Oracle templates

Not supported but they may be used. Note that:

  • They are mostly of type PVM (see PVM/PVHVM limitation above).
  • An assembly/template can define a virtual machine with multiple disks, but Abiquo VM templates only allow one disk.
Oracle HAIf all Oracle VM servers are managed by Abiquo and exist in a clustered server pools, you may enable Oracle VM HA support within Abiquo.
DRS and DPS rulesIf all Oracle VM servers are managed by Abiquo and the server pools are all on the same Abiquo rack, you may use DRS (Distributed Resource Scheduling) and DPS (Distributed Power Management) rules. Note that they are not directly supported by Abiquo.
Overallocation of RAM

Do not create allocation rules to over-allocate memory because the Oracle/Xen hypervisor is based on Xen, which does not recognize memory over-allocation

NetworksIn Oracle VM, a VLAN interface is associated with only one network, and only one VLAN tag. It is NOT possible to have two different VLAN interfaces for the same VLAN tag on the same Ethernet port, or to have two different networks using the same VLAN interface. If Abiquo defines the same VLAN (on the same ethernet port) twice, then the two VLANs will have to share the same network definition. Abiquo can only check that the name of network associated with the VLAN is the expected one. However, this is not reliable enough, because the network can be renamed. The Abiquo Oracle VM hypervisor plugin does not have access to the network ID. 
Remote Service ConfigurationTo create Oracle VM based Instances, the Virtualization Manager Remote Service should be installed on the same machine as the Appliance Manager Remote Service (this is the default configuration for Remote Services installations). If the Virtualization Manager is installed onto a separate server, then you must ensure that the Abiquo repository is mounted locally (using NFS) on that server, and that the 'abiquo.appliancemanager.localRepositoryPath' in the abiquo.properties file correctly specifies that mount point.

Abiquo Support for Hard Partitioning

Hard Partitioning support is disabled by default within Abiquo. When enabled, Abiquo randomly allocates CPUs to a VM when it is created. Random allocation is a simple algorithm that should evenly distribute CPU pinning without requiring cross-machine synchronization or compromising the horizontal scalability of the platform.

Summary of Oracle VM Configuration

This section summarizes the configuration of Oracle VM for use with Abiquo, describing the aspects of Oracle VM configuration are performed automatically by Abiquo and those that require manual intervention.

Oracle VM configuration automated by Abiquo

Abqiuo automatically performs the following Oracle VM configuration:

  • ISCSI storage device discovery
  • Storage device and file server permissions
  • Network handling
  • Repository creation
Oracle VM configuration requiring manual intervention

The administrator must manually perform the following Oracle VM configuration:

  • Add VM servers with Utility Server and VM Server roles
    • Configure separate servers
  • Configure Server Pools
    • Ensure the VM Start Policy is correctly configured to Start on current server
    • Ensure HA is enabled at Server Pool level
      • Check VLAN Group configuration for HA

Configure Abiquo Properties

PropertyDefaultRangeDescription
abiquo.virtualfactory.oraclevm.repositoryLocation  NFS Repository location denoted by nfs-server-address:exportpath. Defaults to the value of 'abiquo.appliancemanager.repositoryLocation' if not explicitly specified in the abiquo.properties file
abiquo.virtualfactory.oraclevm.jobTimeoutMs2880000060000-604800000The maximum length of time the Oracle VM Hypervisor Plugin will wait for an individual API-based job to complete (milliseconds). The default is 8 hours but values can be from 1 minute to 1 week.
abiquo.virtualfactory.oraclevm.jobStatusCheckDelayMs1000100-60000Polling period for the job status check to monitor the status of asynchronous hypervisor jobs (milliseconds). The default is 1 second but values can be from 0.1 second to 1 minute.
abiquo.virtualfactory.oraclevm.hardPartitioningfalsetrue, falseEnable Oracle VM Hard Partitioning support (CPU pinning)
abiquo.virtualfactory.oraclevm.hardPartitioningCPURatio11-64

Expected ratio of virtual CPUs to Physical CPU cores (integer). For best performance, set to 1. To reduce physical CPUs per VM and thus licensing costs, set to a higher value. For example, a value of 1 means a 4xCPU VM will be pinned to 4 physical CPUs and a value of 4 means a 4xCPU VM will be pinned to one physical CPU

abiquo.virtualfactory.oraclevm.createSparseDiskstruetrue, falseCreate additional hard disks on the datastore as sparse.
abiquo.virtualfactory.oraclevm.defaultVMDomainHVMHVM, PVM, PVHVMCreate all Oracle VMs with this domain type. Options are  (Hardware Virtualized [HVM], Paravirtualized [PVM], Hardware Virtualized with Paravirtualized drivers [PVHVM])

abiquo.virtualfactory.oraclevm.ha

falsetrue, falseThis setting controls whether VMs created by Abiquo are configured within Oracle VM for ‘High Availability’.   The default value is ‘false’ (no HA support).   If set to ‘true’, then Abiquo will configure a VM for HA if the Oracle VM Server hosting the VM is part of a Clustered Server Pool.   Additionally, when a VM is re-configured, Abiquo will update the HA status of the VM depending upon whether this property is enabled and the host server is part of an Oracle VM clustered server pool.   Please note that some manual configuration of Oracle VM may be required for HA to be performed successfully - please see the section ‘Oracle VM functionality that requires Manual Configuration’ for additional information.

 

This setting controls the expected ratio of virtual CPUs to Physical CPU cores. For performance reasons, Oracle recommend this value to be '1', but it may be set to a higher value to reduce the number of physical CPUs used by a VM (which may help to reduce its licensing costs). For example: 

  • If this value is set to its default of 1 then a 4 CPU VM will be pinned to 4 Physical CPUs

  • If this value is set to a value of 2 then a 4 CPU VM will be pinned to 2 Physical CPUs

  • If this value is set to a value of 3 then a 4 CPU VM will be pinned to 3 Physical CPUs

  • If this value is set to a value of 4 then a 4 CPU VM will be pinned to 1 Physical CPUs

Add Oracle VM to Abiquo

Abiquo Network Configuration

For each Oracle VM server added to an Abiquo Rack, configure the Oracle VM Server Networks and correctly assign them to the appropriate Abiquo 'Network Service Type'. If networks are not correctly configured, VMs will not deploy correctly. Note the network limitation described above.

Abiquo Allocation Rules 

You will not be able to deploy VMs to an Oracle VM hypervisor if its RAM is fully utilized. Therefore, do not set Abiquo allocation rules to over-allocate RAM at more than 100% for Oracle VM-based Abiquo racks.

Configure Oracle VM

This section describes Oracle VM configuration in more detail.

Abiquo Automatic Configuration

Abiquo configures some elements that an Oracle Administrator would usually configure manually.

ISCSI Storage Device discovery

When you add new storage devices to the Abiquo platform, Abiquo will automatically add them to Oracle VM. However, Abiquo recommends that you should always test an iSCSI storage device before you add it to the Abiquo platform.

Storage Device and File Server permissions

Abiquo will ensure that Oracle VM Servers are granted access to ISCSI Storage Devices and File Servers.

Storage Device, Repository, and File Server Administration

When Abiquo creates storage devices, file servers, and repositories, it will automatically configure the Oracle VM servers which are allowed to administer these resources.

For performance reasons, Abiquo will only configure administrative access for Oracle VM Servers which are assigned the ‘Utility Server’ role.

Therefore you MUST have at least one Oracle VM server with this role. By default all Oracle VM Servers have this role enabled in your Oracle VM deployment. This is because certain operations (such as Repository and Storage Device refresh) can ‘lock’ Oracle VM servers, which stops those servers from performing other tasks. Abiquo recommends that servers with the ‘VM Server’ role do not also have the Utility Server role (i.e. that different servers are assigned different tasks). For more information, please see the Oracle VM documentation on Server Roles.

Oracle VM Server roles allow you to fine tune how your Oracle VM Servers are used in your environment. Roles allow you to designate Oracle VM Servers to perform utility functions and/or to run virtual machines. Oracle VM Manager supports the following server roles: 
• VM Server role: The VM Server role enables an Oracle VM Server to run virtual machines. 
• Utility Server role: A Utility Server will be favored to do operations other than hosting virtual machines, for example, importing virtual machine templates and assemblies, creating virtual machine templates from assemblies, and creating repositories. If no Utility Servers are available, a non-utility server is chosen to do the work. 

When you add an Oracle VM Server to a server pool it has both the Utility Server role and the VM Server role automatically selected by default. The VM Server role is required to run virtual machines. The Utility Server role is not required, it is advisory only. 

You can select certain Oracle VM Servers to have the Utility Server role and others to have the VM Server role. This reduces the likelihood of impeding virtual machine performance due to utility operations.

 

Manually assign server roles to ensure best performance

Network Handling

Abiquo will automatically manage the creation of Oracle VM Networks, VLAN Segments and VLAN Groups as required for each deployed VM. 

Oracle VM HA, manually check VLAN groups.

Repository Creation

Abiquo automatically creates an Oracle VM Repository within the Abiquo Repository. This is used by Abiquo as an intermediate location to create Oracle snapshot images (Abiquo instances), before they are moved to their correct location in the Abiquo Repository.

Add VM Servers with Utility Server and VM Server Roles

Oracle VM Servers can have one or both of the following roles:

  • Utility Server
  • VM Server

Servers with the Utility Server role may be used as Admin Servers. This means they may be used to perform some operations that would negatively affect the performance of virtual machines running on the server, for example, refreshing the file system.  The Abiquo Oracle VM plugin will assign only servers with the Utility Server role as Admin Servers but under certain circumstances, VM Servers may be used. By default, all Oracle VM servers are created with the Utility Server role. 

Abiquo requires that you have at least one server with the Utility Server role. Abiquo cannot detect the use that Oracle VM is making of a server, so VM performance on servers with both roles could be affected by Administration tasks.  Therefore Abiquo recommends that you have separate servers with the Utility Server role and VM Server Role.  And Abiquo recommends that you do not add your Utility Servers to the Abiquo platform as hypervisors.

Configure VM Servers Without Utility Server Role

Most VM Servers should not have the Utility Server role so that Oracle VM should not use them for administration tasks that may affect the performance of the VM.

To set roles, go to the Servers and VMs tab and click the Edit Server button.

  On the Configuration tab, remove the tick by the Utility Server role. Click OK to continue.

 

Configure Admin Servers with the Utility Server Role

By default, when you install a server, it will have the VM Server role selected, as well as the Utility Server role.

Abiquo will add use all servers with the Utility Server role as Admin Servers.

For best performance, Abiquo recommends that you configure some servers with the Utility Server role only. 

 

Configure Server Pools

Configuration of Oracle VM Server Pools is beyond the scope of the Abiquo Oracle VM Hypervisor Plugin. This means that it is the responsibility of the Oracle VM Cloud Administrator to create any required Oracle VM Server Pools, configure appropriate pool settings and policies, and correctly add Oracle VM Servers to the Server Pools.

Configure the VM Start Policy

To use Oracle VM, when creating a server pool, set the VM start policy to Start on current server.

Set VM Start Policy to Start on current server

If the VM Start Policy is set to Start on Best Server, the VM may be started on a different server selected by Oracle instead of the server selected by Abiquo.

Set the VM Start Policy to Start on current server to ensure your virtual machines are deployed on the server selected by Abiquo.

For the Hypervisor Type, select OVM/Xen.

For the Storage for Server Pool, select Network File System.

Then select the Storage Location from the File Server.

 

Enable HA at Server Pool Level

In order to allow Oracle VM HA at VM level, HA must be enabled within the server pool. 

Select Clustered Server Pool, which supports HA by default.  

HA must also be enabled in the abiquo.properties file.

Configure Oracle VM Networking for HA

By default, Abiquo does not enable Oracle VM HA support when deploying VMs from Abiquo, this needs to be enabled via the abiquo.properties file. When Oracle VM moves a VM for HA, it does not migrate the network connections. When Abiquo creates a VM in Oracle VM, Abiquo automatically creates the VLAN Groups and adds servers and ports. However, if you would like to maintain the VM's network configuration after an Oracle VM HA Event, you should check the network configuration in Oracle VM.

Abiquo will create a VLAN Group when you deploy the first VM. 

Check the VLAN Group Configuration for HA

Deploy one or more VMs.

Go the Networking Tab on the VLAN Groups page.

Select a VLAN group and click the pencil button.

Check that all the servers in the relevant clustered server pool have been added to the VLAN group as "Selected Server(s)".

Check that all the appropriate interfaces from the servers in the clustered server pool have been added to the VLAN group as "Selected Port(s)".

 

Perform Periodic Manual Review of Storage LUNs

Oracle VM does not automatically remove references to LUNs that are deleted from storage devices, for example, when volumes or persistent disks are removed in Abiquo. Therefore the Oracle VM administrator must periodically review the storage device configurations to remove LUNs marked as "missing" from Oracle VM.

For more information, see Oracle forum issue (https://forums.oracle.com/thread/2306790) that states that Oracle VM LUNs need to be explicitly removed from Oracle VM. 

When validating a LUN, Abiquo searches for any error-based events in the Oracle VM system for the disk, and ignores the disk 'unacknowledged' error events are found. Note that in Oracle VM, acknowledging an Event 'clears' its state. However, it may take Oracle VM quite some time to raise an event for the missing LUN.

Troubleshooting

I cannot deploy with Oracle VM

Oracle VM checks network conditions, so if you cannot deploy a VM you should check the network configuration as described here.

  1. Incorrect Network Service types
    • Check network service types are correctly assigned to each interface
  2. Duplicate MAC Addresses
    • Oracle will not let you use the same MAC adress in different virtual machines
    • For example, if you deploy a virtual machine using a private IP on Oracle VM, then remove the physical machine, remove the VM, and add the physical machine again. If you try to deploy a new VM using the same IP, the deploy will fail.

Oracle VM and iSCSI Storage

Oracle VM has some issues with iSCSI storage:

  • LUN discovery

  • LUN resize detection

  • LUN deletion detection and handling

Abiquo recommends that you should use the latest possible version of your storage device.

Always test the storage device with Oracle VM before you use it in Abiquo.

  • No labels