Failed to import ova

[nagavenkatab]

I trying to import ova using ova file and the script fails with the below error

2017-11-24T10:24:27.388Z error vpxd[7F996F9F3700] [Originator@6876 sub=SoapAdapter]
--> Required property id is missing from data object of type OvfConsumerOstNode
-->
--> while parsing serialized DataObject of type vim.OvfConsumer.OstNode
--> at line 11, column 8
-->
--> while parsing property "instantiationOst" of static type OvfConsumerOstNode
-->
--> while parsing serialized DataObject of type vim.ImportSpec
--> at line 7, column 6
-->
--> while parsing call information for method ImportVApp
--> at line 5, column 4
-->
--> while parsing SOAP body
--> at line 4, column 2
-->
--> while parsing SOAP envelope
--> at line 2, column 0
-->
--> while parsing HTTP request for method importVApp
--> on object of type vim.ResourcePool
--> at line 1, column 0
2017-11-24T10:24:31.053Z info vpxd[7F996F8F1700] [Originator@6876 sub=vpxLro opID=2dc64d86] [VpxLRO] -- BEGIN task-internal-18777 -- SessionManager -- vim.SessionManager.sessionIsActive -- 520abc27-ebc2-2ccc-42bf-864790e61a93(52ef17e8-51fb-0da9-5fca-d490400cd24d)

Below is the code

"use strict";

let username = '<user>';
let password = '<password>';
let host = '<vCenter host>';
const VConnect = require('vsphere-connect');
const fs = require('fs');

let v = VConnect(host, {ignoreSSL: true})
v.createClient().then(client => {
    return client.login(username, password).then((c) => {
     const vim = client._soapClient.types.vim25;
     return client.retrieve({
       type: 'ClusterComputeResource',
       id: 'domain-c7'
     }).then((clu) => {
       const cluster = clu[0].moRef;
       console.log(cluster);
       return client.retrieve({
         type: 'Network',
         id: 'network-22'
       }).then((net) => {
         const network = net[0].moRef;
         console.log(network);
         return client.retrieve({
           type: 'Datastore',
           id: 'datastore-154'
         }).then((dst) => {
           const ds = dst[0].moRef;
           console.log(ds);
           return client.retrieve({
             type: 'Folder',
             id: 'group-v3'
           }).then((fl) => {
             const folder = fl[0].moRef;
             console.log(folder);
             return client.retrieve({
               type: 'ResourcePool',
               id: 'resgroup-8'
             }).then((rpool) => {
               const rp = rpool[0].moRef;
               console.log(rp);
               // download https://communities.vmware.com/servlet/JiveServlet/downloadBody/21621-102-3-28798/Tiny%20Linux%20VM.ova and untar
               const ovfDescriptor1 = fs.readFileSync('~/Tiny Linux VM.ovf').toString();
               const specParams = getImportSpecParams(vim, network);
               return client.method('CreateImportSpec', {
                 _this: client.serviceContent.ovfManager,
                  ovfDescriptor: ovfDescriptor1,
                  resourcePool: rp,
                  datastore: ds,
                  cisp: { locale: 'US', deploymentOption: '', entityName: 'testvmw' }
               }).then((imspec) => {
                 console.log('imspec', imspec.importSpec);
                 console.log(imspec.importSpec)
                 return client.method('ImportVApp', {
                   _this: rp,
                   spec: imspec.importSpec,
                   folder: folder
                 }).then((vapp) => {
                   console.log('vapp', vapp);
                 })
               }).catch((err1) =>  console.log('err1', err1))
             })
             })
           })
         })
       })
     })
}).catch((err) =>  console.log('err', err))

function getImportSpecParams(vim, network) {
  const propertyMappings = [];
  const ovfPropertyKeys = {
    host_ip: host,
    host_username: username,
    host_password: password,
    proxy: '172.18.192.3',
    static_ip: '172.18.200.186',
    static_netmask: '255.255.255.0',
    static_gateway: '172.18.192.20',
    static_dns: '172.18.192.5'
  };

  for (const key in ovfPropertyKeys) {
    const param = new vim.KeyValue();
    param.key = key;
    param.value = ovfPropertyKeys[key];
    propertyMappings.push(param);
  }

  const networkMapping = new vim.OvfNetworkMapping();
  networkMapping.name = 'Network';
  networkMapping.network = network;

  const importSpecParams = new vim.OvfCreateImportSpecParams();
  importSpecParams.entityName = 'testvm1';
  importSpecParams.diskProvisioning = 'thin';
  importSpecParams.propertyMapping = propertyMappings;
  importSpecParams.ipAllocationPolicy = 'fixedPolicy';
  importSpecParams.deploymentOption = '';

  importSpecParams.locale = 'US';
  importSpecParams.networkMapping = [networkMapping];
  return importSpecParams;
}

[nagavenkatab]

by debugging more I see that configSpec is missing in the spec params of ImportVApp method, not sure is it causing the issue

[bhoriuchi]

I think i see the issue. You are attempting to create new objects with types from vim which is not currently implemented. See code https://github.com/bhoriuchi/soap-connect/blob/development/src/client/types.js#L47-L51 So you are essentially getting back an empty object that only has the fields you are setting on it. I can look into this further and try to implement it as i think it was already on my todo list.

Until then you will need to fully build the specs with javascript objects and it is best to include the type as documented here https://github.com/bhoriuchi/soap-connect/blob/development/src/client/types.js#L47-L51 since the soap library does a best match on the types if the type is not explicitly defined which can lead to incorrect types.

[avpk10]

I can add more info on this issue:

In version 1.1.1 of this package we used to get the attributes returned by the vCenter and that is missing in v2 because of which, when we chain the response of createImportSpec Params to ImportVapp we see an err

[bhoriuchi]

@DrigerG which attributes are those? Also this is definitely because the package now uses a custom soap library instead of node-soap which i found to be problematic and slow due do no wsdl caching features. I'd like to get the code you are running to reproduce as well if you wouldnt mind commenting here or opening another issue.

[avpk10]

I was able to work around the issue by doing the below steps to the returned importSpec:

    delete response.importSpec.instantiationOst
    response.importSpec.instantiationOst.id = ''
    delete response.importSpec.instantiationOst.section;
    delete response.importSpec.instantiationOst.child[0].section;
    delete response.importSpec.instantiationOst.child[0].child;
    response.importSpec['@xsi:type'] = 'vim25:VirtualMachineImportSpec'
    response.importSpec.configSpec.deviceChange[0].device['@xsi:type'] = 'vim25:VirtualCdrom'
    response.importSpec.configSpec.deviceChange[0].device.backing['@xsi:type'] = 'vim25:VirtualCdromRemotePassthroughBackingInfo'
    response.importSpec.configSpec.deviceChange[1].device['@xsi:type'] = 'vim25:VirtualDisk'
    response.importSpec.configSpec.deviceChange[1].device.backing['@xsi:type'] = 'vim25:VirtualDiskFlatVer2BackingInfo'
    response.importSpec.configSpec.deviceChange[1].device.backing['fileName'] = ''
    response.importSpec.configSpec.deviceChange[2].device['@xsi:type'] = 'vim25:VirtualIDEController'
    response.importSpec.configSpec.deviceChange[3].device['@xsi:type'] = 'vim25:VirtualIDEController'
    response.importSpec.configSpec.deviceChange[5].device['@xsi:type'] = 'vim25:VirtualLsiLogicController'
    response.importSpec.configSpec.deviceChange[6].device['@xsi:type'] = 'vim25:VirtualMachineVMCIDevice'
    response.importSpec.configSpec.deviceChange[7].device['@xsi:type'] = 'vim25:VirtualE1000'
    response.importSpec.configSpec.deviceChange[7].device.backing['@xsi:type'] = 'vim25:VirtualEthernetCardNetworkBackingInfo'
    delete response.importSpec.configSpec.cpuFeatureMask;
    delete response.importSpec.configSpec.extraConfig;
    delete response.importSpec.configSpec.vAppConfig.product;
    delete response.importSpec.configSpec.vAppConfig.property;
    delete response.importSpec.configSpec.vAppConfig.eula;
    delete response.importSpec.configSpec.vAppConfig.ovfSection;
    delete response.importSpec.configSpec.vAppConfig.ovfEnvironmentTransport;

This is due the fact I mentioned in the above comment.. Since the attributes are lost, the lib resorts to detecting the xsi types which sometimes turns out to be wrong. There are some minor bugs in the extendedType logics too.

Anyway thanks @bhoriuchi for this wonderful package !

[avpk10]

@bhoriuchi About the attributes:

in v1 @nagavenkatab code returns the importSpecParams with xsi:type attribute, however no such attribute is returned in v2

[bhoriuchi]

@DrigerG thanks! it looks like i need to figure out why the attributes are getting lost. And any suggestions on how to improve the extendedType logic is welcome. I admittedly am not an expert with soap/wsdl and do not devote a lot of time to this project.

[avpk10]

One top of the head logical flaw I saw was the extendedType closest match was comparing the intersection length with the parentType to begin with, which is kinda wrong. Thats what happens in the above case when it encounters ImportSpec.

say I pass 3 params
The ImportSpec has 2 params
The VirtualMachineImportSpec has 1 additional param
The intersection sees 2 > 1 and ignores the VirtualMachineImportSpec

[avpk10]

@bhoriuchi would you mind If we push some pull requests to fix any bugs we see ?

[bhoriuchi]

@DrigerG yes please! I'll try to add some contributing guidelines so people know this project is open to PRs.

[avpk10]

@bhoriuchi Thanks !
In the mean time I am trying to see if v1 solves @nagavenkatab 's code

[bhoriuchi]

@DrigerG what do you think about bringing all the code from the soap client soap-connect into this library? That whole module was written to support this one and while i like the idea of having it as a standalone module i doubt i will ever have time to make it a full featured soap client and it would make development on this project easier.

[avpk10]

@bhoriuchi I agree 100 percent. v1.1.1 is simpler to debug and faster to develop because its in a single project

[avpk10]

I just used v1 to generate @nagavenkatab's ImportSpec and used it in V2 after some xsi label massaging and the import works perfectly !!!

May be we need to revisit v2's attribute returning capability

[bhoriuchi]

@DrigerG I'm going to add the soap-connect code in and set up a lint config and a framework for tests in the development branch so that there are some initial guidelines for that stuff.

[bhoriuchi]

@DrigerG I have updated the lib to include the soap client code, added an eslintrc, test example, and basic contributing guide. feel free to submit PRs.

[avpk10]

the master seems to have this issue now:

ReferenceError: babelHelpers is not defined
    at new SoapConnectClient (C:\testvc\node_modules\vsphere-connect\index.js:1627:9)

[bhoriuchi]

building with rollup has always been problematic. i have switched the build to just use babel. it will now build to dist/