Hot questions for Using Azure in azure resource manager

Top Java Programmings / Azure / azure resource manager

Question:

How to get list of VMs (non-classic) using Java API, which are created using resource Manager? Why we need tenant id, client id and client key to create 'com.microsoft.azure.management.compute.ComputeManagementClient' object?

Can it be done using subscription id and Azure Portal credentials? Sample provided with azure-mgmt-compute project needs these tenant id, client id where as we don't need these details when we create VM (selecting Resource Manager) on Azure Portal.


Answer:

Why we need tenant id, client id and client key to create 'com.microsoft.azure.management.compute.ComputeManagementClient' object?

Behind the scenes, com.microsoft.azure.management.compute.ComputeManagementClient consumes Azure Resource Manager (ARM) REST API for performing Virtual Machines related operations. ARM API makes use of Azure Active Directory (AD) for authentication and authorization. In order to use Azure AD for this purpose, you would need to create an application in your Azure AD and grant that application permission to execute Azure Service Management API. You would need Tenant Id, Client Id and other things for that purpose only. So a user uses your application by allowing the application to be installed in their Azure AD. Tenant Id is the unique id of your application in your user's Azure AD. Client Id is the unique id of your application.

Once everything's been setup properly, in order to use the library user is authenticated against their Azure AD. As a part of authentication/authorization flow, user gets a token and this library makes use of this token to make authenticated request against ARM API to manage Virtual Machines.

Can it be done using subscription id and Azure Portal credentials? Sample provided with azure-mgmt-compute project needs these tenant id, client id where as we don't need these details when we create VM (selecting Resource Manager) on Azure Portal.

If you notice, you would first need to login into Azure Portal using your Microsoft account or Work/School account. Portal software fetches the token as a part of the login process. After that it makes use of tenant id, client id and this token to perform all operations. So essentially it is doing the same thing however it is not visible to you.

Question:

I am looking to copy a file from my storage account to a VM that is being created using an ARM template?

I have already created the storage account and added the files to the storage account. I am busy creating the template but don't see any option to copy the files from the storage account to the OSDisk that gets created when creating the VirtualMachine.

Update I feel I should give some background of what I want to achieve:

  1. I want to create ARM deploy.json and parameters.json
  2. Using the above JSON I would like to use the Azure Java SDK to create the VM in Azure
  3. In the JSON I will be using the docker extension to build out docker containers
  4. The docker containers will use base images from DockerHub and then will be customised using the compose which is supported in the DockerExtension
  5. I need to copy certain files over from the storage account so that the Docker Extension can use these files as part of the docker compose when building the custom docker containers
  6. The Java SDK will make certain decisions based on user input which will then generate the parameters.json for the Azure Java SDK

So based on this I need to somehow copy the files from storage to the VM before the azure docker extension executes.


Answer:

One way of achieving this would be to use a Custom script extension, which is added to your VM. It allows you to run a powershell script at the time of deployment that runs on the VM. In the JSON outline window in Visual Studio, click add resource

Then add a custom script extension and link it to the VM

This will add a blank powershell script to the newly created custom script folder in your project. Here you can put the script to download the file from blob. Like this.

Question:

I'm trying to write a java JUnit test that will deploy template-defined resource groups using the java-sdk for azure.

I came across a couple examples and followed them to create this code segment:

public void azurePoC() throws Exception {

    Azure azure = Azure.configure()
            .withLogLevel(LogLevel.BODY)
            .authenticate(new File("C:/somePath/credentials.azure"))
            .withDefaultSubscription();

    final ObjectMapper mapper = new ObjectMapper();
    JsonNode template = mapper.readTree(new File("C:/somePath/template.json"));

    Logger.report("template", template.toString());

    JsonNode params = mapper.readTree(new File("C:/somePath/parameters.json"));
    Logger.report("params", params.toString());

    azure.deployments()
            .define("myNewResource")
            .withNewResourceGroup("myNewResourceGroup", Region.US_WEST2)
            .withTemplate(template)
            .withParameters(params)
            .withMode(DeploymentMode.INCREMENTAL)
            .create();

    Logger.report("Request for deployment done.");
}

I have verified that the API allows me to lookup resources using azure.virtualMachines().list() method, therefore i conclude that my credentials are okay.

I am using one of the quick-start templates to for my PoC, just to see if i can deploy some resource.

My template.json file:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "adminUsername": {
      "type": "string",
      "metadata": {
        "description": "User name for the Virtual Machine."
      }
    },
    "adminPassword": {
      "type": "securestring",
      "metadata": {
        "description": "Password for the Virtual Machine."
      }
    },
    "dnsLabelPrefix": {
      "type": "string",
      "metadata": {
        "description": "Unique DNS Name for the Public IP used to access the Virtual Machine."
      }
    },
    "ubuntuOSVersion": {
      "type": "string",
      "defaultValue": "16.04.0-LTS",
      "allowedValues": [
        "12.04.5-LTS",
        "14.04.5-LTS",
        "15.10",
        "16.04.0-LTS"
      ],
      "metadata": {
        "description": "The Ubuntu version for the VM. This will pick a fully patched image of this given Ubuntu version."
      }
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
  "variables": {
    "storageAccountName": "[concat(uniquestring(resourceGroup().id), 'salinuxvm')]",
    "imagePublisher": "Canonical",
    "imageOffer": "UbuntuServer",
    "nicName": "myVMNic",
    "addressPrefix": "10.0.0.0/16",
    "subnetName": "Subnet",
    "subnetPrefix": "10.0.0.0/24",
    "storageAccountType": "Standard_LRS",
    "publicIPAddressName": "myPublicIP",
    "publicIPAddressType": "Dynamic",
    "vmName": "MyUbuntuVM",
    "vmSize": "Standard_A1",
    "virtualNetworkName": "MyVNET",
    "subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]"
  },
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[variables('storageAccountName')]",
      "apiVersion": "2018-07-01",
      "location": "[parameters('location')]",
      "sku": {
        "name": "[variables('storageAccountType')]"
      },
      "kind": "Storage",
      "properties": {}
    },
    {
      "apiVersion": "2018-10-01",
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[variables('publicIPAddressName')]",
      "location": "[parameters('location')]",
      "properties": {
        "publicIPAllocationMethod": "[variables('publicIPAddressType')]",
        "dnsSettings": {
          "domainNameLabel": "[parameters('dnsLabelPrefix')]"
        }
      }
    },
    {
      "apiVersion": "2018-10-01",
      "type": "Microsoft.Network/virtualNetworks",
      "name": "[variables('virtualNetworkName')]",
      "location": "[parameters('location')]",
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "[variables('addressPrefix')]"
          ]
        },
        "subnets": [
          {
            "name": "[variables('subnetName')]",
            "properties": {
              "addressPrefix": "[variables('subnetPrefix')]"
            }
          }
        ]
      }
    },
    {
      "apiVersion": "2018-10-01",
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[variables('nicName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[resourceId('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
        "[resourceId('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
      ],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Dynamic",
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]"
              },
              "subnet": {
                "id": "[variables('subnetRef')]"
              }
            }
          }
        ]
      }
    },
    {
      "apiVersion": "2018-10-01",
      "type": "Microsoft.Compute/virtualMachines",
      "name": "[variables('vmName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]",
        "[resourceId('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
      ],
      "properties": {
        "hardwareProfile": {
          "vmSize": "[variables('vmSize')]"
        },
        "osProfile": {
          "computerName": "[variables('vmName')]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "[variables('imagePublisher')]",
            "offer": "[variables('imageOffer')]",
            "sku": "[parameters('ubuntuOSVersion')]",
            "version": "latest"
          },
          "osDisk": {
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "diskSizeGB": 1023,
              "lun": 0,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
            }
          ]
        },
        "diagnosticsProfile": {
          "bootDiagnostics": {
            "enabled": true,
            "storageUri": "[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('storageAccountName')), '2016-01-01').primaryEndpoints.blob)]"
          }
        }
      }
    }
  ],
  "outputs": {
    "hostname": {
      "type": "string",
      "value": "[reference(variables('publicIPAddressName')).dnsSettings.fqdn]"
    },
    "sshCommand": {
      "type": "string",
      "value": "[concat('ssh ', parameters('adminUsername'), '@', reference(variables('publicIPAddressName')).dnsSettings.fqdn)]"
    }
  }
}

My parameters.json file:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "adminUsername": {
      "value": "ghuser"
    },
    "adminPassword": {
      "value": "GEN-PASSWORD"
    },
    "dnsLabelPrefix": {
      "value": "GEN-UNIQUE"
    }
  }
}

Whenever I run the application I am confronted with this exception:

com.microsoft.azure.CloudException: The request content was invalid and could not be deserialized: 'Error converting value "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#" to type 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Data.Definitions.DeploymentParameterDefinition'. Path 'properties.parameters.$schema', line 1, position 4459.'.: The request content was invalid and could not be deserialized: 'Error converting value "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#" to type 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Data.Definitions.DeploymentParameterDefinition'. Path 'properties.parameters.$schema', line 1, position 4459.'.
at com.microsoft.azure.AzureClient.createExceptionFromResponse(AzureClient.java:740)
at com.microsoft.azure.AzureClient.access$100(AzureClient.java:33)
at com.microsoft.azure.AzureClient$3.call(AzureClient.java:160)
at com.microsoft.azure.AzureClient$3.call(AzureClient.java:157)
at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
at retrofit2.adapter.rxjava.CallArbiter.deliverResponse(CallArbiter.java:120)
at retrofit2.adapter.rxjava.CallArbiter.emitResponse(CallArbiter.java:102)
at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:46)
at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:24)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeSingle.call(OnSubscribeSingle.java:81)
at rx.internal.operators.OnSubscribeSingle.call(OnSubscribeSingle.java:27)
at rx.internal.operators.SingleToObservable.call(SingleToObservable.java:39)
at rx.internal.operators.SingleToObservable.call(SingleToObservable.java:27)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.DeferredScalarSubscriber.subscribeTo(DeferredScalarSubscriber.java:153)
at rx.internal.operators.OnSubscribeTakeLastOne.call(OnSubscribeTakeLastOne.java:32)
at rx.internal.operators.OnSubscribeTakeLastOne.call(OnSubscribeTakeLastOne.java:22)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OperatorSubscribeOn$SubscribeOnSubscriber.call(OperatorSubscribeOn.java:100)
at rx.internal.schedulers.CachedThreadScheduler$EventLoopWorker$1.call(CachedThreadScheduler.java:230)
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: retrofit2.Response.class
at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:73)
... 43 more

Any and all help in the matter will be greatly appreciated.


Answer:

For anyone who comes across this question.

Turns out the problem was the parameter file json syntax. The schema, parameters and version fields are redundant

this is the expected syntax using Azure Java SDK:

{
  "adminUsername": {
    "value": "ghuser"
  },
  "adminPassword": {
    "value": "GEN-PASSWORD"
  },
  "dnsLabelPrefix": {
    "value": "GEN-UNIQUE"
  }
}

Question:

VirtualMachineGetResponse get(String resourceGroupName, String vmName) throws IOException, ServiceException, URISyntaxException;

From VirtualMachineGetResponse, I can get VirtualMachine. Is there any way to get VirtualMachine object by just giving instanceId as above code snippet expects resourceGroupName and instanceName?


Answer:

There is not any way to get VM object with only instance name in ARM for any languages. It's depended on Azure REST API Get for Virtual Machines in ARM, and the Java API just wrap it.

The REST API GET request for Virtual Machines in ARM, as below.

GET /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}?api-version=2016-03-30[&$expand]

Besides subscriptionId required for authentication, the parameters resourceGroupName & vmName are required in the api uri.

Question:

I am a Mac user and don't have access to the powershell cli tools. I want to authenticate my java program to the azure in resource manager mode (arm)

According to the docs i have read so far, i have created a app in the Active directory, but I don't see any option to upload any certificate to the application for authentication in the UI. Also, I don't see any option in the UI for assigning privilege to the application for using resource manager api.

Is using powershell the only option to do the above tasks ?


Answer:

I am a Mac user and don't have access to the powershell cli tools. I want to authenticate my java program to the azure in resource manager mode (arm)

I'm not sure why you say that. CLI tools are built using node.js (thus available on all platforms) and are different than Azure PowerShell Cmdlets. Please see this link for using CLI tools with Azure Resource Manager - https://azure.microsoft.com/en-in/documentation/articles/xplat-cli-azure-resource-manager/.

According to the docs i have read so far, i have created a app in the Active directory, but I don't see any option to upload any certificate to the application for authentication in the UI.

With Azure Resource Manager, you don't have to do that. The authentication/authorization will be done via Azure Active Directory. Certificate based authentication is for Azure Service Management API.

Also, I don't see any option in the UI for assigning privilege to the application for using resource manager api.

This, you can do through Azure Preview Portal. See this screenshot below where I have assigned permission to an application on my Azure Subscription.