Hot questions for Using Azure in networking

Top Java Programmings / Azure / networking

Question:

I am not sure whether this question is Mina-related or more Azure-related but it has to do with the networking. I have also added Netty tag since Mina and Netty share many networking principles.

I hope to get an advice where to dig into.

I have used certain Mina application quite long in local network, now I am trying to migrate it into the cloud. I deploy Linux virtual machines in Azure (each has public IP but does this really matter?).

They connect (using Mina) to a machine outside Azure that also has its own public IP. Usual thing:

SocketConnector connector = new NioSocketConnector(numberOfConnectors);
ConnectFuture connectFuture = connector.connect(new
InetSocketAddress(remoteHost, remotePort));
connectFuture.awaitUninterruptibly(connectTimeout);

That Mina machine outside the Azure also runs Mina. Let's call it server machine.

It accepts connections like this:

NioSocketAcceptor acceptor = new NioSocketAcceptor(acceptor_threads);
org.apache.mina.core.buffer.IoBuffer.setUseDirectBuffer(false);
acceptor.getSessionConfig().setTcpNoDelay(true);
acceptor.setReuseAddress(true);
acceptor.getSessionConfig().setSendBufferSize(buffer_size);
acceptor.getSessionConfig().setMinReadBufferSize(64000);
acceptor.getSessionConfig().setReceiveBufferSize(buffer_size);
acceptor.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE, iddle_time);
acceptor.getFilterChain().addLast("codec", new
ProtocolCodecFilter(CodecFactory.getInstance()));
acceptor.setDefaultLocalAddress(new InetSocketAddress(port));

When Azure applications connect to server machine, server saves

IoSession session

to asynchronously push messages back in future like this:

session.write(message);

This worked well inside a local network (without Azure), but in current deployment server sends message

2017-01-17/15:45:19.823/GMT-00:00 [nioEventLoopGroup-3-3] [...] DEBUG
Sending message to /13.94.143.139:41790

and an Azure machine does not receive anything. Moreover, after a while the following exception arises on server machine:

2017-01-17/16:01:11.419/GMT-00:00 [NioProcessor-4] [...] ERROR
Exception in IOHandlerConnection timed out
java.io.IOException: Connection timed out
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
    at sun.nio.ch.IOUtil.read(IOUtil.java:197)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
    at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:280)
    at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:44)
    at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:695)
    at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
    at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
    at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)
    at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1141)
    at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

2017-01-17/16:01:11.424/GMT-00:00 [NioProcessor-3] [...] DEBUG sessionClosed

I use Mina version 2.0.4 (yes, it is old but it works on local network for several years for now).

I setup Azure network with Java Azure SDK 1.0.0-beta3

Network.DefinitionStages.WithCreate creatableNetwork = azure.networks()
                    .define(networkName)
                    .withRegion(region)
                    .withExistingResourceGroup(resourceGroup)
                    .withAddressSpace("10.0.0.0/20");

And create virtual machines as

VirtualMachine.DefinitionStages.WithCreate creatableVirtualMachine =
         azure.virtualMachines()
           .define(String.format(...))
           .withRegion(region)
           .withExistingResourceGroup(resourceGroup)
           .withNewPrimaryNetwork(creatableNetwork)                             
           .withPrimaryPrivateIpAddressStatic(inetAddress.getHostAddress())
           .withNewPrimaryPublicIpAddress(String.format("chr-vm-%04d", i))                  .withPopularLinuxImage(KnownLinuxVirtualMachineImage.UBUNTU_SERVER_16_04_LTS)
           .withRootUserName(linuxUserName)
           .withPassword(linuxUserPassword)                                                                    
           .withSize(VirtualMachineSizeTypes.STANDARD_D2_V2)                                                 
           .withNewStorageAccount(creatableStorageAccount);

I wonder what reasons may prevent traveling messages from server to Azure client machines? Azure network configuration? Mina configuration? (the first messages from client machines to server machine do come after they connect)

I hope that above information may contain a clue.


Answer:

I have solved my problem thanks to Peter Pan - MSFT noting about NSG - Network Security Group.

NSG controls in/out rules like a Windows Firewall. You should create NSG, add rules to it, and assign NSG to a particular entity:

There are at least two options to assign NSG:

  • to a network subnet
  • to a network interface

There is a tutorial 1 and Java code sample 2. In my case, a separate network interface is created for each VM (since each VM has public IP). So, I assigned one NSG to a single subnet.

Fisrt, create NSG:

NetworkSecurityGroup NSG = azure.networkSecurityGroups()
                    .define(networkSecurityGroup)
                    .withRegion(region)
                    .withExistingResourceGroup(resourceGroup)
                    .defineRule("Inbound")
                        .allowInbound()
                        .fromAnyAddress()
                        .fromAnyPort()
                        .toAnyAddress()
                        .toAnyPort()
                        .withAnyProtocol()
                        .withDescription("Incoming messsages")
                        .withPriority(100)
                        .attach()
                    .create();

Than modify the code to explicitly define a subnet and assign NSG to it ( subnet1 is automatically created without NSG if none defined explicitly)

Network.DefinitionStages.WithCreate creatableNetwork = azure.networks()
                    .define(networkName)
                    .withRegion(region)
                    .withExistingResourceGroup(resourceGroup)
                    .withAddressSpace("10.0.0.0/20")
                    .defineSubnet(subnetName)
                        .withAddressPrefix("10.0.0.0/20")
                        .withExistingNetworkSecurityGroup(NSG)
                        .attach();

So, the rest of the code remains the same as posted in the question above.

Helpful links:

  1. Azure Portal Tutorial
  2. Java Azure SDK NSG Example

Question:

I can't find the Java API from the Azure SDK to manage PostgreSQL firewall rules.

The REST API is documented here.

Does anyone know which Maven dependency to use?

Thanks, Chris


Answer:

Not sure if this is what you're looking for but you can find source code for Java SDK for PostgreSQL here: https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/postgresql.

Question:

I can't find the Java API from the Azure SDK to delete a NetworkSecurityRule resource.

The REST API is documented here.

I use this Maven dependency: com.microsoft.azure:azure-mgmt-network:jar:1.31.0

In my code I hold a reference to a NetworkManager instance and I have a collection of NetworkSecurityRule objects.

Does anyone know how to do it?

Thanks, Chris


Answer:

According to my test, we can use the following code. For more details, please refer to the docuemnt 1. create a service principal and assign Reader role for the sp.

az login
az account set --subscription "<your subscription id>"
# it will assign  Contributor to the sp at subscription level
az ad sp create-for-rbac -n "mysample" --role  Contributor

  1. code
public static void main(String[] args){
 String clientId = "your sp appId";
       String secret = "your sp password";
       String domain = "your tenant domain";
       ApplicationTokenCredentials credentials = new ApplicationTokenCredentials(clientId, domain, secret,
            AzureEnvironment.AZURE);
       Azure azure = AzureAzure.configure().withLogLevel(LogLevel.BASIC).authenticate(credentials)
            .withDefaultSubscription();

       NetworkSecurityGroup group = azure.networkSecurityGroups().getById(
            "your nsg resource id");

       for(String i : group.securityRules().keySet()){

            System.out.println(i);

       } 

       group.update().withoutRule.apply();
       group = azure.networkSecurityGroups().getById(
            "/subscriptions/e5b0fcfa-e859-43f3-8d84-5e5fe29f4c68/resourceGroups/testgroup/providers/Microsoft.Network/networkSecurityGroups/test0123");
       System.out.println(group.Name());
       for(String i : group.securityRules().keySet()){

            System.out.println(i);

       } 
}