Hot questions for Using Ubuntu in macos

Question:

I'm working on a personal project for school where I have to user RMI to communicate between server and client.

Project info

The goal of my project is to retrieve stock info (from NYSE) for each day on the server at a specific time (after NYSE is closed). Each stock object is saved in a database. The information is retrieved over http and has nothing to do with RMI.

For the client it is also possible to fetch the stocks. When a user wants to fetch the stock object for the current day, it is directly fetched from the 3th party service. When a user, for example, wants to fetch Google's stock from last month, it is requested on the server over RMI. The server will the look for the stock object in the database and retrieve a Stock object and send it to the client.

Problem

When I start the client application, I have to login. The client will create a User object containing the username and password. When I press the login button, it will take around 2 minutes before the main screen will be shown.

Below the source code where I setup the RMI connection.

Server (main.java)

public static void main(String[] args) throws UnknownHostException {

    InetAddress IP= InetAddress.getLocalHost();
    System.out.println("IP of my system is := "+IP.getHostAddress());

    if(args.length == 1 && args[0].toLowerCase().equals("local")) {
        System.out.println("Running on localhost");
        System.setProperty("java.rmi.server.hostname", IP.getHostAddress());
    } else {
        System.out.println("rmi hostname is set to 37.97.223.70");
        System.setProperty("java.rmi.server.hostname", "37.97.223.70");
    }

    try {
        Registry reg = LocateRegistry.createRegistry(1099);
        StockAppServer server = StockAppServer.getInstance();
        reg.rebind("StockApp", server);
        System.out.println("StockApp bound for StockAppServer object.");
    } catch (RemoteException e) {
        e.printStackTrace();
    }
}

Based on the arguments that are passed to the application when it starts, I set the RMI hostname to my current IP address, or to the remote server address. The remote server address is a static IP, so this won't change.

Server (StockAppServer.java)

This class implements the interfaces that is used by the client to call methods on the server. So this class extends UnicastRemoteObject. When I start the server, registerStockTask() will be called. This method will fetch the ticker symbols (What are ticker symbols?) and then schedule a task to fetch all stock objects at a specific time.

private static StockAppServer _instance;
private List<User> loggedInUsers;
private List<Group> activeGroups;
private List<Notification> registeredNotifications;

private StockAppServer() throws IOException {
    _instance = this;

    this.loggedInUsers = new ArrayList<>();
    this.activeGroups = new ArrayList<>();
    this.registeredNotifications = new ArrayList<>();

    this.registerStockTask();

    clearActiveGroups();
    checkForCompletedNotifications();

    // Start the restful framework to allow incoming connections from the NodeJS server to manage new notification
    Router.getInstance();
}

public static StockAppServer getInstance() {
    try{
        return _instance == null ? new StockAppServer() : _instance;
    } catch (IOException e) {
        e.printStackTrace();
    }

    return null;
}

Client (main.java)

public static void main(String[] arguments) throws Exception {
    args = arguments;
    Application.launch();
}

@Override
public void start(Stage primaryStage) throws Exception {
    InetAddress IP= InetAddress.getLocalHost();
    System.out.println("IP of my system is := "+IP.getHostAddress());



    if(args.length == 1 && args[0].toLowerCase().equals("local")) {
        // Program started with local command, expect that server is running on local host
        reg = LocateRegistry.getRegistry(IP.getHostAddress(), 1099);
        System.out.println("Attempting to connect to RMI server over 127.0.0.1");
    } else {
        // Program started without additional commands. Except that "the server" is available;
        reg = LocateRegistry.getRegistry("37.97.223.70", 1099);
        System.out.println("Attempting to connect to RMI server over 37.97.223.70");
    }

    try {
        StockApp.getInstance().setServerInterfaces((IStockSend) reg.lookup("StockApp"), (IUserHandling) reg.lookup("StockApp"));
    } catch(RemoteException e) {
        AlertMessage.showException("Unable to connect to server.", e);
    } catch (NotBoundException e) {
        AlertMessage.showException("No server has been found with the name \"StockApp\" on the remote host.\nPlease try again later", e);
    }

    LoginController.showMenu();

    //FileNotFoundException e = new FileNotFoundException("Couldn't find file blabla.txt");
    //AlertMessage.showException("Something went wrong. Please try again later.", e);
}

How I tried to solve my problem

When I test my applications local, there is no problem. The login method will be finished within a few milliseconds and I will be represented the main screen.

  • I started by turning of my firewall on my macbook. No result, login method still takes around 2 seconds.
  • I turned off the firewall om my Ubuntu server. No result, both firewalls on server and macbook are turned off. Login method still takes around 2 seconds.
  • On the server runs (thanks to jenkins) another (unrelated) program. This program uses sockets instead of RMI. When this program is not running, the login method still takes around 2 minutes.
  • In StockAppServer.java, I called the following method:

    super(1099); This has the same outcome as the above steps I took.

I don't know what else I can try to solve my problem.

I tried to give as much code as possible for the RMI part. I you need any other source code, just ask and I can update this question. Also, the source code is available via github: https://github.com/juleskreutzer/GSO-Maatwerk. Make sure to run the program with -remote param.

Update 1 (9-1-2017)

As yanys requested in the comments, I should run the following command:

dscacheutil -q host -a name localhost

this returns the following output:

Mac:

name: localhost
ip_address: 127.0.0.1

Ubuntu:

dscacheutil: command not found

Update 2 (9-1-2017)

I checked with the provider of my VPS where I run the java server on. On their side everything should be OK. According to them, it shouldn't be a dns problem. After some research, I found out that RMI uses both DNS and reverse DNS. It this case, reverse DNS was the issue. Please see my answer on how I solved my problem.


Answer:

As EJP pointed out in the comments on the question, it was an DNS problem.

I contacted the support of my hosting provider to see if I had some wrong settings. They helped me a lot in solving this problem.

First we tested the speed of my VPS, this is around 1000mbit download and upload speed. After we checked this, they said there was nothing wrong on their side.

After doing some research, I found out that RMI uses both DNS and Reverse DNS. The problem was that I didn't setup the reverse DNS on my server. I already have a domain name to use for reverse DNS.

I than did the following:

  1. Create a A-record on my website that points to the IP address of the server. I named it vps.mydomain.com
  2. Add the reverse DNS in the control panel of my server
  3. Change the hostname of my server to vps.mydomain.com*

*My server runs Ubuntu 16.04, on ubuntu machines with systemd, you can use the command

sudo hostnamectl set-hostname new-name

to change the hostname

Question:

Recently, the stable version (1.6.2) of apache flink was released. I read this instruction. But when I run the following command:

./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000

I get the following error:

 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 264564a337d4c6705bde681b34010d28)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:486)
    at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:66)
    at org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:92)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:426)
    at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:816)
    at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:290)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:216)
    at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1053)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1129)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
    at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1129)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
    ... 20 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:748)

I found this link:Flink program cannot submit when i follow flink-1.4's quickstart and use "./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000". However, it didn't help. I tried with Flink 1.6.2 with Hadoop® 2.8 as well as Flink 1.5.5 with Hadoop® 2.8 on mac os and ubuntu. But I just got the same error.


Answer:

Works fine for me. Only difference I can see is that I'm using the version of Flink without hadoop, but I doubt that's the issue.

java.net.ConnectException: Connection refused usually means that there is no service listening on port 9000. You should have netcat running on this port via

$ nc -l 9000

(see https://ci.apache.org/projects/flink/flink-docs-stable/quickstart/setup_quickstart.html#run-the-example). If netcat is running, and it's not working, maybe try another port? It might also help to check all the log files for additional clues.