Hot questions for Using Transmission Control Protocol in stream

Question:

I'm trying to transmit real time mic recording to server over TCP socket and server to write input stream to a file. The connection is established but after some time, I'm getting connection refused error at my clientside.

Server Code:

    public class auServer extends Thread{
    private static ServerSocket serverSocket;
    private static int port = 3333; 

    public void run()
    {

        System.out.println("init success");
       while(true)
       {

          try
          {
              serverSocket = new ServerSocket(port);
              serverSocket.setSoTimeout(10000);
              Socket clientSoc = serverSocket.accept();
             System.out.println("Waiting for client on port " +serverSocket.getLocalPort() + "...");
             System.out.println("Just connected to " + clientSoc.getRemoteSocketAddress());
             InputStream in = clientSoc.getInputStream();
             while(in!=null)
             {
                 writeToFile(in);
             }
             System.out.println("socket");

             clientSoc.close();
          }catch(SocketTimeoutException s)
          {
             System.out.println("Socket timed out!");
             break;
          }catch(IOException e)
          {
             e.printStackTrace();
                     System.out.println("some io");
             break;
          } catch (Exception e) {
                    System.out.println("some e");
            e.printStackTrace();
        }
       }
    }

    private void writeToFile(InputStream in) throws IOException {
        // Write the output audio in byte
        String filePath = "8k16bitMono1.wav";
        short sData[] = new short[1024];
        byte[] bData = IOUtils.toByteArray(in);
        FileOutputStream os = null;
        try {
         os = new FileOutputStream(filePath);
        } catch (FileNotFoundException e) {
         e.printStackTrace();
        }
         System.out.println("Short wirting to file" + sData.toString());
         try {
          os.write(bData, 0, 2048);
         } catch (IOException e) {
          e.printStackTrace();
         }
        try {
         os.close();
        } catch (IOException e) {
         e.printStackTrace();
        }

    }


    public static void main(String[] args) {
        // TODO Auto-generated method stub
      try
      {
        Thread serverThread = new auServer();
        serverThread.run();
        System.out.println("runing");
       }catch(IOException e){
         e.printStackTrace();
      }
    }
}

and Client :

private void streamData(byte[] bData) throws UnknownHostException, IOException, InterruptedException {  //bData is byte array to transmit
    Thread.sleep(500);
    Socket client = new Socket("10.221.40.41",3333);
    OutputStream outToServer = client.getOutputStream();
    outToServer.write(bData);
    if(!isRecording)
        client.close();
}

What could be the problem? Thanks in advance.


Answer:

I'll comment on your code piece by piece.

private static ServerSocket serverSocket;

There's no reason for this to be static.

while(true)
{
    try
    {
        serverSocket = new ServerSocket(port);
        serverSocket.setSoTimeout(10000);

The last two lines should be ahead of the loop. This is the reason for the connection refusals, and it will also cause BindExceptions that you haven't mentioned. It isn't clear why you need the timeout.

             Socket clientSoc = serverSocket.accept();
             System.out.println("Waiting for client on port " +serverSocket.getLocalPort() + "...");

No you aren't. He's already connected. You were waiting before the accept().

             System.out.println("Just connected to " + clientSoc.getRemoteSocketAddress());
             InputStream in = clientSoc.getInputStream();
             while(in!=null)

Both the loop and the test are futile. The variable isn't null initially, and there is no way it can ever become null. The loop is futile because the writeToFile() method completely exhausts the input stream, so there is never anything more to read. This will be causing junk data, which you haven't mentioned.

             {
                 writeToFile(in);
             }
             System.out.println("socket");

A pointless message.

             clientSoc.close();

All the code from the line after accept() to here should be executed in a separate thread. The accept loop should do nothing but accept connections and start threads.

          }catch(SocketTimeoutException s)
          {
             System.out.println("Socket timed out!");

What timed out here was the accept(), as the listening socket is the only socket you've set a timeout on. I doubt that you need this.

             break;
          }catch(IOException e)
          {
             e.printStackTrace();
                     System.out.println("some io");

Another futile message.

             break;
          } catch (Exception e) {
                    System.out.println("some e");

Yet another one. When you get an exception, print the exception. Not some futile message of your own devising. Otherwise debugging becomes a mere guessing game.

            e.printStackTrace();
        }
       }
    }

    private void writeToFile(InputStream in) throws IOException {
        // Write the output audio in byte
        String filePath = "8k16bitMono1.wav";
        short sData[] = new short[1024];

Unused. Remove.

        byte[] bData = IOUtils.toByteArray(in);

Don't use this. It wastes space and adds latency. See below for the correct solution.

        FileOutputStream os = null;
        try {
         os = new FileOutputStream(filePath);
        } catch (FileNotFoundException e) {
         e.printStackTrace();
        }

Poor technique. This catch is in the wrong place. Code that depends on the success of code in a try block should be inside the same try block. At present you are falling through this try-catch as though the exception never happened, which will cause a NullPointerException in the code below.

         System.out.println("Short wirting to file" + sData.toString());

Another pointless message. Misspelt; sData has nothing in it; sData.toString() doesn't print anything useful regardless of the contents; and incorrect, as you aren't writing sData at all.

         try {
          os.write(bData, 0, 2048);

This writes exactly 2048 bytes, regardless of the amount read, which could be less or more. If it was less it will throw an ArrayIndexOutOfBoundsException or similar, which I would expect to see on the second call, although you haven't mentioned it, because the array should be zero length on the second and subsequent calls, if any. It should be a loop, of the general form:

int count;
byte[] buffer = new byte[8192]; // or more if you like
while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

Back to your code:

         } catch (IOException e) {
          e.printStackTrace();
         }
        try {
         os.close();
        } catch (IOException e) {
         e.printStackTrace();
        }
    }


    public static void main(String[] args) {
        // TODO Auto-generated method stub
      try
      {
        Thread serverThread = new auServer();
        serverThread.run();

This runs the run() method of the thread. It doesn't start a thread. It should be serverThread.start().

        System.out.println("runing");

Misspelt. Until you fix the start/run problem above, you won't see this message until after the server's run() method has exited.

and Client :

private void streamData(byte[] bData) throws UnknownHostException, IOException, InterruptedException {  //bData is byte array to transmit
    Thread.sleep(500);

Pointless. Remove. Don't put sleeps into networking code.

    Socket client = new Socket("10.221.40.41",3333);

Don't create a new connection per buffer. Use the same connection for the life of the client.

    OutputStream outToServer = client.getOutputStream();
    outToServer.write(bData);
    if(!isRecording)
        client.close();

This should be unconditional, and it should be elsewhere, along with the creation of the socket.

What could be the problem?

Problems. Plural. Multiple. See above.

Question:

I am programming a simple TCP server in Java that is listening on some URL on some port. Some client (not in Java) sends a JSON message to the server, something like this {'message':'hello world!', 'test':555}. I accept the message an try to get the JSON (I am thinking to use GSON library).

Socket socket = serverSocket.accept();
InputStream inputStream = socket.getInputStream();

But how can I get the message from input stream? I tried to use ObjectInputStream, but as far as I understood it waits serialized data and JSON is no serialized.


Answer:

Wrap it with a BufferedReader and start reading the data from it:

BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String content = br.readLine();
System.out.println(content);

Then parse your JSON string with a library like Gson or Jackson.

Question:

I'm creating a java Client program that sends a command to server and server sends back an acknowledgement and a response string.

The response is sent back in this manner

client -> server : cmd_string

server -> client : ack_msg(06)

server -> client : response_msg

When I try to read the input either I'm only able to read to read just the acknowledgement message with one inputstream

My client program is able to read the message somehow(hacky method). To read the input I have to read the ack msg using Bufferedreader.

  1. This buffered reader in only able to read ack_msg not the following msg
  2. DataInputstream code is needed to read response msg.
  3. If I skip step 1 and just use Datainputstream I'm not able to read complete messages.

Client.java

try {
            Socket clientSocket = new Socket(SERVER_ADDRESS, TCP_SERVER_PORT);// ip,port

            System.out.println(" client Socket created .. ");

            PrintWriter outToServer = new PrintWriter(clientSocket.getOutputStream(), true);

            String ToServer = command;
            outToServer.println(ToServer);

            in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
//          
            // while (!in.ready()) {
            // }
            System.out.println(in.read()); // Read one line and output it


            // ===
            byte[] messageByte = new byte[1000];//assuming msg size -need to know eact msg size ?
            boolean end = false;
            String dataString = "";
            int bytesRead = 0;

            try {
                DataInputStream in1 = new DataInputStream(clientSocket.getInputStream());
//
                while (!end) {
                    bytesRead = in1.read(messageByte);
                    dataString += new String(messageByte, 0, bytesRead);
                    if (dataString.length() > 0) {
                        end = true;
                    }
                }
                System.out.println("MESSAGE: " + dataString);
            } catch (Exception e) {
                e.printStackTrace();
            }
            // ===

            in.close();             

            clientSocket.close();
            System.out.print("connection closed");

        } catch (Exception e) {
            System.out.print("Whoops! It didn't work!\n");
            e.printStackTrace();
        }

Output

client Socket created ..

response:6

MESSAGE: ³CO³0³Œ

connection closed

When I skip step 1

client Socket created ..

MESSAGE: -

connection closed

How can I write code that reads all the input from server(acknowledgement msg & response message & it should also check the size of data and read accordingly)?

Referred to How to read all of Inputstream in Server Socket JAVA

Thanks


Answer:

These statements force the stream to be closed as soon as at least 1 byte is sent.

dataString += new String(messageByte, 0, bytesRead);
if (dataString.length() > 0) {
   end = true;
}

As soon as the ack of 6 is received the connection is closed. If the ack is ignored, the response may be sent in multiple packets, the connection would close after the first packet is received.

You must be able to determine when the ack and the response have ended.

  1. If you know how many characters are in the ack and response, you could loop like the question you referred to. You will need a loop for the ack and one for the response.
  2. If you do not know the number of characters, then you will have to look for special characters, like end-of-line or end-of-file to know when the ack line ends and when the server response ends. Look for a special character for the end of each message.
  3. A third possibility is to wait for a specific amount of time. When the time has expired, continue. This is not reliable over TCP, since data can be lost and resent. It may take more time than you allot for a response. I agree with the comment below. Originally, I hesitated to add 3 in the list. This option would most likely be used to detect a problem with the communication channel. If the time expires, then the client would give up with an error condition, it would not continue to process as if everything were alright.

Question:

I've been working on an app to move files between two hosts and while I got the transfer process to work (code is still really messy so sorry for that, I'm still fixing it) I'm kinda left wondering how exactly it handles the buffer. I'm fairly new to networking in java so I just don't want to end up with "meh i got it to work so let's move on" attitude.

File sending code.

    public void sendFile(String filepath, DataOutputStream dos) throws Exception{
    if (new File(filepath).isFile()&&dos!=null){
        long size = new File(filepath).length();
        String strsize = Long.toString(size) +"\n";
        //System.out.println("File size in bytes: " + strsize);
        outToClient.writeBytes(strsize);
        FileInputStream fis = new FileInputStream(filepath);
        byte[] filebuffer = new byte[8192];

        while(fis.read(filebuffer) > 0){
            dos.write(filebuffer);
            dos.flush();
        }

File recieving code

   public void saveFile() throws Exception{
    String size = inFromServer.readLine();
    long longsize = Long.parseLong(size);
    //System.out.println(longsize);
    String tmppath = currentpath + "\\" + tmpdownloadname;
    DataInputStream dis = new DataInputStream(clientSocket.getInputStream());
    FileOutputStream fos = new FileOutputStream(tmppath);
    byte[] filebuffer = new byte[8192];
    int read = 0;
    int remaining = (int)longsize;
    while((read = dis.read(filebuffer, 0, Math.min(filebuffer.length, remaining))) > 0){
        //System.out.println(Math.min(filebuffer.length, remaining));
        //System.out.println(read);
        //System.out.println(remaining);
        remaining -= read;
        fos.write(filebuffer,0, read);
    }

}

I'd like to know how exactly buffers on both sides are handled to avoid writing wrong bytes. (ik how receiving code avoids that but i'd still like to know how byte array is handled)

Does fis/dis always wait for buffers to fill up fully? In receiving code it always writes full array or remaining length if it's less than filebuffer.length but what about fis from sending code.


Answer:

In fact, your code could have a subtle bug, exactly because of the way you handle buffers.

When you read a buffer from the original file, the read(byte[]) method returns the number of bytes actually read. There is no guarantee that, in fact, all 8192 bytes have been read.

Suppose you have a file with 10000 bytes. Your first read operation reads 8192 bytes. Your second read operation, however, will only read 1808 bytes. The third operation will return -1.

In the first read, you write exactly the bytes that you have read, because you read a full buffer. But in the second read, your buffer actually contains 1808 correct bytes, and the remaining 6384 bytes are wrong - they are still there from the previous read.

In this case you are lucky, because this only happens in the last buffer that you write. Thus, the fact that you stop reading on your client side when you reach the pre-sent length causes you to skip those 6384 wrong bytes which you shouldn't have sent anyway.

But in fact, there is no actual guarantee that reading from the file will return 8192 bytes even if the end was not reached yet. The method's contract does not guarantee that, and it's up to the OS and underlying file system. It could, for example, send you 5000 bytes in your first read, and 5000 in your second read. In this case, you would be sending 3192 wrong bytes in the middle of the file.

Therefore, your code should actually look like:

byte[] filebuffer = new byte[8192];
int read = 0;
while(( read = fis.read(filebuffer)) > 0){
    dos.write(filebuffer,0,read);
    dos.flush();
}

much like the code you have on the receiving side. This guarantees that only the actual bytes read will be written.

So there is nothing actually magical about the way buffers are handled. You give the stream a buffer, you tell it how much of the buffer it's allowed to fill, but there is no guarantee it will fill all of it. It may fill less and you have to take care and use only the portion it tells you it fills.

Another grave mistake you are making, though, is to just convert the long that you received into an int in this line:

int remaining = (int)longsize;

Files may be longer than an integer contains. Especially things like long videos etc. This is why you get that number as a long in the first place. Don't truncate it like that. Keep the remaining as long and change it to int only after you have taken the minimum (because you know the minimum will always be in the range of an int).

long remaining = longsize;
long fileBufferLen = filebuffer.length;

while((read = dis.read(filebuffer, 0, (int)Math.min(fileBufferLen, remaining))) > 0){
    ...
}

By the way, there is no real reason to use a DataOutputStream and DataInputStream for this. The read(byte[]), read(byte[],int,int), write(byte[]), and write(byte[],int,int) are inherited from the underlying InputStream and there is no reason not to use the socket's OutputStream/InputStream directly, or use a BufferedOutputStream/BufferedOutputStream to wrap it. There is also no need to use flush until you have finished writing/reading.

Also, do not forget to close at least your file input/output streams when you are done with them. You may want to keep the socket input/output streams open for continued communication, but there is no need to keep the files themselves open, it may cause problems. Use a try-with-resources to guarantee that they are closed.

Question:

I've pasted a server side code snippet below. This server code works under normal circumstances, however, the following scenario manages to break the code. Server and client are on the same machine. I used the loopback address, and the actual IP address, it makes no difference.

Scenario

  1. Server is online, Client makes request (WritableByteChannel.write(ByteBuffer src) returns 12 byte, which is the correct size, but as research revealed that only means the 12 bytes are written to the TCP buffer).
  2. Server program is turned off. Client notices that the channel is closed on the remote side and closes it on its own side, it doesn't make any requests.
  3. Server is online again.
  4. Client tries to make a request, but fails, because the channel is closed/invalid and can't be reused (even though server is online again).
  5. Client checks server's online status, gets positive result, connects again and immediately makes another request.
  6. Server accepts client (code below), after that processes the if clause with the key.isReadable() condition, but then fails on the read, which indicates end-of-stream.

It would be too complex to create an SSCCE, please comment if important information is missing or this is too abstract and I'll provide further information.

Question

How can a freshly created/accepted channel fail on the read operation? What am I missing? What steps can I undertake to prevent this?

I already tried wireshark, but I can't capture any packets on the designated TCP port, even if the communication is acutally working.


Problem/Additional Info

  • It's possible to capture packets into .pcap file with RawCap
  • The problem was the way the client checked the server status. I've added the method below.

Code snippets

Snippet 1

    while (online)
    {
      if (selector.select(5000) == 0)
        continue;

      Iterator<SelectionKey> it = selector.selectedKeys().iterator();
      while (it.hasNext())
      {
        SelectionKey key = it.next();
        it.remove();

        if (key.isAcceptable())
        {
          log.log(Level.INFO, "Starting ACCEPT!");
          ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
          SocketChannel channel = serverSocketChannel.accept();

          channel.configureBlocking(false);
          channel.register(selector, SelectionKey.OP_READ);

          log.log(Level.INFO, "{0} connected to port {1}!",
              new Object[] {channel.socket().getInetAddress().getHostAddress(), isa.getPort()});
        }

        boolean accepted = false;
        if (key.isReadable())
        {
          log.log(Level.INFO, "Starting READ!");
          SocketChannel channel = (SocketChannel) key.channel();

          bb.clear();
          bb.limit(Header.LENGTH);
          try
          {
            NioUtil.read(channel, bb); // server fails here!
          }
          catch (IOException e)
          {
            channel.close();
            throw e;
          }
          bb.flip();

Snippet 2

  public static ByteBuffer read(ReadableByteChannel channel, ByteBuffer bb) throws IOException
  {
    while (bb.remaining() > 0)
    {
      int read = 0;
      try
      {
        read = channel.read(bb);
      }
      catch (IOException e)
      {
        log.log(Level.WARNING, "Error during blocking read!", e);
        throw e;
      }

      // this causes the problem... or indicates it
      if (read == -1)
      {
        log.log(Level.WARNING, "Error during blocking read! Reached end of stream!");
        throw new ClosedChannelException();
      }
    }

    return bb;
  }

Snippet 3

  @Override
  public boolean isServerOnline()
  {
    String host = address.getProperty(PropertyKeys.SOCKET_SERVER_HOST);
    int port = Integer.parseInt(address.getProperty(PropertyKeys.SOCKET_SERVER_PORT));

    boolean _online = true;
    try
    {
      InetSocketAddress addr = new InetSocketAddress(InetAddress.getByName(host), port);
      SocketChannel _channel = SocketChannel.open();
      _channel.connect(addr);
      _channel.close();
    }
    catch (Exception e)
    {
      _online = false;
    }
    return _online;
  }

Solution

The problem was not the method that checked, if the service is available/the server is online. The problem was the second point EJP mentioned.

Specific input was expected from the server and it was left in an inconsistent state if that conditions were not met. I've added some fallback measures, now the the reconnect process - including the check method - is working fine.


Answer:

Clearly the client must have closed the connection. That's the only way read() returns -1.

Notes:

  1. You're throwing the inappropriate ClosedChannelException when read() returns -1. That exception is thrown by NIO when you've already closed the channel and continue to use it. It has nothing to do with end of stream, and shouldn't be used for that. If you must throw something, throw EOFException.

  2. You also shouldn't loop the way you are. You should only loop while read() is returning a positive number. At present you are starving the select loop while trying to read data that may never arrive.

Question:

is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully? In my case I'm using it for sending a simple string through a java.net.Socket. In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock. What I would like to do is wait for the output to end, and then start reading the input.

Socket sk = new Socket("192.168.0.112", 3000);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write(message.getBytes());
bo.flush();

BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
if (line.equals("ack")) {
    System.out.println("ack");
}


sk.close();

Update

ServerSocket:

ServerSocket ss = new ServerSocket(3000);
        System.out.println("server socket open");
        while (true) {
            Socket sk = ss.accept();
            System.out.println("new connection");
            BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
            String line = br.readLine();
            System.out.println("received line: " + line);
            BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
            bo.write("ack".getBytes()); bo.flush();
            sk.close();
        }

Update:

@Global Variable - the reason that read was blocking the socket is that it was waiting for the \n, indeed. Using

bo.write("ack\n".getBytes());

instead of

bo.write("ack".getBytes());

made it work.

Regarding the initial question, is there a way of knowing if flush() method has finished successfully, @Stephen C provided the answer:

there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.

This "reply" is implemented in the code sample and it works.


Answer:

Is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully?

It depends on what you mean by "finished successfully".

The flush() method ensures that all unsent data in the pipeline has been pushed as far as the operating system network stack. When that is done, then you could say that flush() has finished successfully. The way that you know that that has happened is that the flush() call returns.

On the other hand, if you want some assurance that the data has (all) been delivered to the remote machine, or that the remote application has read it (all) ... there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.


In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock.

The code that you are talking about is basically the correct approach. The way to wait for the response is to read it like that.

If it is not working, then you need to compare what the client and server side are doing:

  • Is the server waiting for the client to send something more? Maybe an end of line sequence?

  • Did the server sends the response?

  • Did it flush() the response?

A mismatch between what the client and server are doing can lead to a form or deadlock, but the solution is to fix the mismatch. Waiting for some kind of hypothetical confirmation of the flush() is not the answer.

UPDATE

The problem is indeed a mismatch. For example, the server writes "ack" but the client expects "ack\n". The same happens in the client -> server case ... unless message always ends with a newline.

Question:

My ultimate goal here is to stream a user's voice input into an Android device over to a Desktop application.

For the Android device, it will be obviously be running a java based Android application. For the Desktop application, I'm considering writing a Java applet to accept the stream.

These are the benefits and drawbacks of TCP and UDP as explained by Wikipedia

Transmission Control Protocol is a connection-oriented protocol, which means that it requires handshaking to set up end-to-end communications. Once a connection is set up, user data may be sent bi-directionally over the connection.

Reliable – TCP manages message acknowledgment, retransmission and timeout. Multiple attempts to deliver the message are made. If it gets lost along the way, the server will re-request the lost part. In TCP, there's either no missing data, or, in case of multiple timeouts, the connection is dropped. Ordered – If two messages are sent over a connection in sequence, the first message will reach the receiving application first. When data segments arrive in the wrong order, TCP buffers delay the out-of-order data until all data can be properly re-ordered and delivered to the application. Heavyweight – TCP requires three packets to set up a socket connection, before any user data can be sent. TCP handles reliability and congestion control. Streaming – Data is read as a byte stream, no distinguishing indications are transmitted to signal message (segment) boundaries. User Datagram Protocol is a simpler message-based connectionless protocol. Connectionless protocols do not set up a dedicated end-to-end connection. Communication is achieved by transmitting information in one direction from source to destination without verifying the readiness or state of the receiver.

Unreliable – When a UDP message is sent, it cannot be known if it will reach its destination; it could get lost along the way. There is no concept of acknowledgment, retransmission, or timeout. Not ordered – If two messages are sent to the same recipient, the order in which they arrive cannot be predicted. Lightweight – There is no ordering of messages, no tracking connections, etc. It is a small transport layer designed on top of IP. Datagrams – Packets are sent individually and are checked for integrity only if they arrive. Packets have definite boundaries which are honored upon receipt, meaning a read operation at the receiver socket will yield an entire message as it was originally sent. No congestion control – UDP itself does not avoid congestion. Congestion control measures must be implemented at the application level. Broadcasts - being connectionless, UDP can broadcast - sent packets can be addressed to be receivable by all devices on the subnet.

I would like the user's voice stream to be uninterrupted and steady when the Android is transmitting it to the desktop application.

Beyond simply considering the benefits and drawbacks of the two different technologies as Wikipedia lists, what else should I be aware of when choosing a technology?

Which is the better choice UDP or TCP?


Answer:

In my opinion, it also rely on your project's specification. The lost of signal is acceptable or not. UDP is extremely faster than TCP which is suitable to stream a user's voice input. It looks like traditional telephone call, sometime you hear some "noise" but the call is still good enough for us to use. Moreover, using TCP would bring you a significant delay and UDP was born for streaming.

Question:

I'm creating a Java Client program that sends a command to server and server sends back an acknowledgement and a response string.

The response is sent back in this manner

client -> server : cmd_string

server -> client : ack_msg(06)

server -> client : response_msg

Client code

public static void readStream(InputStream in) {

    byte[] messageByte = new byte[20];// assuming mug size -need to
                                        // know eact msg size ?
    boolean end = false;
    String dataString = "";
    int bytesRead = 0;

    try {
        DataInputStream in1 = new DataInputStream(in);
        // while ctr==2 todo 2 streams
        int ctr = 0;
        while (ctr < 2) {//counter 2 if ACK if NAK ctr=1 todo
            bytesRead = in1.read(messageByte);

            if (bytesRead > -1) {
                ctr++;
            }
            dataString += new String(messageByte, 0, bytesRead);

            System.out.println("\ninput byte arr "+ctr);
            for (byte b : messageByte) {
                char c=(char)b;
                System.out.print(" "+b);
            }
        }


        System.out.println("MESSAGE: " + dataString + "\n bytesread " + bytesRead + " msg length "
                + dataString.length() + "\n");
        char[] chars = dataString.toCharArray();
        ArrayList<String> hex=new ArrayList<>();
        // int[] msg ;
        for (int i = 0; i < chars.length; i++) {
            int val = (int) chars[i];
            System.out.print(" " + val);
        hex.add(String.format("%04x",  val));
        }
        System.out.println("\n"+hex);

    } catch (Exception e) {
        e.printStackTrace();
    }
    // ===

}

Output

    client Socket created .. 
     response:

     input byte arr 1
     6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    input byte arr 2
     2 -77 67 79 -77 48 -77 3 -116 0 0 0 0 0 0 0 0 0 0 0
    MESSAGE: ##³CO³0³##
     (where # is some not supported special character )
     bytesread 9 msg length 10

     dec: 6 2 179 67 79 179 48 179 3 338
    hex: [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]
    bytes: 2 -77 67 79 -77 48 -77 3 -116 0 0 0 0 0 0 0 0 0 0 0 (bytes recieved in 2nd packet)
    connection closed

Problem: I'm reading the last value incorrect, I have verified using wireshark the server has sent back the response as 06 02 b3 43 4f b3 30 b3 03 8c

Some how I'm reading the last value in correctly. Is there some issue with the reading stream?

EDIT

Rest of the response is read correctly but the last character should be 8c But is read as 0152Hex

Response from server : 06 02 b3 43 4f b3 30 b3 03 8c

Read by program : [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]

issue with reading the last character

EDIT 2

Response is received as 2 packets/streams

packet 1 byte arr :  6 (ACK)

packet 2 byte arr:  2 -77 67 79 -77 48 -77 3 -116 (response)

complete response read by client

  dec: 6 2 179 67 79 179 48 179 3 338
  hex: [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]

Thanks


Answer:

The problem in this question was a matter of signed variables versus unsigned variables. When you have a number in computer memory, it is represented by a bunch of bits, each of them 0 or 1. Bytes are generally 8 bits, shorts are 16 etc. In an unsigned number, 8 bits will get you from positive 0 to 255, but not to negative numbers.

This is where signed numbers come in. In a signed number, the first bit tells you whether the following bits represent a negative or positive value. So now you can use 8 bits to represent -128 to +127. (Notice that the positive range is halved, from 255 to 127, because you "sacrifice" half of your range to the negative numbers).

So now what happens if you convert signed to unsigned? Depending on how you do it, things can go wrong. In the problem above, the code char c=(char)b; was converting a signed byte to an unsigned char. The proper way to do this is to "make your byte unsigned" before converting it to a char. You can do that like this: char c=(char)(b&0xFF); more info on casting a byte here.

Essentially, you can just remember that except for char, all java numbers are signed, and all you need to do is paste the &0xFF to make it work for a byte, 0xFFFF to make it work for a short, etc.

The details about why this works are as follows. Calling & means a bitwise and, and 0xFF is hexadecimal for 255. 255 is above the limit of a signed byte (127), so the number b&0xFF gets upgraded to a short by java. However, the short signed bit is on bit 16, while the byte signed bit is on bit 8. So now the byte signed bit becomes a normal 'data' bit in the short, and so the sign of your byte is essentially discarded.

If you do a normal cast, java recognizes that doing direct bitconversion like above would mean that you lose the sign, and java expects you don't like that (at least, that is my assumption), so it preserves the sign for you. So if this isn't what you want, you explicitly tell java what to do.

Question:

Right now I am displaying a live stream on a Surface using the vitamio librairy.

I wish to record it in order to watch it later as the feed comes from a drone.

Right now I am periodically saving a series of bitmaps using jcodec :

 SequenceEncoder enc = new SequenceEncoder(new File("filename"));
// GOP size will be supported in 0.2
// enc.getEncoder().setKeyInterval(25);
for(...) {
    BufferedImage image = ... // Obtain an image to encode
    enc.encodeImage(image);
}
enc.finish();

My problems are :

  • The h264 output from jcodec is corrupted (there is a lot of color patches on the frames)
  • I can't record at a very high framerate (I'd like 30fps).

Moreover I still need to find a way to mux it.. I looked into Mp4Parser without much succes.


Answer:

Why not to use data input stream:

DataInputStream in = new DataInputStream (recording.getInputStream());
FileOutputStream videoFile = new FileOutputStream(targetFile);
int len;
byte buffer[] = new byte[8192];

while((len = in.read(buffer)) != -1) {
    videoFile.write(buffer, 0, len);
}

videoFile.close();

Then, you can play the video from file.

Question:

Netty TCP Server is running at port 8000 receiving NMEA format data. It uses Marine API library to convert the gibberish to a meaningful information which needs input stream from the socket.

SentenceReader sentenceReader = new SentenceReader(socket.getInputStream());
sentenceReader.addSentenceListener(new MultiSentenceListener());
sentenceReader.start();

How can i get inputstream for netty server port being used?


Answer:

SentenceReader does not have any method to accept "streamed in" data, however with subclassing, it can be made to accept the data.

The core of SentenceReader uses a DataReader for its data, normally this datareader is polled from a seperate thread SentenceReader itself, and we can modify this structure to get what we need.

First, we subclass SentenceReader with our own class, give it the proper constructor and methods we want, and remove the effect of the start and stop methods. We provide null as the file for now (and hope future versions provide a method to pass a datareader in directly)

public class NettySentenceReader extends SentenceReader {
    public NettySentenceReader () {
        super((InputStream)null);
    }

    @Override
    public void start() {
    }

    @Override
    public void stop() {
    }
}

We now need to implement all functionality of the internal class DataReader inside our own Netty handler, to replicate the same behaviour

public class SentenceReaderHandler extends
         SimpleChannelInboundHandler<String> {
    private SentenceFactory factory;
    private SentenceReader parent;

    public SentenceReaderHandler (SentenceReader parent) {
        this.parent = parent;
    }

    @Override
    public void channelRegistered(ChannelHandlerContext ctx) {
        if(!ctx.channel().isActive())
            return;
        //ActivityMonitor monitor = new ActivityMonitor(parent);
        this.factory = SentenceFactory.getInstance();
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) {
        //ActivityMonitor monitor = new ActivityMonitor(parent);
        this.factory = SentenceFactory.getInstance();
    }

    @Override
    // This method will be renamed to `messageReceived` in Netty 5.0.0
    protected void channelRead0(ChannelHandlerContext ctx, String data)
             throws Exception {
        if (SentenceValidator.isValid(data)) {
            monitor.refresh();
            Sentence s = factory.createParser(data);
            parent.fireSentenceEvent(s);
        } else if (!SentenceValidator.isSentence(data)) {
            parent.fireDataEvent(data);
        }
    }

    @Override
    public void channelInactive(ChannelHandlerContext ctx) {
        //monitor.reset();
        parent.fireReadingStopped();
    }

    @Override
    public void channelUnregistered(ChannelHandlerContext ctx) {
        if(!ctx.channel().isActive())
            return;
        //monitor.reset();
        parent.fireReadingStopped();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable e) {
        parent.handleException("Data read failed", e);
    }
}

Finally, we need to integrate this into a Netty pipeline:

SentenceReader reader = new NettySentenceReader();
bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
    private static final StringDecoder DECODER = new StringDecoder();
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
        pipeline.addLast(DECODER);
        pipeline.addLast(new SentenceReaderHandler(reader)); 
    }
});

Question:

Android/Java provides socket APIs for receiving data from TCP server, like:

socket.getInputStream().read(payload);

The payload is data stripped off TCP header as well as IP header. My question is that possible to get the TCP header? Thanks.


Answer:

You can't access TCP headers as explained here: Reading and writing TCP header (options) in Java

The thing is that without monitoring your network, you won't be able to access that information from the Java API.

What you might want to try is looking at this library if you really need the TCP headers http://jnetpcap.com.

This library is able to read the packets in real time. You would just have to match the packet you received with your socket in Java to the correct packet received by the library.

Question:

A little bit of context: the client is sending to the server a SOSPFPacket object (via TCP) that has various attributes, such as a Vector<LSA> lsaArray. The LSA itself has a LinkedList<LinkDescription> links attribute. In my test case, there are two messages being sent. In both messages, there is only one LSA in the vector. In the first message, the LSA has one LinkDescription, in the second, it has two. When I send a message, I increment the messageId.

The server receives both messages with proper ids, but in the second message, the links only contain one link instead of two. I'm clueless...

Here are the object implementations:

import java.io.*;
import java.util.Vector;

public class SOSPFPacket implements Serializable {
  public final static short HELLO = 0;
  public final static short LSU = 1;
  public final static short OVER_BURDENED = 2;
  public static int id = Integer.MIN_VALUE;

  public String srcProcessIP;
  public short srcProcessPort;
  public String srcIP;
  public String dstIP;
  public short sospfType; //0 - HELLO, 1 - LinkState Update, 2 - Over Burdened
  public String routerID;
  public int messageId = id++;
  public String neighborID; //neighbor's simulated IP address
  public Vector<LSA> lsaArray = new Vector<>();
  public String lsaInitiator = null;
}

import java.io.Serializable;
import java.util.LinkedList;

public class LSA implements Serializable {
  public String linkStateID;
  public int lsaSeqNumber = Integer.MIN_VALUE;
  public LinkedList<LinkDescription> links = new LinkedList<LinkDescription>();

  @Override
  public String toString() {
    StringBuffer sb = new StringBuffer();
    sb.append(linkStateID + ":").append(lsaSeqNumber + "\n");
    for (LinkDescription ld : links) {
      sb.append(ld);
    }
    sb.append("\n");
    return sb.toString();
  }
}

import java.io.Serializable;

public class LinkDescription implements Serializable {
  public String linkID;
  public int portNum;
  public int tosMetrics;

  public LinkDescription() {}

  public LinkDescription(String linkID, int portNum, int tosMetrics) {
    this.linkID = linkID;
    this.portNum = portNum;
    this.tosMetrics = tosMetrics;
  }

  public String toString() {
    return linkID + ","  + portNum + "," + tosMetrics;
  }
}

To send the message, I do it via a Client.java thread implementing Runnable. Here are the relevant methods:

public void run() {
    try {
        _outputStream = new ObjectOutputStream(_clientSocket.getOutputStream());
        sendMessage(SOSPFPacket.HELLO);
        _inputStream = new ObjectInputStream(_clientSocket.getInputStream());
        SOSPFPacket message = Util.receiveMessage(_inputStream);

        if (message.sospfType == SOSPFPacket.OVER_BURDENED) {
            System.out.println("Removing link with router " + message.srcIP + "...");
            _router.removeLink(_remoteRouterIP);
            return;
        }

        _remoteRouterDescription.setStatus(RouterStatus.TWO_WAY);
        _router.addLinkDescriptionToDatabase(_remoteRouterDescription, _link.getWeight());
        sendMessage(SOSPFPacket.HELLO);
        message = Util.receiveMessage(_inputStream);

        if (message.sospfType == SOSPFPacket.LSU) {
            _router.synchronize(message.lsaArray);
        }

        _router.propagateSynchronization(message.lsaInitiator, message.srcIP);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

private void sendMessage(short messageType) {
    try {
        SOSPFPacket message = Util.makeMessage(_rd, _remoteRouterDescription, messageType, _router);
        _outputStream.writeObject(message);
        _outputStream.flush();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

public class Util {
    public static SOSPFPacket makeMessage(RouterDescription local, RouterDescription external, short messageType, Router rd) {
        SOSPFPacket message = new SOSPFPacket();
        message.srcProcessIP = local.getProcessIPAddress();
        message.srcProcessPort = local.getProcessPortNumber();
        message.srcIP = local.getSimulatedIPAddress();
        message.dstIP = external.getSimulatedIPAddress();
        message.sospfType = messageType;
        message.routerID = local.getSimulatedIPAddress();
        message.neighborID = external.getSimulatedIPAddress();
        rd.getLsd().getStore().forEach((k, v) -> message.lsaArray.addElement(v));
        message.lsaInitiator = messageType == SOSPFPacket.LSU ? message.srcIP : null;

        return message;
    }

    public static SOSPFPacket receiveMessage(ObjectInputStream inputStream) {
        SOSPFPacket receivedMessage = null;

        try {
            receivedMessage = (SOSPFPacket) inputStream.readObject();

            String messageType;

            switch (receivedMessage.sospfType) {
                case SOSPFPacket.HELLO:
                    messageType = "HELLO";
                    break;
                case SOSPFPacket.LSU:
                    messageType = "LINKSTATEUPDATE";
                    break;
                case SOSPFPacket.OVER_BURDENED:
                    messageType = "OVER_BURDENED";
                    break;
                default:
                    messageType = "UNKNOWN_STATE";
                    break;
            }

            System.out.println("received " + messageType + " from " + receivedMessage.srcIP + ";");
        } catch (ClassNotFoundException e) {
            System.out.println("No message received.");
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

        return receivedMessage;
    }
}

And the server instantiates a private ClientServiceThread when it receives a new connection, which is in charge of receiving the message.

private class ClientServiceThread implements Runnable {
    Socket _clientSocket;
    Thread _runner;

    ClientServiceThread(Socket s) {
        _clientSocket = s;
        _runner = new Thread(this);
    }

    public Thread getRunner() { return _runner; }

    public void run() {
        ObjectInputStream inputStream = null;
        ObjectOutputStream outputStream = null;

        try {
            inputStream = new ObjectInputStream(_clientSocket.getInputStream());
            outputStream = new ObjectOutputStream(_clientSocket.getOutputStream());

            while (true) {
                try {
                    SOSPFPacket receivedMessage = Util.receiveMessage(inputStream);

                    //some logic not relevant since the receivedMessage is already not correct
                }
            }
        }
    }
}

Again, all SOSPFPacket fields are correctly received, except for the Vector<LSA> lsaArray...


Edit: I also tried sending a third sendMessage(SOSPFPacket.HELLO) after _router.propagateSynchronization(message.lsaInitiator, message.srcIP);. This time, the message being sent contains two LSA, the first one having two LinkDescription, the second one having one. Both LSA are received by the server, but still, only the first LinkDescription is received in the first LSA. The message id is correct in all three messages. If I run everything a second time (i.e. I create a new Client and a new ClientService Thread for the already running routers), only then does the server finally receive two LinkDescription in the first LSA.


Answer:

Java sends references to objects that have already been serialized, to preserve the integrity of object graphs.

You should call ObjectOutputStream.reset() after each writeObject().

Or use ObjectOutputStream.writeUnshared(), but note that it still shares referenced objects, i.e. if you try to send a list with both added and changed element objects, it will send the new list and new element objects, but not the element objects which have been changed.

Question:

I'm trying to send a blank TCP packet to a server running on port 80 however it gives me an EOFException when the server responds.

java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2624) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3099) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853) at java.io.ObjectInputStream.(ObjectInputStream.java:349) at me.adamstephenson.test.main.main(main.java:106)

ArrayList<String> results = getIPs();

    for (String ip : results){
        System.out.println(ip);

        String pingip = ip;
        pingip = pingip.replaceAll("[^\\d.]", "");
        System.out.println(pingip);

        Socket socket = null;
        ObjectOutputStream oos = null;
        ObjectInputStream ois = null;
        try {
            socket = new Socket(pingip, 80);
            oos = new ObjectOutputStream(socket.getOutputStream());

            ois = new ObjectInputStream(socket.getInputStream());  // error here

            String msg = "blank";
            oos.write(msg.getBytes());
            oos.flush();

            //read the server response message
            String message = (String) ois.readObject();

            System.out.println(message);

            ois.close();
            oos.close();
            socket.close();
        } catch (ConnectException | EOFException e) {
            System.out.println("Failed to connect to: "+pingip);
            e.printStackTrace();
        }


    }

Does the EOFException mean that the response was blank?


Answer:

It means that the server didn't even create an ObjectOutputStream before closng the socket. Probably you sent it something it didn't understand, and maybe the application protocol it uses doesn't involve Serialization at all.

Port 80 is reserved for HTTP after all.

Question:

I have a Java server that will have two difference types of clients, a Java based console, and a Lua based client that the server will be controlling. I have got the Lua client to talk back and forth with

out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));

I am now trying to get a Java based client to connect and I would like to use

out = new ObjectOutputStream(new BufferedOutputStream(socket.getOutputStream()));
in = new ObjectInputStream(new BufferedInputStream(socket.getInputStream()));

The problem I am having is understanding the best way to go about accepting connections, not knowing if it is a Lua or Java client on the other end. Then setting up the proper streams.

Should I just use different ports on the initial connection? I was hoping something more along the lines of sending a string from ObjectOutputStream that BufferedReader could read. Or should I use something lower level to see what is connecting before setting up if it is PrintWriter or ObjectOutputStream.

A link to an in depth tutorial or actual code would work. I am having trouble finding resources beyond the basic setup of one or the other.

Thanks in advanced.


Answer:

Inasmuch as you propose to provide what sounds like two completely different services, it would be natural and appropriate to provide them at different ports. To implement that, your server would create and manage two separate ServerSockets, listening on different ports. One would provide one service; the other would provide the other. You could use either threads or a Selector or a combination of those to handle the two server sockets and all the clients.

As a separate matter, I urge you to think carefully before relying on Object streams. If you can implement your service on top of a simpler and/or more general protocol then you will avoid the multiple joys of Java Serialization, and you will also leave open the door for clients written in other languages.

Question:

Hy guys I've been trying to write a program that uses TCP to transmit JSON strings. The problem is that when I call BufferedInputStream.read() I can't receive any -1 to indicate the end of a JSON string.

My server code is:

public void listen() {
    try {
        if (getClient() != null) {
            String jst = null;
            Message msg = null;
            InputStream in = getClient().getInputStream();
            BufferedInputStream inBuf = new BufferedInputStream(in);
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            while (getClient() != null) {
                System.out.println("Receiving Data...");
                try {
                    int i;  
                    while ((i = inBuf.read()) != -1) {  
                        baos.write(i);  
                    }  
                    jst = baos.toString();
                } catch (IOException e) {
                    System.err.println("Bad Connection.");
                    stopStream();
                    break;
                }
                System.out.println(jst);
                msg = MessageFactory.fromJSON(jst);
                if (msg.getType().equals(StartMessage.defaultType)) {
                    startStream((StartMessage) msg);
                } else if (msg.getType().equals(StreamMessage.defaultType)) {
                    streaming((StreamMessage) msg);
                } else if (msg.getType().equals(
                        StopStreamMessage.defaultType)) {
                    stopStream();
                }
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
        System.err.println(e.getMessage());
    }
}

My client code is:

public void sendStart(OutputStream out) throws IOException {
    String jst = makeStart();
    out.write(jst.getBytes());
    System.out.println("data sent: " + jst);
}

When I was debugging it I saw that there was data that baos.write(i); wrote into i. However the it seems like there is no -1 in the data so I can only see jst = baos.toString(); either I terminate the client or wait for the time out.

This is what I got:

{"format":"raw","width":320,"type":"startstream","height":240}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}{"data":"[B@41635617","type":"image"}
Unexpected token LEFT BRACE({) at position 62.
    at org.json.simple.parser.JSONParser.parse(Unknown Source)
    at org.json.simple.parser.JSONParser.parse(Unknown Source)
    at org.json.simple.parser.JSONParser.parse(Unknown Source)
    at au.edu.benjamin.message.MessageFactory.fromJSON(MessageFactory.java:14)
    at au.edu.benjamin.transmission.TransListenerThread.listen(TransListenerThread.java:78)
    at au.edu.benjamin.transmission.TransListenerThread.run(TransListenerThread.java:53)
    at java.lang.Thread.run(Thread.java:745)
null
Exception in thread "Thread-2" java.lang.NullPointerException
    at au.edu.benjamin.transmission.TransListenerThread.listen(TransListenerThread.java:79)
    at au.edu.benjamin.transmission.TransListenerThread.run(TransListenerThread.java:53)
    at java.lang.Thread.run(Thread.java:745)

The exception is because of the unexpected '{', which should be separated from the former JSON string but it didn't.

Can someone tell me where is the problem? Thanks a lot!


Answer:

A ImputStream.read() call on a TCP stream doesn't return -1 once the message ends, instead it is waiting for a new byte (in fact, such a stream only returns -1 if it is closed).

In order to achieve the functionality you want, you could use a string termination character in the end (\0) if you don't use it somewhere else in your message or you could send the size of the JSON-message before you send it itself and read in that size.

Question:

in my quest to learn Java / Android development, I’m running into lots of roadblocks. Mainly because I don’t really know much about threading and communication between threads/processes. I’m trying to stream the IMU data from an android device to a python application on a computer. Whenever the sensor values change, a sensor listener saves the current values into a variable for the network handler to access.

The network handler in turn is supposed to run on a timer, sending the values and a current timestamp at a more or less fixed rate of 33Hz (perhaps a bit fast? well, I’d be willing to accept as slow as 10Hz, but no slower than that). Anyway, when I tested this, I could see on the computer interface that the data isn’t nearly coming in at a steady pace of 30 per second, but rather comes in surges, sometimes not coming at all for a second, and overall accumulating quite the delay (ie. the later the values are, the more delayed they come in). I understand there might be some variability in the network and some lags, but I would at least like the overall pace to at least be correct, ie that it doesn’t get worse and worse the longer I’m sending.

Considering the devices are both on a common wifi network, and I’m capable of streaming 1080p video without any lags over wifi, I’m fairly confident that the protocol should be able to handle a 64 Byte string every 30ms without troubles. To eliminate the sensor reader as an problem source, I made a minimum working example that simply sends a string every 30ms, without any sensor reading. I basically got this code from various stackoverflow posts, and modified it until it more or less did what I wanted. The problem is that the network interface runs in an AsynchronousTask, for which I am uncertain how to access it once it has been started. My theory is that it’s wasting resources to open a new socket for every new data packet, but I’m unsure how to open the socket once in the background and then pass the values to it on a timer and tell it to send.

Here’s the basic activity I made to test this:

package com.jamesdoesntlikejava.motionlearning15;

import android.support.v7.app.ActionBarActivity;
import android.os.Bundle;
import android.view.Menu;
import android.view.MenuItem;
import android.widget.Toast;
import java.util.Timer;
import java.util.TimerTask;


public class SendValuesActivity extends ActionBarActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_send_values);
        // creating timer task, timer
        final Timer timer = new Timer();
        TimerTask taskNew = new TimerTask() {
            @Override
            public void run() {
                int counter = 0;
                int numsteps = 333;
                String params[] = new String[2];
                if (counter < numsteps) {
                    params[0] = "192.168.1.33";
                    long currentTime = System.currentTimeMillis();
                    params[1] = Long.toString(currentTime)+"blablabla";
                    new ServerCommunicationTask().execute(params);
                    counter++;
                } else  {
                    timer.cancel();
                    timer.purge();
                }
            }
        };
        // scheduling the task at fixed rate delay
        Toast.makeText(this, "Sending Values in 1s...", Toast.LENGTH_SHORT).show();
        timer.scheduleAtFixedRate(taskNew,1000,30);
    }


    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_send_values, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }
}

And this is the class for doing the networking:

package com.jamesdoesntlikejava.motionlearning15;

import android.os.AsyncTask;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.net.Socket;
import java.net.UnknownHostException;

public class ServerCommunicationTask extends AsyncTask<String, Void, String> {

    public final static int TCP_SERVER_PORT = 13337;
    // params are 0: the target IP and 1: the message to send.
    @Override
    protected String doInBackground(String[] params) {

        String TCP_SERVER_IP = params[0];
        try {
            Socket s = new Socket(TCP_SERVER_IP, TCP_SERVER_PORT);
            BufferedWriter out = new BufferedWriter(new OutputStreamWriter(s.getOutputStream()));
            //send output msg
            String outMsg = params[1];
            out.write(outMsg);
            out.flush();
            //close connection
            s.close();
        } catch (UnknownHostException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return "success";
    }

    @Override
    protected void onPostExecute(String response) {
    }
}

Running on a Moto G LTE (updated 1st gen) with android 5.1. Any hints are appreciated, thanks!


Answer:

Instead of AsyncTask and always opening new connection, you can use Thread.

import java.io.BufferedWriter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.ArrayList;

public class ServerCommunicationThread extends Thread {

    public final static int TCP_SERVER_PORT = 13337;

    private ArrayList<String> mMessages = new ArrayList<>();
    private String mServer;

    private boolean mRun = true;

    public ServerCommunicationThread(String server) {
        this.mServer = server;
    }

    @Override
    public void run() {

        while (mRun) {
            Socket s = null;
            try {
                s = new Socket(mServer, TCP_SERVER_PORT);
                BufferedWriter out = new BufferedWriter(new OutputStreamWriter(s.getOutputStream()));

                while (mRun) {
                    String message;

                    // Wait for message
                    synchronized (mMessages) {
                        while (mMessages.isEmpty()) {
                            try {
                                mMessages.wait();
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                        }
                        // Get message and remove from the list
                        message = mMessages.get(0);
                        mMessages.remove(0);
                    }

                    //send output msg
                    String outMsg = message;
                    out.write(outMsg);
                    out.flush();
                }

            } catch (UnknownHostException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
                //close connection
                if (s != null) {
                    try {
                        s.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }
        }
    }

    public void send(String message) {
        synchronized (mMessages) {
            mMessages.add(message);
            mMessages.notify();
        }
    }

    public void close() {
        mRun = false;
    }
}

You can keep the thread running with connection opened and send message when needed.

ServerCommunicationThread thread = new ServerCommunicationThread("192.168.1.33");
thread.start();
...
thread.send("blablabla");
...
thread.send("blablabla");
...
thread.close();

Please note that this code is not tested.

Question:

I am connecting to some legacy server socket, using spring integration framework.

The below is my client factory and adaptor:

<int-ip:tcp-connection-factory id="client"
                               type="client" 
                               host="${tcpServer}" 
                               port="${tcpPort}" 
                               single-use="false"
                               using-nio="false" />

<int-ip:tcp-inbound-channel-adapter id="inboundServer"
                                    client-mode="true"
                                    channel="inputStream" 
                                    error-channel="errorChannel"
                                    retry-interval="${retryInterval}"
                                    connection-factory="client" />

And below the stream to string converter:

    <int:transformer id="clientBytes2String" 
                 input-channel="inputStream"
                 output-channel="inputString" 
                 expression="new String(payload)" />

                 <int:channel id="inputString" />

And the following section is empty, as I am not sure what to implement here, so that it can call my router and router will do it's business.


I have tried with splitter, it did work, if the stream comes in desired format of "ABCD EFGH WXYZ" or "ABCD" but if the stream comes like "ABCD XXXX EFGH WXYZ" then its failing. Desired results is it should process 3 messages and 1 error. But instead it processed 1 messages and all rest are ignored.

The code below:

 <int:splitter input-channel="inputString"
              output-channel="preRouter2"                  
              method="splitMessage"
              ref="messageSplitterBean"/>

And MessageSpliterBean class as follows:

@Splitter
public List<Message<?>> splitMessage( Message<?> message ) {

    List<Message<?>> msgFragments = new ArrayList<Message<?>>();

    //Let say I am assuming message will be coming ABCD EFGH WXYZ
    String str[] = message.getPayload().toString().split(" ");
    int counter = 1;
    for ( String s : str ) {

        Message<String> resultMessage = MessageBuilder.withPayload( s )
                .copyHeaders(message.getHeaders())
                .build();
        msgFragments.add(resultMessage);
    }

    return msgFragments;
}

And the following will be my router, which will send to respective channel based on some expression:

<int:recipient-list-router id="customRouter" input-channel="preRouter2">
<int:recipient channel="input1" selector-expression="payload.toString().startsWith('ABCD')"/>   
<int:recipient channel="input2" selector-expression="payload.toString().startsWith('EFGH')"/>
<int:recipient channel="input3" selector-expression="payload.toString().startsWith('WXYZ')"/>

Need your expert view on this: on what I am doing wrong, or what will be the best approach.

The input from server socket will be in stream of data with fixed length and space as separator. And each fixed length I need to convert them into a message and send it to concerned channel.

Regards.


Answer:

First of all no reason to implement your own Splitter because the default one has delimiters option:

Another my point is about redundant <int:transformer> for the byte[] -> String. The Spring Integration provides for you ObjectToStringTransformer out-of-the-box.

And your issue is <int:recipient-list-router>. As you say you may have something wrong in your data and your router isn't ready to process such a message and it fails for your. That's just because (AbstractMessageRouter):

else {
    throw new MessageDeliveryException(message, "No channel resolved by router '" + this.getComponentName()
                + "' and no 'defaultOutputChannel' defined.");
}

Which happens when no one selector-expression accepts your wrong message.

In this case it is sent to the error-channel="errorChannel" on your <int-ip:tcp-inbound-channel-adapter> and it stops the further process just because the MessageDeliveryException.

As you see it looks like to fix your issue you should add default-output-channel to the recipient-list-router configuration.

Question:

I am connecting to an device with opening an socket. To get incoming data I perform an readaction on the InputStream in a different thread. When I take away the electricity of the peer device I am connected to, my InputStream doesn't recognize the loss of connection.

This is my code to wait for input:

StringBuilder sb = new StringBuilder();
String result = "";
int c;
try
{
  log.info( "waiting for data..." );
  while ( ( ( c = inputStream.read() ) >= 0 ) )
  {
    if ( c == -1 )
    {
      log.info( "is -1" );
    }
    /*
     * TODO: <LF> Can't always be the delimiter to define the end of an message. This should be
     * parameterized.
     */
    if ( c == 0x0a /* <LF> */ )
    {
      result = sb.toString();
      sb.delete( 0, sb.length() );
    }
    else if ( c != 0x0d /* <CR> */ )
    {
      sb.append( (char) c );
    }
    if ( !result.isEmpty() )
    {
      log.info( getName() + ": received message: " + result );
      listener.MessageReceived( result.getBytes() );
      result = "";
    }
  }
  log.info( "stream ended" );
  disconnect();
  listener.closed();
}
catch ( IOException | ResourceException e )
{
  try
  {
    log.info( "in catch block" );
    disconnect();
    listener.closed();
    throw new ResourceException( "An error occured during the receiving of a message for the device, or connection timed out.", e );
  }
  catch ( ResourceException e1 )
  {
    e1.printStackTrace();
  }
}

This is inside of an JCA connector if that information is for use in any case. To my knowledge the InputStream receives -1 when the Stream is interrupted and normally he should jump to my stream endedlog but it doesn't happen.

why doesn't it recognize that the connection can't be available, since the remote peer is powered off?


Answer:

As you say, you don't want a timeout because you need to wait for the peer even if it doesn't send for hours. Barring special measures, there is no difference between a peer that doesn't send for hours and a peer that has been turned off. As long as no packets are sent, it's impossible to detect the difference.

You can do one thing to ensure that packets are sent: you can turn on the SO_KEEPALIVE socket option using the method Socket.setKeepAlive(true).

The problem is that you can't control from Java how often the keep-alive probes are sent. This typically depends on settings in your operating system kernel. Still, it will allow you to detect a dead (or unreachable) peer quicker than "never".

Question:

I am getting EOFException from the following code:

if (!(in.read() == -1))
{
    CANDataInfo canData = (CANDataInfo) in.readObject();
    System.out.println(canData.toString());
    bw.write(canData.toString());
}
else
{
    System.out.println("in.read() == -1 "+in.readObject());
    jLab0x28.setText("No more bytes to read ");
}

I am doing an Socket programming where server is sending continuos data to client at some interval. The Data whichis passed from server to client in via socket is of type CANDataInfo object which i have developed. At client side while i am printing the data i am getting exception. Since object's read is always -1 i am not able to log the data on some file.

Server Side Code:

private ServerSocket server = null;
private Socket client = null;
private ObjectOutputStream out;
public static final String TAG = "APP1";

private void structureData(CANDataInfo canDataInfo)
{
    try 
    { 
        if(server == null) 
        {   
            server = new ServerSocket(38301);
            server.setSoTimeout(0);
        }
        client = server.accept();
        Log.e("Server ", ""+client.isConnected());
        Log.e("Data ", ""+canDataInfo.toString());

        if(!client.isConnected())
        {   
            Log.e("Server ", "client.isConnected() "+client.isConnected());
            server.close();
        }

        out = new ObjectOutputStream(client.getOutputStream());
        out.writeObject(canDataInfo);

        out.close();
    }
    catch (Exception ex)
    {
        Log.e(CANManagerSetUp.TAG, "" + ex);
    }
}

Client Side Code {Not a clean solution, Refer Answer from EJP}

   package com.cnh.socket.client;

import java.io.BufferedWriter;
import java.io.EOFException;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.Socket;

import javax.swing.JLabel;

import cantest.setup.CANDataInfo;


public class ThreadListener
{   
    Socket client = null;
    ObjectInputStream in = null;
    ListenFor0X28 runnableListenFor0X28 = null;
    boolean continueMe;


    public class ListenFor0X28 implements Runnable 
    {   
        JLabel jLab0x28; 

        public ListenFor0X28(){}

        public ListenFor0X28(boolean stop, JLabel jLab0x28) 
        {
            continueMe = stop;
            this.jLab0x28 = jLab0x28;
        }

        public void run() 
        {   

            while(continueMe)
            {
                try
                {
                    client = new Socket("localhost", 38301);
                    in = new ObjectInputStream(client.getInputStream());
                    if(client.isConnected())
                    {   
                        jLab0x28.setText("Connected to Server");
                        appendFile(continueMe, jLab0x28, client);

                    }
                    else
                    {   
                        System.out.println("Client is trying to connect");
                        jLab0x28.setText("Client is trying to connect");
                    }
                }
                catch(Exception ex)
                {   
                    ex.printStackTrace();
                    System.err.println("Before Append "+ex.toString());
                }
            }
        }
    }
    BufferedWriter file = getFile("C:\\ISSUE124_Resolved.txt");
    private void appendFile(boolean continueMe, JLabel jLab0x28, Socket client)
    {   
        try
        {   
            if(!client.isClosed())
            {   
                try
                {   
                    CANDataInfo canData = (CANDataInfo) in.readObject();
                    System.out.println(canData.toString());
                    file.write(canData.toString());
                    file.flush();

                }
                catch (EOFException exp)
                {   
                    continueMe = true;
                    System.out.println("A Stream has finished "+exp.toString()+"\n");
                }
                catch (ClassNotFoundException exp) 
                {
                    exp.printStackTrace();
                    System.err.println(exp.toString());
                    continueMe = false;
                }
            }

            if(!continueMe)
            {
                file.close();
                client.close();
                in.close();
                jLab0x28.setText("Socket is closed "+client.isClosed());
            }

        }
        catch(IOException exp)
        {
            exp.printStackTrace();
            System.err.println("Exception "+exp.toString());
            jLab0x28.setText(exp.getMessage());
            continueMe = false;
        }
    }

    public BufferedWriter getFile(String path) 
    {
        try 
        {
            File file = new File(path);
            if (!file.exists()) 
            {
                file.createNewFile();
            }
            FileWriter fw = new FileWriter(file.getAbsoluteFile());
            return new BufferedWriter(fw);
        }
        catch (IOException e) 
        {
            e.printStackTrace();
        }
        return null;
    }
}

Exception Stack: {Before Resolving}

java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
    at java.io.ObjectInputStream.readObject0(Unknown Source)
    at java.io.ObjectInputStream.readObject(Unknown Source)
    at com.cnh.socket.client.ThreadListener.appendFile(ThreadListener.java:73)
    at com.cnh.socket.client.ThreadListener.access$0(ThreadListener.java:65)
    at com.cnh.socket.client.ThreadListener$ListenFor0X28.run(ThreadListener.java:48)
    at java.lang.Thread.run(Unknown Source)
Data received in unknown format java.io.EOFException

Answer:

In the client

if (!(in.read() == -1))
{
    CANDataInfo canData = (CANDataInfo) in.readObject();
    System.out.println(canData.toString());
    bw.write(canData.toString());
}

The first line reads one byte from the input stream. This is actually the first byte of the object that was written by the server. Thus the stream is no longer aligned correctly so the following readObject() fails.

Question:

I'm writing a simple Golang TCP Client that talks with a Java TCP Server.

I have successfully gotten my Golang Client to send a message to my Java Server.

However, my Java code is expecting an End of Stream (where inputStream.read() returns -1) to know that it's time to stop reading Client messages.

It looks like Golang does not send an End of Stream message via connection.Write() unless I Close() the connection first.

Below is my Java Server code:

package com.mycompany.app;

import com.google.gson.*;
import com.google.common.primitives.Bytes;
import java.io.*;
import java.net.Socket;
import java.util.ArrayList;
import java.util.List;
import java.net.ServerSocket;
import java.net.SocketException;

public class MyApp {
    public static void main(String[] args) {
        int port = 9000;

        // Start listening for messages.
        try (ServerSocket serverSocket = new ServerSocket(port)) {
            System.out.println("Listening on port " + port + "...");

            // Never stop listening for messages.
            while (true) {
                try {
                    // Accept a client connection.
                    Socket socket = serverSocket.accept();

                    // Read message from the client.
                    DataInputStream inputStream = new DataInputStream(socket.getInputStream());

                    List<Byte> list = new ArrayList<Byte>();
                    int input = inputStream.read();

                    while (input != -1) {
                        list.add((byte) input);
                        input = inputStream.read();
                    }

                    byte[] jsonBytes = Bytes.toArray(list);

                    if (jsonBytes.length > 0) {
                        String jsonString = new String(jsonBytes);

                        System.out.println("Received: " + jsonString);

                        // Unmarshal from JSON.
                        Person person = new Gson().fromJson(jsonString, Person.class);

                        System.out.println("Person: " + person.Name + " " + person.Age);
                    }
                } catch (EOFException e) {
                    // A client has disconnected. Do nothing.
                } catch (SocketException e) {
                    // A client connection has been terminated unexpectedly. Do nothing.
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

class Person {
    String Name;
    int Age;
}

Below is my Golang Client code:

package main

import (
    "bufio"
    "encoding/json"
    "fmt"
    "net"
    "os"
    "strings"
)

type Person struct {
    Name string
    Age  int
}

func main() {
    reader := bufio.NewReader(os.Stdin)

    for {
        func() {
            // Dial the Java Server.
            conn, err := net.Dial("tcp", "localhost:9000")
            if err != nil {
                fmt.Println(err)
            }

            defer conn.Close()

            fmt.Print("Enter Name: ")
            strName, err := reader.ReadString('\n')
            strName = strings.TrimSpace(strName)

            var person Person
            person.Name = strName
            person.Age = 18

            jsonBytes, err := json.Marshal(person)
            if err != nil {
                fmt.Println(err)
                return
            }

            // Write our message to the connection.
            _, err = conn.Write(jsonBytes)
            if err != nil {
                fmt.Println(err)
                return
            }

            fmt.Println("Sent: " + string(jsonBytes))
        }()
    }
}

Any advice on how to tell Java that we're done writing messages from Golang? Or is there a better way to handle Java x Golang TCP Client-Server connections?

This is quite a problem because one of the basic scenarios for TCP connections is Send-and-Receive:

e.g.

  1. Client sends a message and waits for the result.

  2. Server processes the message and returns the result.

  3. Client does something with the result.


Answer:

For starters, you probably want to make sure your client code reliably closes the socket. But that's not all you should do.

The client process should trigger a TCP FIN as a result of closing the socket. And the socket should get closed as a result of the client process cleanly exiting. Not sure why this isn't the case for you. Are you waiting long enough?

However, so that your server is not indefinitely hung, you need to guard for this condition. Consider the case where there's an active TCP connection between a client and your server. And while the server is awaiting data from the client, the power goes out where the client is at. Because there's no way for the client to send a FIN, the server connection isn't going to have any traffic. That TCP connection will stay hung forever unless....

You either set a keep-alive option or a timeout option on the server side.

Keep-alive: https://docs.oracle.com/javase/7/docs/api/java/net/SocketOptions.html#SO_KEEPALIVE

Timeout: https://docs.oracle.com/javase/7/docs/api/java/net/SocketOptions.html#SO_TIMEOUT

Question:

I am not sure what the best solution for following situation is:

My Java-program is permanently reading from a tcp-stream. At the same time it is neccessary to persist these data to a database. The amount of data which should be written to the database could differ

I have read a lot of about message-queueing-systems and so on. In detail, my solution would consider using a LinkedBlockingQueue. Thereby, there are two threads: a) Starting a producer-threat which will perform reading from the tcp-stream b) Starting a consumer-threat which will write the (parsed) data from the stream to the database

The (example-)code looks like following:

Main.java

public static void main(String[] args) {
        LinkedBlockingQueue queue = new LinkedBlockingQueue(50);
        Producer producer = new Producer(queue);
        Consumer consumer = new Consumer(queue, producer);

Produer.java

public class Producer implements Runnable {
  private LinkedBlockingQueue queue;
  private boolean running;

  public Producer(LinkedBlockingQueue queue) {
        this.queue = queue;
        running = true;
    }

 @Override
  public void run() {
    //read TCP-Stream here and save parsed messages to queue
  }
  public boolean isRunning() {
        return running;
  }

Consumer.java

public class Consumer implements Runnable {
    private Producer producer;
    private LinkedBlockingQueue queue;

    public Consumer(LinkedBlockingQueue queue, Producer producer) {
        this.queue = queue;
        this.producer = producer;
    }
   @Override
   public void run() {
       //insert data into database here
       if(producer.isRunning()) {
          //while producer is running, data needs to be inserted to database
       }
   }

Is this a solution you would recommend to use? Or do you know better solutions?

Thank you!


Answer:

Your own suggestion is very good.

Ultimately what you are trying to solve is the problem of back pressure, i.e. if you are receiving data faster than you can write them to database. This may happen just because there's a lot of data arriving or simply because your destination is temporarily unavailable. Either way it is a situation you need to deal with.

In your proposed solution this is handled by an in-memory staging area (= your queue). As long as you have enough memory and you are not too concerned about loosing data on power outages then the in-memory strategy will work fine for you. A burst will be absorbed by growing memory in your Java application. Not a problem per se but remember that when your queue is finally drained the JVM GC will kick in and release the memory again from the JVM heap. But from the outside, i.e. from the point of view of the OS, the memory will possibly never be released. The JVM is very, very conservative about releasing memory back to the OS. Again, in most cases this is not a problem.

If you have more strict needs then you need to consider a more "robust" staging area instead of RAM, for example local disk. From my experience your proposed solution fits 95% of use cases.

Question:

Socket's methods do not appear to function in the way their names or documentation suggest. For example. I create a client socket to connect a remote serversocket. When the connection establishes, the serversocket.accept() method returns a corresponding socket which to getinputstream from the client socket. But the problem is, if I close the client socket, the socket on the server still returns false for the isClosed() method; and, more absurdly, the Socket's InputStream on the server starts to continuously return value and no longer blocks when the client socket has closed and sending no output to the server. Below is my code:

Client code:

public class MainActivity extends AppCompatActivity {
private Button startButton, stopButton;
public byte[] buffer;
public Socket socket;
public Socket tempSocket;
private int port = 50005;
InetSocketAddress address;
private int r_port = 50006;
AudioRecord recorder;
AudioTrack audioTrack;
private int sampleRate = 16000; // 44100 for music
private int channelConfig = AudioFormat.CHANNEL_IN_MONO;
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;
int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
public static boolean s_status = true;
public static boolean r_status = true;
Thread r_Thread;
Thread s_Thread;
private boolean isPlay = true;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    startButton = (Button) findViewById(R.id.start_button);
    startButton.setOnTouchListener(talkListener);
    if (socket == null) {
        socket = new Socket();
        address = new InetSocketAddress("192.168.0.2", port);
        try {
            socket.setReuseAddress(true);
            System.out.println("connecting-");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

private final View.OnTouchListener talkListener = new View.OnTouchListener() {
    @Override
    public boolean onTouch(View v, MotionEvent event) {
        switch (event.getAction()) {

            case MotionEvent.ACTION_DOWN:
                startButton.setBackgroundColor(Color.parseColor("#0670c0"));
                try {

                    startStreamings();
                } catch (Exception e) {
                    e.printStackTrace();
                }
                return true; // if you want to handle the touch event

            case MotionEvent.ACTION_UP:
                startButton.setBackgroundColor(Color.parseColor("#353535"));

                try {
                    s_status = false;

                } catch (Exception e) {
                    e.printStackTrace();

                    return true; // if you want to handle the touch event
                }
        }
        return false;
    }
};
public void startStreamings(){
    s_status=true;
    buffer = new byte[minBufSize];
    s_Thread = new Thread(new Runnable() {

        @Override
        public void run() {

            if (socket == null) {
                try {
                    socket = new Socket();
                    socket.setReuseAddress(true);

                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (!socket.isConnected()) {
                try {
                    socket.connect(address);
                    System.out.println("create new connection in startStreaming");
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleRate, channelConfig, audioFormat, minBufSize * 10);
            try {
                if (s_status == true) {
                    recorder.startRecording();
                }
            } catch (IllegalStateException e) {
                e.printStackTrace();
                return;
            }

            OutputStream os = null;
            while (s_status == true) {

                //reading data from MIC into buffer
                recorder.read(buffer, 0, buffer.length);
                try {
                    os = socket.getOutputStream();
                    os.write(buffer, 0, minBufSize);
                } catch (Exception e) {
                    e.printStackTrace();
                    s_status = false;
                }
                //System.out.println("streaming out: " + buffer.length + "fff" + c++);
            }

            if (recorder != null) {
                recorder.stop();
                recorder.release();
                recorder = null;
            }
        }
    });
    s_Thread.start();
}

}

Server code:

    public TcpServerSocket(){

}

public static void main(String args[]) throws Exception {
 ServerSocket serverSocket = new ServerSocket(port);
 sockets = new ArrayList<Socket>();   
    while(isListenning){

        Socket socket = serverSocket.accept();
        isMatched = false;
        for(int i =0;i<sockets.size();i++){
            Socket preSocket = sockets.get(i);
            if(preSocket.getInetAddress().equals(socket.getInetAddress())){
                sockets.remove(preSocket);
                sockets.add(socket);
                isMatched = true;
            }
        }

        if(!isMatched){
            sockets.add(socket);
            socket.setKeepAlive(false);
            new Thread(new TcpServerSocket(socket)).start();
            System.out.println("new Connection");
        }


        }
    serverSocket.close();



 }

@Override
public void run() {


       byte[] receiveData = new byte[1280];
       byte[] emptyData = new byte[1280];
        InputStream baiss = null;

        OutputStream os;
        while (isRunning){
            try {
                baiss = csocket.getInputStream();

                if(csocket.isClosed()||!csocket.isConnected()){
                    isRunning = false;
                    sockets.remove(csocket);
                    System.out.println("socket closed!");
                }


                int numOfBytes = baiss.read(receiveData);
                if(numOfBytes==-1){
                    isRunning=false;
                    sockets.remove(csocket);
                    csocket.close();
                    System.out.println("socket closed!");
                }

            } catch (IOException e) {
                sockets.remove(csocket);
                System.out.println("socket closed!");
                e.printStackTrace();
            }
            int socketsLen = sockets.size();
            for(int i = 0;i<socketsLen;i++){
                Socket client = sockets.get(i);

                if(!client.getInetAddress().equals(csocket.getInetAddress())){
                    try {
                    os = client.getOutputStream();
                    os.write(receiveData,0,1280);
                    } catch (IOException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                    }

                }else if(!client.equals(csocket)){
                    csocket = client;
                    System.out.println("switched!");
                }
            }

        System.out.println(csocket.getInetAddress().toString()+"fff"+socketsLen);
        }
        try {
            baiss.close();
            csocket.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }


}

Can you guys give me any suggestions to close the client socket perfectly so I won't keep getting input after I close the client? Any help is much appreciated!


Answer:

socket.getInputStream() still get data after I closed the other side socket

That could be because there was still data in transit that hadn't been read yet. Closing a connection doesn't abort pending data transfers. But as it turns out, it doesn't get data at all. There is simply a bug in your code.

I am really upset with socket in recent days.

I suggest you get over being upset and adopt a rational attitude towards your chosen profession.

I find socket's methods just do not function in the way their names or documentations suggest.

Let's see.

For example. I create a client socket to connect a remote serversocket. When the connection establishes,the serversocket.accept() method returns a corresponding socket which to getinputstream from the client socket. But the problem is, if I close the client socket, the socket on the server still returns true for the isClosed() method

No it doesn't. It returns false. The server's socket is still open. The client's socket is closed, and so is the connection, but isClosed() tells you about the state of the socket it is called on, not anything else, and specifically not the connection.

and more absurd, the socket.getInputStream() on the server starts to continuously return value and no longer blocks when the client socket has closed and sending no outputstream to the server.

Only if there was data in flight before the peer closed. Otherwise it is due to a bug in your code, and here it is:

//reading data from MIC into buffer
recorder.read(buffer, 0, buffer.length);

read() returns -1 at end of stream, and you are ignoring it. That's why you get a continous loop. The correct way to write this code is as follows:

int count = recorder.read(buffer, 0, buffer.length);
if (count == -1) {
    recorder.close();
    socket.close();
    break;
}
try {
    os = socket.getOutputStream();
    os.write(buffer, 0, count);

You have a similar problem with your client code. You just don't seem to care about end of stream:

baiss = csocket.getInputStream();
if(csocket.isClosed()||!csocket.isConnected()){
    isRunning = false;
    sockets.remove(csocket);
    System.out.println("socket closed!");
}
baiss.read(receiveData);
// ...
os = client.getOutputStream();
os.write(receiveData,0,1280);

The correct way to write this is as follows:

baiss = csocket.getInputStream();
int count = baiss.read(receiveData);
if(count == -1){
    isRunning = false;
    sockets.remove(csocket);
    System.out.println("socket closed!");
}
// ...
os = client.getOutputStream();
os.write(receiveData, 0, count);

Can you guys give me any suggestions to close the client socket perfectly so I won't keep getting input after I close the client?

You are closing it perfectly. The problem is that you aren't detecting it correctly at the other end.

Question:

I wrote a Java socket server which will keep connection alive until client disconnected. And my client code will keep pushing message to this server app.

But when I run those programs a while, I also seem an unusual condition that Server will hangs while reading input stream from client within unpredictable period. It always hang at inData.read(b) because I see it printed "receiving..." on log when this problem occurred"; even I killed my client, server app still hangs right there.

But when I press Ctrl+C at the console which runs server app after this problem occurred, it will continue to work. This is really annoying.

Is there anyway to solve this Unusual problem nicely?

Server Code:

    static ServerSocket server;

    try {
        server = new ServerSocket("1234");
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    Socket socket = null;
    String inIp = null;
    BufferedInputStream inData;
    BufferedOutputStream outData;

    while (true) {
        try {
            synchronized (server) {
                socket = server.accept();
            }
            inIp = String.valueOf(socket.getInetAddress());
            if (Log4j.log.isEnabledFor(Level.INFO)) {
                Log4j.log.info("Incoming connection " + inIp);
            }
            while (true) {
                inData = new BufferedInputStream(socket.getInputStream());
                outData = new BufferedOutputStream(socket.getOutputStream());
                String reply = "Hey";

                byte[] b = new byte[10240];
                String data = "";
                int length;

                if (Log4j.log.isEnabledFor(Level.INFO)) {
                    Log4j.log.info("InetAddr = " + inIp + ", receiving...");
                }


                // read input stream
                length = inData.read(b);
                data += new String(b, 0, length);
                if (Log4j.log.isEnabledFor(Level.INFO)) {
                    Log4j.log.info("Data Length: " + length + ", Received data:  " + data);
                }


                // output result
                outData.write(reply.getBytes());
                outData.flush();
            }
        } catch (Exception e) {
            String tempStr = e.toString();
            Log4j.log.error("Service error during executing: " + tempStr);
        }
    }

Client Code:

    Socket client = new Socket();
    InetSocketAddress isa = new InetSocketAddress("127.0.0.1", "1234");
    String data = "Hi";

    while(true) {
        try {
            if(!client.isConnected())
                client.connect(isa, 30000);

            BufferedOutputStream out = new BufferedOutputStream(client.getOutputStream());
            BufferedInputStream in = new BufferedInputStream(client.getInputStream());

            // send msg
            out.write(data.getBytes());
            out.flush();


            System.out.println("Message sent, receiving return message...");


            // get return msg
            int length;
            byte[] b = new byte[10240];

            // read input stream
            length = in.read(b);
            retMsg = new String(b, 0, length);

            System.out.println("Return Msg: " + retMsg);

            Thread.sleep(60000); 

        } catch (java.io.IOException | InterruptedException e) {
            System.out.println("Socket Error!");
            System.out.println("IOException :" + e.toString());
        }
    }

Answer:

try {
    server = new ServerSocket("1234");
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

Don't write code like this. The catch block should be at the end, and all the code that depends on the success of new ServerSocket should be inside the try block.

synchronized (server) {
    socket = server.accept();
}

Synchronization is not necessary here.

while (true) {
    inData = new BufferedInputStream(socket.getInputStream());
    outData = new BufferedOutputStream(socket.getOutputStream());

A large part of the problem, if not all of it, is here. You keep creating new buffered streams, every time around this loop, which means that anything the previous streams have buffered is thrown away. So you are losing input. You should create both these streams before the loop.

while(true) {
    try {
        if(!client.isConnected())
            client.connect(isa, 30000);

This is pointless. Remove. You haven't shown how the client socket was created, but if you created it unconnected you should have connected it before entering this loop.

        BufferedOutputStream out = new BufferedOutputStream(client.getOutputStream());
        BufferedInputStream in = new BufferedInputStream(client.getInputStream());

Here again you must create these streams ahead of the loop.

Question:

I have a TCP Client in Java which communicates with a C# TCP Server and vice versa. They communicate by sending over byte arrays. I'm having problems with reading the byte arrays in the Client. The byte arrays have a fixed length of 4.

When for example the server sends:

[2, 4, 0, 2]

[2, 4, 0, 0]

The client output is:

Connecting to port :10000

Just connected to /192.168.1.101:10000

Server says [2, 4, 0, 0]

How can I solve this? It seems like the first array gets overwritten?

TCPClient.java

public class TCPClient {

private OutputStream outToServer=null;
private DataOutputStream out=null;
private ByteProtocol byteProtocol;
Socket client;
InputStream inFromServer;
DataInputStream in;

public void initConnection(){
     try {
        int serverPort = 10000;
        InetAddress host = InetAddress.getByName("192.168.1.101");
        System.out.println("Connecting to port :" + serverPort);
        client = new Socket(host, serverPort);
        System.out.println("Just connected to " + client.getRemoteSocketAddress());
        outToServer = client.getOutputStream();
        out=new DataOutputStream(outToServer);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

public void readBytes(){

    try {
        inFromServer = client.getInputStream();
        in = new DataInputStream(inFromServer);

        byte[] buffer = new byte[4];
        int read = 0;
        while ((read = in.read(buffer, 0, buffer.length)) != -1) {
            in.read(buffer);
            System.out.println("Server says " + Arrays.toString(buffer));
        }
    } catch (IOException ex) {
        ex.printStackTrace();
    }
}

public void sendBytes(byte [] byteArray) throws IOException{
     out.write(byteArray);
}

public void closeClient() throws IOException{
    client.close();
 }
}

Answer:

It looks like you are reading into the buffer twice:

    while ((read = in.read(buffer, 0, buffer.length)) != -1) {
        in.read(buffer);

So the contents read in the loop head are overwritten by the second read().

Question:

I'm trying to use BufferedReader.read(char[]) method instead of the easier, but less versatile BufferedReader.readLine() method for receiving an answer from a Server. BufferedReader is used in parallel with BufferedOutputStream and, in the code below, the read(char[]) method blocks everything, the last console output is "new buffer, waiting to read."

Client:

public class MessageSender extends Thread {
private String message;
MessageSender(String message) {
    this.message = message;
}

public void run() {
    try {
        Socket sk = new Socket("192.168.1.4", 3000);
        BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
        bo.write(message.getBytes());
        bo.flush();

        char[] c = new char[100];

        BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
        StringBuffer sb = new StringBuffer();
        System.out.println("new buffer, waiting to read.");
        int ix = 0;

        while (ix != -1) {
            ix = br.read(c);
            sb.append(new String(c));
        }

        String message = sb.toString();

        System.out.println("reply: " + message);

        sk.close();
    } catch (IOException e) {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

}

Server:

public class MessageReceiver extends Thread {
    public void run() {
        try {
            ServerSocket ss = new ServerSocket(3000);
            System.out.println("server socket open");
            while (true) {
                Socket sk = ss.accept();
                System.out.println("new connection");
                BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
                String line = br.readLine();
                System.out.println("received line: " + line);
                BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
                bo.write("ack".getBytes()); bo.flush(); //bo.close();
                sk.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Main:

public class Main {

    public static void main(String[] args) {
        MessageReceiver mr = new MessageReceiver();
        mr.start();
        while (true) {
            String msg = new Scanner(System.in).nextLine();
            MessageSender ms = new MessageSender(msg+"");
            ms.start();
        }
    }
}

Everything works fine as long as the BufferedReader.read is not called. But as the code is right now, the output doesn't seem to get sent to the server.

UPDATE

As answered by @JB Nizet, the problem lies in the server script that uses readLine() and waits for either EOL character or the connection end. Therefore, adding "\n" to the message sent from the client side solved the deadlock:

bo.write((message+"\n").getBytes());

Answer:

When the server accepts a connection from the client, the first thing it does is:

String line = br.readLine();

So, it blocks until the client sends a complete line of text. The server only knows the line is complete if it reads an EOL character, or if the stream is closed by the client.

When the client starts, the first thing it does is

bo.write(message.getBytes());

And message is a line of text, without any EOL. Then the client does

ix = br.read(c);

so it waits for a response from the server, which is itself waiting for an EOL from the client.

You have implemented a networked deadlock.

Question:

I want to use the Spring Cloud Stream App Starter TCP Source project (maven artifact) in order to be able to receive TCP message through socket/port, process them and then push the result to a message broker (e.g. RabbitMQ).

This TCP source project seems to do exactly what I want, but it automatically sends the received message to the output channel. So, Is there a clean way to still use the TCP source project but intercept the TCP incoming message to transform them internally before to output them to my message broker?


Answer:

See aggregation.

You create an aggregated app using the source and processor.

Spring Cloud Stream provides support for aggregating multiple applications together, connecting their input and output channels directly and avoiding the additional cost of exchanging messages via a broker. As of version 1.0 of Spring Cloud Stream, aggregation is supported only for the following types of applications:

sources, sinks, processors ...

They can be aggregated together by creating a sequence of interconnected applications, in which the output channel of an element in the sequence is connected to the input channel of the next element, if it exists. A sequence can start with either a source or a processor, it can contain an arbitrary number of processors and must end with either a processor or a sink.

EDIT

As a work around to the Source autowiring problem, you could try something like...

@EnableBinding(Source.class)
@EnableConfigurationProperties(TcpSourceProperties.class)
public class MyTcpSourceConfiguration {

    @Autowired
    private Source channels;

    @Autowired
    private TcpSourceProperties properties;

    @Bean
    public TcpReceivingChannelAdapter adapter(
            @Qualifier("tcpSourceConnectionFactory") AbstractConnectionFactory connectionFactory) {
        TcpReceivingChannelAdapter adapter = new TcpReceivingChannelAdapter();
        adapter.setConnectionFactory(connectionFactory);
        adapter.setOutputChannelName("toMyProcessor");
        return adapter;
    }

    @ServiceActivator(inputChannel = "toMyProcessor", outputChannel = Source.OUTPUT)
    public byte[] myProcessor(byte[] fromTcp) {
        ...
    }

    @Bean
    public TcpConnectionFactoryFactoryBean tcpSourceConnectionFactory(
            @Qualifier("tcpSourceDecoder") AbstractByteArraySerializer decoder) throws Exception {
        TcpConnectionFactoryFactoryBean factoryBean = new TcpConnectionFactoryFactoryBean();
        factoryBean.setType("server");
        factoryBean.setPort(this.properties.getPort());
        factoryBean.setUsingNio(this.properties.isNio());
        factoryBean.setUsingDirectBuffers(this.properties.isUseDirectBuffers());
        factoryBean.setLookupHost(this.properties.isReverseLookup());
        factoryBean.setDeserializer(decoder);
        factoryBean.setSoTimeout(this.properties.getSocketTimeout());
        return factoryBean;
    }

    @Bean
    public EncoderDecoderFactoryBean tcpSourceDecoder() {
        EncoderDecoderFactoryBean factoryBean = new EncoderDecoderFactoryBean(this.properties.getDecoder());
        factoryBean.setMaxMessageSize(this.properties.getBufferSize());
        return factoryBean;
    }

}

Question:

I'm working on a tcp base communication protocol . As i know there are many ways to determine when to end reading.

  1. Closing the connection at the end of the message
  2. Putting the length of the message before the data itself
  3. Using a separator; some value which will never occur in the normal data (or would always be escaped somehow)

Typically i'm trying to send a file over the WiFi network (that may be Unstable and Low speed)

  • Cause of RSA and AES communication I don't like to close the connection each time (Can't use 1)
  • It's a large file that i cant predict the length of it so i cant act as method (Can't use 2)
  • Checking for something special when reading and escape it when writing need a lot of process (Can't use 3)
  • This method should be compatible with both c# and java.

What you suggest ?


More general problems :

How to identify end of InputStream in java

C# - TcpClient - Detecting end of stream?

More Iformation

I'm coding a TCP client server communication

At first server generates and sends a RSA public code to the client.

Then the client will generate AES(key,IV) and send it back using RSA encryption.

Till here everything is fine.

But i want to send a file over this network. here is my current packet EncryptUsingAES(new AES.IV(16 byte) +file.content(any size))

In the server i can't capture all the data sent by client. So i need to know how much data to read with (TcpClient.GetStream().read(buffer , 0 , buffersize) ) Current code:

List<byte> message = new List<byte>();
    int bytes = -1;
    do
    {
        byte[] buffer = new byte[bufferrSize];
        bytes = stream.Read(buffer, 0, bufferrSize);
        if (bytes > 0)
        {
            byte[] tmp = new byte[bytes];
            Array.Copy(buffer, tmp, bytes);
            message.AddRange(tmp);
        }
    } while (bytes == bufferrSize);

Answer:

Your second method is the best one. Prefixing each packet with the packet's length will create a reliable message framing protocol which will, if done correctly, ensure that all your data is received even in the same size you sent it (that is, no partial data or data being lumped together).

  • Recommended packet structure:

    [Data length (4 bytes)][Header (1 byte)][Data (?? bytes)]
    

    - The header in question is a single byte you can use to indicate what kind of packet this is, so that the endpoint will know what to do with it.

Sending files

The sender of a file is in 90% of the cases aware of the amount of data it is about to send (after all, it usually has the file stored locally), which means there will be no problem knowing how much of the file has been sent or not.

The method I use and recommend is that you start by sending an "info packet", which explains to the endpoint that it is about to receive a file and also how many bytes that file consists of. After that you start sending the actual data - most preferrably in chunks since it's inefficient to proccess the entire file at once (at least if it's a large file).

  • Always keep track of how many bytes of the file you've received so far. By doing so the receiver can automatically tell when it has received the whole file.
  • Send a file a few kilobytes at a time (I use 8192 bytes = 8 kB as a file buffer). That way you don't have to read the entire file into memory nor encrypt it all at the same time.

Encrypting the data

Dealing with encryption will not be a problem. If you use length-prefixing just encrypt the data itself and leave the data length header untouched. The data length header must then be generated by the size of the encrypted data, like so:

  1. Encrypt the data.
  2. Get the length of the encrypted data.
  3. Produce the following packet:

    [Encrypted data length][Encrypted data]
    

    (Insert a header byte in there if you need to)

Receiving an encrypted file

Receiving an encrypted file and knowing when everything has been received is infact not very hard. Assuming you're using the above the described method for sending the file, you would just have to:

  1. Receive the encrypted packet → decrypt it.
  2. Get the length of the decrypted data.
  3. Increment a variable keeping track of the amount of file-bytes received.
  4. If the received amount is equal to the expected amount: close the file.

Additional resources/references

You can refer to two of my previous answers that I wrote about TCP length-prefixed message framing:

  • C# Deserializing a struct after receiving it through TCP
  • TCP Client to Server communication

Question:

I've been using org.apache.commons.net.telnet to connect to a control stream and send commands to the camera. I send a datastream request and open a new thread which scans an image from the camera's sensor and sends the data as raw bytes to me. I'm using standard java.io instream and outstream to read. The outstream I'm writing to a file.... just the raw bytes. However, I'm getting stuck in an infinite loop reading the data the socket sends. A instream.read() > -1 keeps me there... I've done instream.available() > 0, but this often cuts the image short (understandably so). I've even tried various and/ors and can never get a complete read.

I've confirmed in Wireshark that everything is passing through to my program and that a FIN is sent, but for some reason, JAVA is not picking up the FIN and giving me the -1. The oustream to the file remains open, and I never get a "complete" image scan from the sensor. (I've been manually killing the stream and reading in the image. The end goal is to toss this image into a label and use it for on the fly occasional camera exposure updates.)

Is there some way to detect the FIN message and tell the loop to kill outside of instream.read()?

Other info: Win 7 (enterprise build), Netbeans IDE


Answer:

A instream.read() > -1 keeps me there

Of course it does. It throws away a byte and it's not a valid test in the first place. Your read loop should look like this:

int ch;
while ((ch = instream.read()) != -1)
{
    // ... Cast ch to a byte and use it somehow ...
}

or this:

int count;
byte[] buffer = new byte[8192];
while ((count = instream.read(buffer)) > 0)
{
    // ...
    // for example:
    out.write(buffer, 0, count);
}

Question:

As the title says, could DataInputStream.read() override previously read bytes if there are more bytes available than the size of buffer, if in the first read some bytes were missed ?

There are fixed size packets exchanged between peers, and it is possible that two packets are available at the Socket. Let's say that size of one packet is 500, and there are two packets of total size 1000 available at the socket. Additionally, let's say that read fetches 400 bytes out of available 1000.

  • Is it even possible that read() doesn't read all 500 bytes if they are available ?

  • What will happen when read is invoked again, is it possible that more than 100 bytes are read ?

It is not really clear to me what happens in that case given the javadoc:

The first byte read is stored into element b[0], the next one into b[1], and so on. The number of bytes read is, at most, equal to the length of b.

I would like to know if below block of code should be modified as shown in comment to read only one packet fully.

    while ((readBytes = stream.read(buffer)) != -1) {

        totalBytes += readBytes;

        if (totalBytes < buffer.length) { // must this be != instead of < ?

            continue;
        }

        // all bytes are available
        else {

            break;
        }

Answer:

Each time you call read(byte[]), it will:

  • Block until at least one byte is read from the input, or the input is closed
  • Copy bytes from the input into the array, starting at index 0 of the array
  • Return when there are no more bytes available (typically, anyway - it could have some delay where it waits a while for more to become available)

There's no memory of previous read calls - it won't start writing into the array at the previous index. If you want that behaviour, you need to write it yourself:

byte[] buffer = new byte[500];
int totalRead = 0;
while (totalRead < buffer.length) {
    // Pass in an offset and length, so we can keep reading into the
    // next part of the array.
    int bytesRead = input.read(buffer, totalRead, buffer.length - totalRead);
    if (bytesRead == -1) {
        throw new EOFException(); // Or whatever... stream ended early
    }
    totalRead += bytesRead;
}

... or call readFully() which will basically do the same thing.

Question:

I am sending data via

data.writeInt()

where Data is a DataOutput, which has a socket's output stream.

On the other end I have a socket's InputStream and a DataInput and I read by

data.readInt()

I have confirmed that each write is matched by the appropiate read. Then I used wireshark to look at the packages, everyting is there as it should.

However, I get 1114112, when I send a 17. This is shifted by 2 bytes, so is it possible I loose two bytes somewhere?

I did some experiments with adding flush(), but the problem stayed as it was. Also I have not (yet) managed to write a smaller example which shows the problem (which is why I do not include code here). However, the large example is stable, it happens at the same location every time.

With different data, I get 285212672 right after the 17 (should be a 0). This looks like the last byte of the 17 is read twice.

Any ideas what this might be or suggestions how to debug this?

Alex


Answer:

I have confirmed that each write is matched by the appropriate read.

Clearly that isn't the case. Recheck. Clearly you are out of sync with the sender. Possibly you are for example using read() where you should be using readFully().

NB TCP doesn't lose data.

This is shifted by 2 bytes, so is it possible I loose two bytes somewhere?

It is certain: 'somewhere' being in your code.

I did some experiments with adding flush(), but the problem stayed as it was.

Only to be expected. There's no reason why flush() should make any difference.

Also I have not (yet) managed to write a smaller example which shows the problem (which is why I do not include code here). However, the large example is stable, it happens at the same location every time.

Again this is only to be expected. You have an application protocol error.

Question:

Using Android, I am reading data (a signed byte array) from a microcontroller. I understand that BufferedReader has a buffer size of 8192 bytes. I'm sending about 200-300 bytes every second, this means at the moment I have to wait about 25-30 seconds (8192/300) before the debugger will read the byte array.

I want to be reading this every time the Android receives the 200-300 byte array (every full TCP packet) is there a way to alter the maximum byte size of the BufferedReader, or is there an alternative way of doing this?

private BufferedReader input;
  input = new BufferedReader(new InputStreamReader(
      this.clientSocket.getInputStream()));
  ...
  ...
  while(true)
  {
    try
    {
      Log.i("WaitingForSomeData","");
      String read = input.readLine();
      byte[] bytes = read.getBytes();
    }

Answer:

I am reading data (a signed byte array) from a microcontroller

You should be using a BufferedInputStream if you want to buffer binary data.

You are currently attempting to read a character stream. Your readLine() method will block until either \r, \n or \r\n is received. I'm guessing this causes your delay, since you are waiting for an 0x0A or 0x0D byte to conveniently arrive.

Question:

In client socket, I wrote a thread to read the socket's inputStream continuously. Here I have used a while loop to read infinitely. However it takes more CPU; hence is it possible to reduce the CPU. Please add your suggestions. Also is it possible to add listeners for inputStream.

Thread code:

public void run() {
while (!shutdown) {
    try {
        if(socketClient != null) {
            String message = socketClient.getMessage();
            logger.info ("Message size:" + message.length ());
            if(!message.equals("EmptyString")) {
                process(message);
            }
        } 
    } catch (Exception exception) {
        logger.info("Unable to read the socket message" +exception);
    }
}

}

SocketClient.java

public class SocketClient{
private volatile boolean isConnected;
private int              port;
private int              retryCount;
private long             startTime;
private String           hostName;
private DataInputStream  input;
private DataOutputStream output;
private Socket           socket;

public SocketClient(int port, String hostname) throws IOException {
    this.port     = port;
    this.hostName = hostname;
    establishConnection();
}


public void shutdown() {
    try {
        shutdown = true;
        input.close();
        output.close();
        socket.close();
    } catch (Exception e) {
        logger.debug("Exception in shutdown:" + e.getMessage());
    }
}

        public String getMessage() {
    BufferedReader reader = null;

    try {
        StringBuilder builder = new StringBuilder();
        reader = new BufferedReader(new 
                        InputStreamReader(tcpSocket.getInputStream()));

        do {
            builder.append(reader.readLine());
        } while((reader.ready()));

        if (builder.length() == 0)
            return "EmptyString";

        return builder.toString();
    } catch (IOException e) {
            return "EmptyString";
    } finally {
        try {
            if(reader != null)
                reader.close();
        } catch(IOException e) {
            logger.error("unable to close reader");
        }
    }
}

    private void establishConnection() {
        retryCount = 1;
        startTime  = System.currentTimeMillis();

        while (!shutdown) {
            try {
                if(!isConnected) {
                    socket = new Socket(hostName,port);
                    socket.setKeepAlive(true);
                    input       = new DataInputStream(socket.getInputStream());
                    output      = new DataOutputStream(socket.getOutputStream());
                    isConnected = true;
                    shutdown    = true;
                } 
            } catch (Exception exception) {
                isConnected = false;
                sleepFewSeconds();
                reconnectSocket();
            }
        }
    }

    private void reconnectSocket() {
        long endTime = startTime + 120000L;

        if(!(System.currentTimeMillis() < endTime)) {
            shutdown = true;
        }   
    }

    private void sleepFewSeconds() {
        try {
            TimeUnit.MILLISECONDS.sleep(20);
        } catch (InterruptedException interruptedException) {
            shutdown = true;
        }
    }

}

Answer:

I am going to critique the entire class here. The answer to your specific question will appear.

public class SocketClient{
private volatile boolean isConnected;

You don't need this. socket == null would do just as well.

private int              port;
private int              retryCount;
private long             startTime;
private String           hostName;
private DataInputStream  input;
private DataOutputStream output;
private Socket           socket;

public SocketClient(int port, String hostname) throws IOException {
    this.port     = port;
    this.hostName = hostname;
    establishConnection();
}


public void shutdown() {
    try {
        shutdown = true;
        input.close();
        output.close();
        socket.close();

You don't need all these closes, and you're doing them in the wrong order anyway. output.close() is sufficient and in any case it should certainly be first.

    } catch (Exception e) {
        logger.debug("Exception in shutdown:" + e.getMessage());
    }
}

public String getMessage() {
    BufferedReader reader = null;

The BufferedReader should be an instance variable, not a local variable. It's buffered. If you make it a local variable you will lose data.

    try {
        StringBuilder builder = new StringBuilder();
        reader = new BufferedReader(new 
                        InputStreamReader(tcpSocket.getInputStream()));

        do {
            builder.append(reader.readLine());
        } while((reader.ready()));

You don't need all this. If the message is a single line, all you need is return reader.readLine(), and you need the caller to check whether it was null, and if so close the socket, cease reading, etc. If the message is more than one line, this is a misuse of ready(): it is certainly not an indicator of end of message. It appears from comments under your question that you shouldn't even have the method: just connect the socket input stream directly to your XML parser and let it do the reading.

        if (builder.length() == 0)
            return "EmptyString";

Don't do this. Return "" or null. Don't make up new magic strings for your application to have to decode.

        return builder.toString();
    } catch (IOException e) {
            return "EmptyString";

Ditto.

    } finally {
        try {
            if(reader != null)
                reader.close();

You should not close the reader here. Closing it will close the socket, so you can never get another message.

        } catch(IOException e) {
            logger.error("unable to close reader");
        }
    }
}

private void establishConnection() {
        retryCount = 1;
        startTime  = System.currentTimeMillis();

        while (!shutdown) {
            try {
                if(!isConnected) {
                    socket = new Socket(hostName,port);
                    socket.setKeepAlive(true);
                    input       = new DataInputStream(socket.getInputStream());
                    output      = new DataOutputStream(socket.getOutputStream());
                    isConnected = true;
                    shutdown    = true;

Why are you setting shutdown to true here? Nothing is shutdown yet. It's a brand new socket.

                } 
            } catch (Exception exception) {
                isConnected = false;
                sleepFewSeconds();
                reconnectSocket();
            }

Poor practice. Socket.connect(), which is called internally by new Socket(...), already retries, and also you should distinguish between connection-failure exceptions rather than adopt the same strategy for them all. For example, a 'connection timeout' will already have blocked for a minute or so: you don't need another sleep; and 'connection refused' means there is nothing listening, so retrying is completely pointless.

    private void reconnectSocket() {
        long endTime = startTime + 120000L;

        if(!(System.currentTimeMillis() < endTime)) {
            shutdown = true;
        }   
    }

    private void sleepFewSeconds() {
        try {
            TimeUnit.MILLISECONDS.sleep(20);

This is not a 'few seconds'. It is 20 milliseconds, and that is not enough by at least two orders of magnite in network programming, to the extent that there should be any sleep at all of course.

        } catch (InterruptedException interruptedException) {
            shutdown = true;

shutdown appears to be never false. I doubt that you've thought through what it really means, and I doubt that you really need it at all.

As for your calling code:

public void run() {
while (!shutdown) {
    try {
        if(socketClient != null) {

If socketClient is null this loop will spin meaninglessly. Surely this method should construct the socket client?

            String message = socketClient.getMessage();
            logger.info ("Message size:" + message.length ());

Here you are failing to check for null and failing to respond appropriately, which would be to close the socket and exit the loop. Instead you will get an NPE here.

            if(!message.equals("EmptyString")) {
                process(message);

See above. Don't send yourself special text messages. What happens if the peer needs to send that one day?

            }
        } 
    } catch (Exception exception) {
        logger.info("Unable to read the socket message" +exception);

Unacceptable. This catch is inside the loop and it essentially ignores the exception. The result is that, again, this loop will spin meaninglessly on any exception. And the methods you're calling should be declared to throw IOException, and that is all you should catch here. At present you will spin even on NullPointerException.

Question:

I have the following code for upload file from client to server tcp but when i try to open manually the file is empty why the good weight.. I have look lot of post on stackOverflow but nothing make change Thx (Sorry for my bad english) Server:

public class ThreadServer extends Thread{

private Socket soc;
private FileOutputStream fos;
private BufferedOutputStream bos;
private InputStream in;

public ThreadServer (Socket soc) {
    this.soc = soc;
}

public void run(){
    try {
        fos = new FileOutputStream("C:/Users/erwan/workspace/Word/server/text.txt");
    } catch (FileNotFoundException e1) {
        e1.printStackTrace();
    }
    bos = new BufferedOutputStream(fos);
    byte[] buffer = new byte[1024];
    try {
        in = soc.getInputStream();
        int count = 0;
        while((count= in.read(buffer, 0 , buffer.length)) != -1) {
             System.out.println(count+" octets received...");                 
             bos.write(buffer);
        } 
        bos.flush();
        bos.close();
        in.close();
        soc.close();
        System.out.println("File sent succesfully!");
    }catch(IOException e){
        e.printStackTrace();
        System.out.println("Une erreur est survenu");
    }
}

}

client:

public class Client {
private static Socket as;
private static FileInputStream fis;
private static BufferedInputStream bis;
private static OutputStream out;
public static void main( String[] args ){
    as = null;
    try{
        as = new Socket(InetAddress.getLocalHost(),4020);

        File f = new File (args[0]);
        byte [] buffer  = new byte [(int) f.length()];
        fis = new FileInputStream(f);
        setBis(new BufferedInputStream(fis));
        out = as.getOutputStream();
        System.out.println("uploading...");
        out.write(buffer,0,buffer.length);
        out.flush();
        out.close();
        System.out.println("the file is uploaded.");
        as.close();
    }catch(IOException e){
        e.printStackTrace();
    }   

}


Answer:

The buffer in client does not seem to be populated with data. It is initialized as an array of bytes with the length of the file, but there are no read method calls done on the input stream. For the purpose of testing a fis.read(buffer) would probably quickly get some data into the buffer. Keep in mind that reads are not guaranteed to fill the whole length of the buffer. So especially if your file contains zeros then the lack of reading actual data into the buffer (of the client) is the likely culprit.

Other than that the server code also assumes that the read method fully populates the buffer, so the write method call should specify a length (count). So change bos.write(buffer) into bos.write(bos, 0, count). This will probably become apparent at the end of the file (if the file is more than 1024 bytes long), as the end of the file would become a repetition of some of the data from the previous chunk.

Question:

in my Client-Server application I found a strange error. I got the following Methods :

sendLoginResponse();
sendPlayerList();
sendCurrentLevelState();

Each Methods sends a byte array to the Client-Side

If I only call 2 of them, everything works fine and the Client-Side gets all the sent byte arrays. But if I call all 3 of them only the first and second one arrive the Client, the order of the following methods doesnt matter. but the Server says all of them were send. To write to the Client iam using the write(byte[] b, int off, int len); method all The lenghts within the packages make sense too. Here comes the strange point:

if I add a Thread.sleep(1000); after the second Method, the third one does now arrive the Client after the sleep. I Have also tried to flush the DataOutputStream after every write call, but this didnt help.

EDIT:

So let's say I'd send 3 Login-Response

The Method's that gives me the byte[]:

public byte[] getLoginResponse(int playerID){
    byte[] msg = new byte[4];
    short shortMsgLength = 4;
    byte[] msgLength = shortToBytes(shortMsgLength);
    msg[0] = 2;
    msg[1] = msgLength[0];
    msg[2] = msgLength[1];
    msg[3] = (byte) playerID;
    return msg;
}

private byte[] shortToBytes(short value) {
    byte[] returnByteArray = new byte[2];
    returnByteArray[0] = (byte) (value & 0xff);
    returnByteArray[1] = (byte) ((value >>> 8) & 0xff);
    return returnByteArray;
}

And the Send Method:

private void sendLoginResponse() {
    try{
        byte[] msg = rfcObject.getLoginResponse(playerID);
        out.write(msg,0,msg.length);
    }catch(Exception e){
        System.err.println(e.getMessage());
        System.exit(0);
    }
}

So if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times. If i add a

Thread.sleep(1000); `after the second Method-Call, everything works fine..`

The Client that reads the message runs in a Thread:

public void run(){
    while(true){
        try {
            byte[] data = new byte[MAX_DATA_SIZE]; // MAX_DATA = 255
            byteCount = in.read(data);

        } catch (IOException ex) {
            handleExceptionError(ex);
        }
    }
}

thank you!


Answer:

if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times.

This is because TCP is a stream-oriented protocol. Meaning it doesn't know or care how your messages are delimited. There's no concept of individual messages in TCP, just a stream of bytes, with the guarantee that the order of bytes is preserved.

So when the sender calls three write, the three byte arrays are simply concatenated over the connection and arrives at the receiver in the same order, but the receiver doesn't necessarily need three read to get all the bytes, and even if it does take three read, the read doesn't necessarily gives you the same byte array passed to each corresponding write.

Your message already have the necessary information to get the individual message back from the byte stream:

// Client code for reading individual messages from a TCP connection

byte type = din.readByte();

// Read the message length, little-endian.
// We cannot use din.readShort because it's big-endian
int lenLo = din.read();
int lenHi = din.read();
short len = (short)(lenLo | (lenHi << 8));
byte [] body = new byte[len];
din.readFully(body);

Question:

The remote Tcp Server device sends out 8 bytes of data at a time and does so at relatively fast pace(but not horribly fast). Anyways, i want to receive the 8 bytes and enact on the 8 bytes given. However, i appear to be receiving bytearrays with multiples of these 8 bytes back to back to where my count(in code) is several multiples of 8 bytes, varying in length from 16 to 120+ bytes.

Checking Wireshark, the remote Tcp Server appears to be sending out the 8 bytes correctly and not combining them. This shows me that the reading of the socket is too slow and thus combining the 8 bytes intervals back to back for me, which i don't want.? An example being:

8bytes-8bytes-16bytes-8bytes-8bytes-56bytes.

here is a timestamping of how soon data streams in. with as little as 10 ms.

(13:44:23.210) 00h 10h 01h 06h FFh 3Ah FFh FFh
(13:44:23.290) 00h 10h 01h 06h FFh 3Fh FFh FFh
(13:44:23.300) 00h 10h 01h 06h FFh 44h FFh FFh

How do i efficiently read from the port to be as fast as possible while ensuring no multiples of the 8 byte data?

The code below is a segment that reads data of unknown size and posts it back to the NetworkManager for further processing. I have tried using different streams in hopes of getting the desired effect, but neither stream ensures that.

    private class Receive extends Thread {

        private InputStream inputStream;
        private DataInputStream dataInputStream;
        private BufferedInputStream bufferedInputStream;
        private ByteArrayOutputStream byteArrayOutputStream;

        public void run() {
            if (DEBUG) Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":receive run.");
            bundle = new Bundle();
            buffer = new byte[1024];
            try {
                bufferedInputStream = new BufferedInputStream(socket.getInputStream());
            } catch (IOException ioException) {
                Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":IOException", ioException);
            }
            while (connected && !closed) {
                if (DEBUG) Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":retry read.");
                try {
//                    socket.setSoTimeout(timeout);
                    int count = bufferedInputStream.read(buffer);
                    byteArrayOutputStream = new ByteArrayOutputStream(count);
                    if (count == -1) {
                        networkManager.stopTcpNetwork(name);
                        closed = true;
                        Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + "Connection Lost?!?!");
                    } else {
                        if (DEBUG) Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":call back." + " Count: " + String.valueOf(count));
                        byteArrayOutputStream.write(buffer, 0, count);
                        bundle.putByteArray(name, byteArrayOutputStream.toByteArray());
//                        bundle.putByteArray(name, buffer);
                        message = networkManager.obtainMessage();
                        message.what = Messages.MESSAGES_TCP_DATA.getInt();
                        message.setData(bundle);
                        networkManager.sendMessage(message);
                    }
                } catch (SocketTimeoutException socketTimeoutException) {
                    if (DEBUG) Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":no data yet.");
                } catch (Exception exception) {
                    Log.d(TAG + " \"" + name + "\"", String.valueOf(getId()) + ":Exception", exception);
                }
            }
        }
    }

Answer:

The socket just buffers the data before it gets read into a buffer i guess. As others have pointed out, the correct answer is to either read everything in from the socket or only read an expected number of bytes from the socket.

anyways the project grew and the data that was needed to pull in also grew. that being said, the easiest way to done was to send the data back to the other thread to be locally buffered and then ultimately cycled through for potential messages(that are checked to be complete) and processed onward.

So, Hopefully this helps others.. if not, oops and sorry. Thanks to all who commented and helped!

Question:

EDIT:

I have changed the code so that the integrity of the file is checked after the transfer.

But now when I put a breakpoint in the client at dos.write(buffer, 0, count), kill the server, and then resume client code execution, it hangs at serverMD5[i] = dataInputStream.readByte() indefinitely.

Even though the user now knows that the transfer was not successful (the application hangs and needs to be restarted), once again this is not doing what I expected it to do (throw an IOException).

Original post with changed code:

I have created an android client that connects to a server using SSLSocket and sends some data. Here is the relevant client and server code

Client:

try {
  SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket();
  sslsocket.connect(new InetSocketAddress(SERVER_IP, UPLOAD_PORT), 2000);

  OutputStream outputStream = sslsocket.getOutputStream();

  DataOutputStream dataOutputStream = new DataOutputStream(outputStream);
  dataOutputStream.writeInt(DEVICE_ID);
  dataOutputStream.writeLong(FILE_LENGTH);

  MessageDigest md = MessageDigest.getInstance("MD5");
  DigestOutputStream dos = new DigestOutputStream(outputStream, md);

  InputStream readingsInputStream = new FileInputStream(FILE_NAME);

  int count;
  byte[] buffer = new byte[10 * 1024];
  while ((count = readingsInputStream.read(buffer)) > 0) {
    dos.write(buffer, 0, count);
  }
  readingsInputStream.close();

  byte[] md5 = md.digest();
  byte[] serverMD5 = new byte[16];

  DataInputStream dataInputStream = new DataInputStream(sslsocket.getInputStream());

  for (int i = 0;i<16;i++) {
    serverMD5[i] = dataInputStream.readByte();
    if (md5[i] != serverMD5[i]) throw new Exception("MD5 mismatch");
  }

  sslsocket.close();
} catch (Exception e) {
  ...
}

Server:

try {
  SSLSocket sslSocket = (SSLSocket) sslServerSocket.accept();

  InputStream inputStream = sslSocket.getInputStream();

  DataInputStream dataInputStream = new DataInputStream(inputStream);
  int deviceID = dataInputStream.readInt();
  long fileLength = dataInputStream.readLong();

  MessageDigest md = MessageDigest.getInstance("MD5");
  DigestInputStream dis = new DigestInputStream(inputStream, md);

  OutputStream readingsOutputStream = new FileOutputStream("Device"+deviceID+".txt", false);

  int count;
  byte[] buffer = new byte[1];
  do {
    count = dis.read(buffer);
    readingsOutputStream.write(buffer, 0, count);
    fileLength -= count;
  } while (fileLength > 0);

  readingsOutputStream.close();

  byte[] md5 = md.digest();

  DataOutputStream md5OutputStream = new DataOutputStream(sslSocket.getOutputStream());
  for (int i = 0;i<16;i++) md5OutputStream.writeByte(md5[i]);

  sslSocket.close();
} catch (Exception e) {
  ...
}

Normally this all works as expected but the problem occurs when I put a breakpoint in the client at the line dos.write(buffer, 0, count) and then kill the server upon reaching the breakpoint.

After continuing code execution on the client, it doesn't throw an exception and just goes through the rest of this code block, leading me to believe that the file was successfully written to the server.

Of course this is not the case since the server was shutdown before the OutputStream was written to. This results in an empty DeviceX.txt (X being the number of the device) file on the server.

This is a big problem since the user might think that the data was successfully transferred and delete it from the device (the data that is sent gets deleted at some point after due to the nature of the application).

Since I have managed to produce this bug I figure it is a possibility that it will happen in a real-world scenario as well. This is my first time working with sockets and I am at a loss as to what to do to fix this issue.

Also if anyone notices anything else that could go wrong with this code block (another scenario where the result is not as expected but an exception isn't thrown) please let me know.


Answer:

This is normal operation of TCP. Leaving SSL aside, your sends are buffered in the socket send buffer, and transmitted asynchronously after the send() function has returned. It is therefore impossible for the send() function to detect a peer outage immediately. If you keep sending, TCP's retries of the pending data will eventually fail and cause a subsequent send to fail, in the case of Java with an IOException: connection reset.

inputStream.read(deviceIDbuffer);

You can't assume that read() fills the buffer. You should use DataInputStream.readInt() here.

Question:

My problem is that when i send data ( with socket.write()) from the nodejs server to the java client the java client is stuck in the datastream as long as there is no FIN packet( which would be send when i use socket.end() in nodejs) from the server.

My quesiton is now wether there is a way java can read it without the FIN package.

I thought there has to be a way because it works perfectly when you create a client with the net module of NodeJS.

Server
var server = require("net").Server();

function cl(t){console.log(t)};

server.listen("5022","127.0.0.1", function(req,res){
cl("Server started...");

});


server.on("connection", function(socket){
var ip = socket.remoteAddress;

socket.setEncoding("utf8");

cl("connection --> "+ip);

socket.write("Welcome...");

socket.on("data",function(d){
    var data = JSON.stringify(d);
    cl("Data arrived "+data);

});

socket.on("end", function(){
    cl("end");
})

socket.on("close",function(){
    cl("Disconnect --> "+ip+"\n");
})

socket.on("error", function(err){
    cl("ERROR "+err);

});
});

Note: So as is said, when i would add socket.end() a FIN packet would be send and the java client gets out of the datastream and returns the data. So at the moment i can send data from the server once in the entire session.

part of Client
Socket sc = new Socket(ip, port);

BufferedReader in = new BufferedReader(new
InputStreamReader(sc.getInputStream()));

DataOutputStream out = new DataOutputStream(sc.getOutputStream());
String input;

while(true)
{
   while (!in.ready()) {Thread.sleep(2000);}
   input = in.readLine();
   System.out.println("Message : " + input);
   out.writeUTF(input);
}

Note: Sending data to the server does work from this java client.


Answer:

in.readLine() is doing just what it says, reading a line, which means it's looking for a newline character to know when to stop reading. So in your node code just append a \n when you .write() so that the Java code can resume. Example: socket.write("Welcome...\n");

Question:

I have created a TCP socket connection using Android service. The activity starts when the button is clicked in the MainActivity. And it works properly and closes the connection when the Activity is destroyed. MyService.java

public class MyService extends Service{
    public static Socket clientsocket;
    public static PrintWriter printer;
    SendMessage sender;

    @Override
    public void onCreate() {
    Toast.makeText(this, "Created", Toast.LENGTH_LONG).show();
    super.onCreate();
    }

    @Override
    public void onDestroy() {
    Toast.makeText(this, "Stoped", Toast.LENGTH_LONG).show();
    if(clientsocket!=null){
            try{
            clientsocket.close(); 
            Toast.makeText(this, "Socket Closed", Toast.LENGTH_LONG).show();
            }catch(IOException e){
                e.printStackTrace();
            }
        }
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        Toast.makeText(this, "Running", Toast.LENGTH_LONG).show();
        SendMessage sender=new SendMessage();
        sender.execute();
        return super.onStartCommand(intent, flags, startId);
    }

    @Override
    public IBinder onBind(Intent arg0) {
        return null;
    }

    class SendMessage extends AsyncTask<Void, Void, Void>{

        @Override
        protected Void doInBackground(Void... params) {
            try {
                clientsocket = new Socket("192.168.237.1", 6666);
            } catch (UnknownHostException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return null;
        }

        @Override
        protected void onPostExecute(Void result) {
            if(clientsocket!=null){
                Toast.makeText(MyService.this, "Connected", Toast.LENGTH_LONG).show();
            }else{
                Toast.makeText(MyService.this, "Lost Connection", Toast.LENGTH_LONG).show();
            }
            super.onPostExecute(result);
        }

    }
}

MainActivity.java

public class MainActivity extends Activity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

    }

    public void start(View v){
        startService(new Intent(this,MyService.class));
    }
    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
         getMenuInflater().inflate(R.menu.main, menu);
         return true;
    }

    class PrintMessage extends AsyncTask<Void, Void, Void>{

        @Override
        protected Void doInBackground(Void... arg0) {
            return null;
        }
    }

    @Override
    protected void onDestroy() {
        stopService(new Intent(this,MyService.class));
        super.onDestroy();
    } 
}

Now I have some questions:

  1. Is it possible to create PrintWriter objects in MainActivity using Socket object (clientsocket) of service?
  2. Is it possible to pass values from MainActivity to service(i.e is IP address and port)?
  3. Is it possible to check socket connection in MainActivity(i.e socket.isConnected)?

Please help .


Answer:

1.Is it possible to create PrintWriter objects in MainActivity using Socket object (clientsocket) of service?

You can use messager and send your object in bundle from service to activity.

see this ho to send data from service to activity: Send JSON data from a service to UI in Android

2.Is it possible to pass values from MainActivity to service(i.e is IP address and port)? yes you can pass ip/port via an intent and get them in onStartcomant of service

i.e

i.putExtra(name, value);

i.getStringExtra(name)

3.Is it possible to check socket connection in MainActivity(i.e socket.isConnected)?

yes you can send any update from service to mail activity via messanger

for sending printwriter:

Message msg = Message.obtain();
Bundle data = new Bundle();
data.putSerializable("key", new printwriterHolder(mPrintWriter));

//class to hold print writer
class printwriterHolder implements Serializable{
    PrintWriter mPrintWriter;

    public printwriterHolder(PrintWriter mPrintWriter){
        this.mPrintWriter=mPrintWriter;
    }

    public PrintWriter getPrintWriter(){

        return mPrintWriter;
    }
}

Question:

I am trying to get the linked example code to work, but am experiencing a SocketException (below). I am using java 7.

https://github.com/smartystreets/LiveAddressSamples/blob/master/java/LiveAddressExample.java

IO Exception Error: java.net.SocketException: Connection reset
Exception in thread "main" java.lang.NullPointerException
    at com.icat.coastal.processor.SmartyStreetAddress.main(SmartyStreetAddress.java:71)

I think this is due to this line:

DataOutputStream outgoing = new DataOutputStream(urlConn.getOutputStream());

I tried adjusting the firewall as suggested here, but it did not work: http://www.javaprogrammingforums.com/java-networking/17319-java-net-socketexception-connection-reset.html

How do I dig deeper to determine the problem, or what do you think could be wrong?


Answer:

I just tried a fresh attempt, copying straight from the github example and inserting my auth-id and auth-token. Here's what my terminal session looked like:

$ java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

$ ll
drwxr-xr-x   6 mike  staff   204B Feb  9 11:49 ./
drwxr-xr-x  18 mike  staff   612B Dec 22 23:01 ../
-rw-r--r--   1 mike  staff   3.2K Feb  9 11:49 LiveAddressExample.java
-rw-r--r--   1 mike  staff   349B Dec 22 23:01 README.md
-rw-r-----@  1 mike  staff    23K Feb  9 11:45 json-simple-1.1.1.jar

$ javac -cp json-simple-1.1.1.jar LiveAddressExample.java 
Note: LiveAddressExample.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: LiveAddressExample.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

$ java -cp .:json-simple-1.1.1.jar LiveAddressExample
3785 Las Vegas Blvd S
Las Vegas NV 89109-4333

1600 Amphitheatre Pkwy
Mountain View CA 94043-1351

So at this point I'm having trouble replicating the error. A coworker of mine tried the same thing with java 1.6 and that also worked. Are there any other details you can provide about your environment or any details related to edits you've done to the code?

Question:

I'm communicating with peers using TCP Sockets and I see that when I read the inputStream for the first incoming message, all goes well. Then when I read the inputStream for the second incoming message, the inputStream skip the first n bytes (n is a different positive number in each run).

How do I know that inputStream skip n bytes? Using Wireshark, I can see that the second message received well but Java TCP socket still ignore the first n bytes.

Moreover, Wireshark itself show me something strange - by looking at the first message in Wireshark, it contains at the end: the start of the second message. And by looking at the second message in Wireshark, the start of the message appears here also.

I can't understand what is going on.

Technical details + Wireshark photos:

  1. The first message I receive is a 'Handshake' message.
  2. The second message I receive is different each time but most of the time it's 'extended' message.
  3. I checked in my code and I only read the same InputStream in 2 places: When I'm waiting for 'Handshake' and when I'm waiting for the rest of the messages which is not equal to 'Handshake' message.

The first message I receive:

 * Offset       Size            Name        value
 * 0            8-bit           byte        pstrLength
 * 1            pstrlen-bit     bytes       pstr
 * 1+pstrlen    64-bit          byte        reserved
 * 9+pstrlen    20-bit          String      torrentInfoHash
 * 29+pstrlen   20-bit          String      peerId
 * 49+pstrlen
public HandShake(InputStream dataInputStream) throws IOException {
    byte[] data = new byte[1];
    dataInputStream.read(data);
    byte pstrLength = ByteBuffer.wrap(data).get();
    data = new byte[pstrLength + 48];// how much we need to read more.
    dataInputStream.read(data);

    ByteBuffer byteBuffer = ByteBuffer.allocate(1 + pstrLength + 48);
    byteBuffer.put(pstrLength);
    byteBuffer.put(data);
    HandShake handShake = HandShake.createObjectFromPacket(byteBuffer.array());

Details: 13 until 45 is the content of the first message - Handshake. 00 until 3a is the first n bytes fo the second message which will appear also in here:

The second message I receive:

public static PeerMessage create(Peer from, Peer to, InputStream inputStream) throws IOException {
    byte[] data = new byte[4];
    boolean isPeerClosedConnection = (inputStream.read(data) == -1);
    if (isPeerClosedConnection)
        throw new IOException("the peer closed the socket:" + from.toString());
    int lengthOfTheRest = ByteBuffer.wrap(data).getInt(); // how much do we need to read more
    data = new byte[lengthOfTheRest];

    isPeerClosedConnection = (inputStream.read(data) == -1);
    if (isPeerClosedConnection)
        throw new IOException("the peer closed the socket:" + from.toString());

    ByteBuffer byteBuffer = ByteBuffer.allocate(4 + lengthOfTheRest);;
    byteBuffer.putInt(lengthOfTheRest);
    byteBuffer.put(data);

    return create(from, to, byteBuffer.array()); // initialize message object from byte[]
}

Details: 00 until 3a is the first n bytes of the second message.

When I read the InputStream, I get the following bytes: from 6d to 65.

Why Wireshark shows the same data twice and why my InputStream skip the first n bytes of the second message?


Answer:

You wrote:

I calculate how much to read and I use every byte.

You coded:

data = new byte[pstrLength + 48];// how much we need to read more.
dataInputStream.read(data);

This code does not conform with your description. The second read() is not guaranteed to fill the buffer. See the Javadoc. Change it to readFully().

NB There is another problem, in your isPeerConnected test. You are reading a byte of input and throwing it away. This will cause you to lose synchronization with the peer if it is still connected.

Question:

I am trying to develop some kind of chat application (learning purposes) but when I send my objects from the client to the server only the first one seems to get received. No error is shown or nothing so I don't really know whats wrong.

Server code:

import chattychat.domain.Message;
import chattychat.domain.User;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.ServerSocket;
import java.net.Socket;

 /**
 *
 * @author 
 */

public class Server {
private final int PORT;

private ServerSocket server;

public Server(){
    this.PORT = 6565;
}

public Server(int port){
    this.PORT = port;
}


public void listenSocket(){
    try{
        this.server = new ServerSocket(this.PORT);
    } catch (IOException e) {
        System.out.println("Could not listen on port 4444");
        System.exit(-1);
    }
    while(true){
        ClientWorker w;
        try{
            //server.accept returns a client connection
            w = new ClientWorker(server.accept());
            Thread t = new Thread(w);
            t.start();
        } catch (IOException e) {
            System.out.println("Accept failed: 4444");

        }
    }
}

class ClientWorker implements Runnable {
    private Socket client;
    private ObjectInputStream in;

    //Constructor
    ClientWorker(Socket client) {
        this.client = client;
    }

    public void run(){
        try{
            this.in = new ObjectInputStream(this.client.getInputStream());
            Object obj = this.in.readObject();
            System.out.println("called");
            if(obj instanceof Message){
                Message msg = (Message)obj;
                System.out.println(msg);
            }else if(obj instanceof User){
                User user = (User)obj;
                System.out.println(user);
            }
            System.out.println();
        }catch (IOException e) {
            System.out.println("Read failed");
            //System.exit(-1);
        } catch (ClassNotFoundException ex) {
            System.out.println("Object not found");
            //System.exit(-1);
        }
    }
}

@Override
protected void finalize(){
    try{
        this.server.close();
    } catch (IOException e) {
        System.out.println("Could not close socket");
        System.exit(-1);
    }
}

public static void main(String[] args){
    Server s = new Server();
    //s.communicate();
    s.listenSocket();
}

}

Client code:

import chattychat.domain.Message;
import chattychat.domain.User;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.net.Socket;
import java.net.SocketException;

/**
 *
 * @author laurensputseys
 */
public class Client {

    private final int PORT = 6565;
    private String SERVER = "localhost";

    private Socket socket;
    private ObjectInputStream in;
    private ObjectOutputStream out;
    private boolean isConnected;

    public Client(){
        this.isConnected = false;
    }

    public Client(String server){
        this.SERVER = server;
    }

    public String getServer(){
        return this.SERVER;
    }

    public int getPort(){
        return this.PORT;
    }

    public void communicate(Message msg){
        while(!isConnected){
            try{
                this.socket = new Socket(this.getServer(), this.getPort());
                isConnected = true;
                this.out = new ObjectOutputStream(this.socket.getOutputStream());
                this.out.writeObject(msg);
                this.out.flush();
                this.out.close();
            }catch(SocketException se){
                se.printStackTrace();
            }catch(IOException e){
                e.printStackTrace();
            }
        }
    }

    public void register(User usr){
        while(!isConnected){
            try{
                this.socket = new Socket(this.getServer(), this.getPort());
                isConnected = true;
                this.out = new ObjectOutputStream(this.socket.getOutputStream());
                this.out.writeObject(usr);
                this.out.flush();
                this.out.close();
            }catch(SocketException se){
                se.printStackTrace();
            }catch(IOException e){
                e.printStackTrace();
            }
        }
    }

    public boolean isConnected(){
        return isConnected;
    }

    public static void main(String[] args){
        Client c = new Client();
        User u1 = new User("firstname 1", "lastname1", "testuser");
        User u2 = new User("firstname 2", "lastname2", "Testuser");
        Message m1 = new Message(u1, u2, "Hello from P1 to P2");
        Message m2 = new Message(u2, u1, "Hello from P2 to P1");
        System.out.println("Send 1");
        c.communicate(m1);
        System.out.println("Send 2");
        c.communicate(m2);
        System.out.println("Send 3");
        c.register(u1);
    }
}

message class:

import java.io.Serializable;
import java.util.Objects;

/**
 *
 * @author 
 */
public class Message implements Serializable{

    private static final long serialVersionUID = 7526472295622776147L;
    private User from;
    private User to;
    private String message;

    public Message(User from, User to, String message){
       this.setFrom(from);
       this.setTo(to);
       this.setMessage(message);
    }

    private void setFrom(User from){
        this.from = from;
    }

    private void setTo(User to){
        this.to = to;
    }

    private void setMessage(String message){
        this.message = message;
    }

    public String getMessage(){
        return this.message;
    }

    @Override
    public boolean equals(Object o){
        if(this == o) return true;
        if(o == null || getClass() != o.getClass()) return false;
        Message tmpMsg = (Message)o;
        return this.to.equals(tmpMsg.to) && this.from.equals(tmpMsg.from) && this.getMessage().equals(tmpMsg.getMessage());
    }

    @Override
    public int hashCode() {
        int hash = 7;
        hash = 59 * hash + Objects.hashCode(this.from);
        hash = 59 * hash + Objects.hashCode(this.to);
        hash = 59 * hash + Objects.hashCode(this.message);
        return hash;
    }

    @Override
    public String toString(){
        return "From: " + this.from + " to: " + this.to + " message: " +this.getMessage();
    }
}

User class:

import java.io.Serializable;
import java.util.Objects;

/**
 *
 * @author
 */
public class User implements Serializable{

    private static final long serialVersionUID = 7526456787322776147L;
    private String firstname;
    private String lastname;
    private String password;

    public User(String firstname, String lastname, String password){
        setFirstname(firstname);
        setLastname(lastname);
        setPassword(password);
    }

    private void setFirstname(String firstname){
        this.firstname = firstname; //TODO
    }

    private void setLastname(String lastname){
        this.lastname = lastname; //TODO
    }

    private void setPassword(String password){
        this.password = password; //TODO
    }

    @Override
    public boolean equals(Object o){
        if(this == o) return true;
        if(o == null || getClass() != o.getClass()) return false;
        User tmpUser = (User)o;
        return this.firstname.equals(tmpUser.firstname) && this.lastname.equals(tmpUser.lastname);
    }

    @Override
    public int hashCode() {
        int hash = 7;
        hash = 59 * hash + this.firstname.length();
        hash = 59 * hash + this.lastname.length();
        hash = 59 * hash + this.password.length();
        return hash;
    }

    @Override
    public String toString(){
        return this.firstname + " " + this.lastname;
    }
}

Thanks in advance!


Answer:

Without reading your code very thoroughly. Your clients communicate methods sets connected to true on the first call. So the while loop will not be entered on the second call to communicate. You should separate connecting to server and sending messages. Also, your client worker thread in server seems to return after one object read. The run method should return when connection is broken, not when one message is read.

Question:

I a simple server that receives bytes using TCP and then saves them to a file stream. Through many tests I have seen that the first packet received is always just the filename with no other data. The second packet received only has one byte and it is the first letter of the input text file. After this all packets are sent correctly, but I can't seem to figure out what is messing up the second packet. It also appears that the last packet is written twice. Can anyone see what I am doing wrong? Here is an example Input/Output: https://www.diffchecker.com/srclclrx

    InputStream in = clntSock.getInputStream(); //server's input stream - gets data from the client
        OutputStream out = clntSock.getOutputStream(); //server's output stream - server sends data to the client

        byte[] byteBuffer = new byte[BUFSIZE];
        int count = in.read(byteBuffer, 0, BUFSIZE);
        String firstRead = new String(byteBuffer, 0, count);
        int fileNameEnd = firstRead.indexOf("\r\n");
        String fileName = firstRead.substring(0, fileNameEnd);

        FileOutputStream fout = new FileOutputStream(fileName); //unzipped file output stream

        int contentBegin = fileNameEnd+2;
        byte[] oldBuffer = Arrays.copyOfRange(byteBuffer, contentBegin, count);
        int oldCount = count-contentBegin;
        String oldString = new String(byteBuffer, contentBegin, count-contentBegin, "US-ASCII");

        while((count = in.read(byteBuffer, 0, BUFSIZE)) != -1) { // read from origin's buffer into byteBuffer until origin is out of data
            String newString = new String(byteBuffer, 0, count, "US-ASCII");
            String combinedString = oldString + newString;
            int index = combinedString.indexOf("--------MagicStringCSE283Miami");
            if(index != -1){
                System.out.println("Final Print");

                byte[] combinedBuffer = concat(oldBuffer, byteBuffer);

                for(int i=0; i<index; i++){
                    System.out.print((char)combinedBuffer[i]);
                }
                System.out.println("");

                fout.write(combinedBuffer, 0, index);
                fout.flush();
                fout.close();
                break;
            }
            System.out.println(+ oldCount);
            fout.write(oldBuffer, 0, oldCount); //write the byteBuffer's data to the client via the zip output stream
            fout.flush(); //push all data out of the zipOutputStream before continuing
            if(count == 1){
                for(int i=0; i<count; i++){
                    System.out.println((char)byteBuffer[i]);
                }
            }

            oldBuffer = byteBuffer;
            oldCount = count;
            oldString = newString;
        }

Edit: Another peculiarity to me is that the second to last packet is always just "-" and then the last packet has the remainder of the magic string which terminates the file output stream.


Answer:

Are you really sure that you are taking the full content of the data you receive ?

while((count = in.read(byteBuffer, 0, BUFSIZE)) != -1) { // read from origin's buffer into byteBuffer until origin is out of data
     add logic here to print count
     add logic here to print the content of the byteBuffer 
}

It is very likely that in your logic you are mistreating what you receive and somehow loose part of the data. For instance your second packet where you claim to receive only '-' is the count then just equal to 1 ? It is possible that indeed this is the case with TCP, but you really have to verify that you are indeed processing everything that you receive. Based on your explanation I think you are dropping data, well not processing it correctly really.

Question:

I am using Java sockets (TCP ,read time out set to 30 seconds) to communicate with an external third party server. The server data is continuous stream comprising of application specific packets.

But from past few days the input stream is returning data which represents byte value 0.

 while (connectionIsValid){
 final byte[] buffer= new byte[2];
 in.read( buffer);
  //Print buffer
 //Log line
 Byte[0] is 0 
 Byte[1] is 0

//POST processing if buffer bytes is not equal to OK.
//Other wise bypass post processing
}

- There is no exception logged.And as no stream /socket related exception is generated I know java client socket is not closed/time out.

And application is lopping through while loop.

[Updated]

    private byte[] readWrapper(InputStream stream, int totalLen ) throws IOException {
        byte[] buffer;
        byte[] bufferingBuffer = new byte[0];
        while ( totalLen != 0 ) {
            buffer = new byte[totalLen];
            final int read = stream.read( buffer );
            if(read==-1 || read==0){ 
              //throw exception
             } 
            totalLen = totalLen - read;
            buffer = ArrayUtils.subarray( buffer, 0, read );
            bufferingBuffer = ArrayUtils.addAll( bufferingBuffer, buffer );

        }
        return bufferingBuffer;
    }

Answer:

You don't need all that. The JDK already contains a method for it:

private byte[] readChunkedData(InputStream stream, int size) throws IOException {
    byte[] buffer = new byte[size];
    DataInputStream din = new DataInputStream(stream);
    din.readFully(buffer);
    return buffer;
}

This version (a) works (b) doesn't require third-party classes and (c) throws EOFException if there aren't size bytes left in the stream.