Hot questions for Using Transmission Control Protocol in byte

Question:

I am using C# to create a server software for Windows and Java to create the client software. It works fine most of the time, except for those few exceptions that I don't understand.

I am generally using .ReadLine() and .WriteLine() on both ends to communicate, unless I try to send binary data. That's when I write and read the bytes directly. This is how the software is supposed work:

  1. Client requests the binary data
  2. Server responds with the length of the binary data as a string
  3. Client receives the length and converts it into an integer and starts reading (length) bytes
  4. Server starts writing (length) bytes

It works in most cases, but sometimes the client app doesn't receive the full data and blocks. The server always immediately flushes after writing data, so flushing is not the problem.

Furthermore I've noticed this usually happens with larger files, small files (up to ~1 MB) usually are not a problem.

NOTE It seems like the C# server does send the data completely, so the problem is most likely somewhere in the Java code.

EDIT - Here are some logs from the client side

Working download: pastebin.com/hFd5TvrF

Failing download: pastebin.com/Q3zFWRLB

It seems like the client is waiting for 2048 bytes at the end (as it should be, as length - processed = 2048 in this case), but for some reason the client blocks.

Any ideas what I'm doing wrong? Below are the source codes of both server and client:

C# Server:

public void Write(BinaryWriter str, byte[] data)
{
    int BUFFER = 2048;
    int PROCESSED = 0;
    // WriteString sends the String using a StreamWriter (+ flushing)
    WriteString(data.Length.ToString());
    while (PROCESSED < data.Length)
    {
        if (PROCESSED + BUFFER > data.Length)
            BUFFER = data.Length - PROCESSED;

        str.Write(data, PROCESSED, BUFFER);
        str.Flush();

        PROCESSED += BUFFER;
    }
}

Java Client:

public byte[] ReadBytes(int length){
    byte[] buffer = new byte[length];
    int PROCESSED = 0;
    int READBUF = 2048;
    TOTAL = length;
    progress.setMax(TOTAL);
    InputStream m;
    try {
        m = clientSocket.getInputStream();
        while(PROCESSED < length){
            if(PROCESSED + READBUF > length)
                READBUF = length - PROCESSED;

            try {
                PROCESSED += m.read(buffer, PROCESSED, READBUF);
            } catch (IOException e) {
            }
            XPROCESSED = PROCESSED;
        }
    } catch (IOException e1) {
        // Removed because of sensitive data
    }

    return decryptData(buffer);
}

Answer:

I've found a fix. As of now, the server sends the length and right after sends the byte array. For some reason this does not work.

So what I've changed is:

  1. Send length and wait for the client to respond with "OK"
  2. Start writing bytes

Not sure why, but it works. Ran it in a while(true) loop and it's been sending data 1000 times in 4 minutes straight and no problems, so I guess it's fixed.

Question:

I'm using an InputStream to read bytes from a TCP server (written in C#) into a byte[], and encoding them into a string using new String(byteArray, "UTF-16LE"). This method encodes characters in the Basic Multilingual Plane just fine, but does not handle supplementary characters.

I understand that bytes in C# are unsigned whereas Java bytes are signed, and that a supplementary character can be composed of either one or two unicode values.

        ByteBuffer wrapped = ByteBuffer.wrap(dataBytes);
        wrapped.order(ByteOrder.LITTLE_ENDIAN);
        short noOfSites = wrapped.getShort();

        for(int i = 0; i < noOfSites; i++){
            short siteNo = wrapped.getShort();
            short textLength = wrapped.getShort();
            byte[] textBytes = new byte[textLength];
            wrapped.get(textBytes, 0, textLength);

            for(byte bite : textBytes){
                System.out.print(bite+" ");
            } //just to see what's in the byte array

            String siteText = new String(textBytes, "UTF_16LE");
            System.out.println(siteNo + ": " + siteText);
            siteList.add(new Site(siteNo, siteText));
            publishProgress(siteNo + " - " + siteText);
        }

In this instance, dataBytes is the byte array containing the bytes read from the server, noOfSites is the number of objects to be read from the server, siteNo is an ID, textLength is the number of bytes containing the name of the site, and textBytes is the array that holds these bytes.

When receiving the word "MÜNSTER" from the server, the bytes read into the buffer are: 77 0 -3 -1 78 0 83 0 84 0 69 0 82 0. However, the "Ü" character is unrecognised, which I suppose is down to the -3 -1 UTF-16 value that Java is trying (and failing) to encode. I understand that in C#, "Ü" is represented by DC-00, but I don't understand why this becomes -3 -1 in Java. Any help would be greatly appreciated.


Answer:

The "Û" character is not being encoded in your source - the sequence that is getting to your sink side "-3, -1", is 0xfffd - UTF 16 LE encoding for the replacement character.

Without seeing the server-side code it is hard to tell what is going on, but its bad. Utf-16 can handle chars like "Ü" without going out of its way. Actually, it is not even out of the first 256 unicode codepoints, much less outside the Base Multilingual Plane. (That is a character common enough in lots of western languages, and even latin character, how could it be out of a the plane designed to hold characters for all languages in the World?)

What is happening is that the code path from your text to the utf-16 meant for wire-transfer is, at some point, being explicitly instructed to set the replacement character for any char which is not ASCII only (legacy unicode code-points 0x20-0x7f, which include just unaccented latin characters).

To be clear, in other words: the data is being corrupted server-side, and all non-ASCII fitting characters there will likely be squashed to the "replacement character". No amount of fiddling on your client-side code can fix that.

Question:

How should i implement service activator to read specific number of bytes ?

My context.xml,

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
    xmlns:int-ip="http://www.springframework.org/schema/integration/ip"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
        http://www.springframework.org/schema/integration/ip http://www.springframework.org/schema/integration/ip/spring-integration-ip.xsd">

    <int:annotation-config />

    <context:component-scan base-package="com.spring.integration.tcp" />

<bean id="tcpDeserializer"
    class="org.springframework.integration.ip.tcp.serializer.ByteArrayLengthHeaderSerializer">
    <property name="maxMessageSize" value="300" />
</bean>

    <int-ip:tcp-connection-factory id="tcpConnectionFactory" type="server" port="9070" using-nio="true" deserializer="tcpDeserializer" />

    <int:channel id="tcpRequestChannel" />

    <int-ip:tcp-inbound-gateway id="tcpInboundGateway" connection-factory="tcpConnectionFactory" request-channel="tcpRequestChannel" />

    <bean id="messageHandler" class="com.spring.integration.tcp.MessageHandler" />

    <int:service-activator id="tcpServiceActivator" input-channel="tcpRequestChannel" ref="messageHandler" method="receiveAndSend" />

</beans>

My message handler,

package com.spring.integration.tcp;

import java.io.InputStream;
import java.util.Arrays;


public class MessageHandler
{
    public byte[] receiveAndSend(final InputStream inputStream) throws Exception
    {
        int bytesToRead = 50;
        final byte[] requestMessageBytes = new byte[bytesToRead];
        int read = 0;
        while (bytesToRead > read)
        {
            final int c = inputStream.read(requestMessageBytes, read, bytesToRead - read);
            if (c == -1)
            {

                throw new Exception("EOF");
            }
            read += c;
        }
        System.out.println("server received - length [" + requestMessageBytes.length + "] bytes " + Arrays.toString(requestMessageBytes));
        final byte[] responseMessageBytes = new byte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 };
        System.out.println("server sending - " + Arrays.toString(responseMessageBytes));
        return responseMessageBytes;
    }
}

I would like to read first 4 bytes and determine the incoming message length(from first 4 bytes) and then read message bytes whose length is specified by first 4 bytes.

After adding deserializer I get the following exception. Though i send a byte array of length 161.

final byte[] data = new byte[] { 2,....};

Exception,

Sep 28, 2015 7:03:28 PM org.springframework.integration.ip.tcp.connection.TcpNetConnection handleReadException
SEVERE: Read exception localhost:54756:9070:a580527b-95bd-42c7-b1cc-e0726b433199 IOException:Message length 38159362 exceeds max message length: 300

Should the client be based on spring integration to be able to send a message to spring integration based server ?


Answer:

You should configure ByteArrayLengthHeaderSerializer as a deserializer on the <int-ip:tcp-connection-factory> definition.

See its JavaDocs for more info:

 * The default length field is a 4 byte signed integer. During deserialization,
 * negative values will be rejected.
 * Other options are an unsigned byte, and unsigned short.

Question:

I have an application that show CCTV feed from mobile. I successfully develop iOS application and now my client wants me to port to Android. The porting was fine until I'm stuck at this part of code.

This part of code, I have to connect to TCP socket server. When connected, I don't have to send server any thing, instead, server will send me a JPEG image. So, after connected, I'll have to keep reading until I received JPEG end marker (0xFF, 0xD9) and then close the connection.

What I plan to do is

Socket s = new Socket("Server IP Addreess", SERVER_PORT);
DataInputStream dis = new DataInputStream(socket.getInputStream());

ArrayList<Byte> bytes = new ArrayList<Byte>();
boolean err = false;

while (true) {
    byte b = dis.readByte();

    if (b == -1) {
        err = true;
        break;
    }        

    bytes.add(b);

    if ((bytes.get(bytes.size() - 1) == (byte) 0xFF) &&
        (bytes.get(bytes.size() - 2) == (byte) 0xD9)) {
        break;
    }
}

socket.close();

if (!err) {
    // create bitmap from byte array and show in ImageView
}

But I'm not sure that this is correct or not. The other solution I'm thinking about is

Socket s = new Socket("Server IP Addreess", SERVER_PORT);
BitmapFactory.decodeStream(s.getInputSteam());

But, also, I don't know how to close socket when server send 0xFF, 0xD9. Or will the BitmapFactory will detect that marker and close socket connection for me?

Any suggestion are welcome. Thank you!

P.S. I don't have test environment as they took it back when I delivered iOS app. So, I have to develop and then deliver to them to test (by playing with the app). Then if it's not working, they will tell me to correct it but I won't be able to access LogCat or other useful information. This is just like trying to sew a button in dark room, and I can't turn on the light.

: |

EDIT

More information about this server

  • Server won't send length of file, so, number of bytes for each file is unknown.
  • I have to detect 0xFF, 0xD9 that indicate end of file.
  • Problem is, I have to terminate socket connection from my side, server won't do it.
  • I can't change the way server works, it's hardware that my client purchase.

Also, I'm getting one image for each TCP connection, I'm not getting multiple images for single TCP connection.


Answer:

This is a bad idea to just look for some magic bytes to determine the end of the data. While these magic bytes should be at the end of the file they can also happen inside the data.

Better would be to either prefix the data with the length or use a separate TCP connection for each jpeg file and just read until the end of connection (that is until you don't get any more data). If this is not possible you have to do more advanced parsing, see Detect Eof for JPG images.

Question:

this is my simplified code

sock.getOutputStream().write(buff); //send a readRequest
byte[] rbuff = new byte[19210]; //Answer has 19210 bytes of data
sock.getInputStream().read(rbuff);

In wireshark I see that it is splitted in a few tcp packages. But shouldn't the read wait as long as 19210 byte have arrived?

I get the data, but from a certain position all bytes are 0. This is mostly the same position but varies a bit.

Any idea what I do wrong here?


Answer:

You can try to read it in chunks like this:

ByteArrayOutputStream b= new ByteArrayOutputStream();

byte[] buf = new byte[1024];
while (count = in.read(buf)) > 0)
{
  b.write(buf,0,n);
}

byte data[] = b.toByteArray();

Question:

I'm attempting to test packet loss across a TCP connection by comparing a messages sent counter maintained on the client side with a messages received counter on the server side. The messages I'm sending are 25 kilos with the first 5 bytes containing the count from the client in the form of a string padded with asterisks to maintain a 5 byte length. However after 3 or 4 successful messages the server fails to find the count. Is there something that I am forgetting to consider? My guess is that the server is having trouble keeping up with the client as the number of successful messages varies from 3 - 4 before error.

CLIENT SIDE CODE:

try{
        clientSocket = new Socket("192.168.0.15", SERVER_PORT);   
        DataInputStream in = new DataInputStream(clientSocket.getInputStream());
        DataOutputStream out = new DataOutputStream(clientSocket.getOutputStream());
        int messageCounter = 0;
        byte [] message1 = new byte[25 * 1024];
        byte [] returnMessage1 = new byte[25 * 1024];
        byte [] count;
        long startTime1 = System.nanoTime();
        while (messageCounter < 100000) {
            System.out.println(messageCounter);
            addCountToMessage(formatCount(messageCounter).getBytes(), message1);
            out.write(message1);
            in.read(returnMessage1);     
            messageCounter ++;    
        }  
    }

SERVER SIDE CODE:

try {
      byte [] data = new byte[35 * 1024];
      System.out.println("STARTING LOOP");
      System.out.println("***********************************");
      int messageCounter = 0;
      int clientMessageCount = 0;
      String clientCountString = "";
      byte[] clientCount = new byte[5];

      while (true){
          in.read(data);
          clientCount[0] = data[0];
          clientCount[1] = data[1];
          clientCount[2] = data[2];
          clientCount[3] = data[3];
          clientCount[4] = data[4];

          clientCountString = new String(clientCount);
          System.out.println("Received Count String: \"" + clientCountString + "\"");
          clientCountString = clientCountString.replace("*", "");
          clientMessageCount = Integer.parseInt(clientCountString);

          System.out.println("Client Count: " + clientMessageCount + ", Server Count: " + messageCounter);
          out.write(data);
          System.out.println("***********************************");
          messageCounter ++;
      }
   }

OUTPUT OF SERVER:

java TCPServer

STARTING LOOP


Received Count String: "0****" Client Count: 0, Server Count: 0


Received Count String: "1****" Client Count: 1, Server Count: 1


Received Count String: "2****" Client Count: 2, Server Count: 2


Received Count String: "3****" Client Count: 3, Server Count: 3


Received Count String: ""

Exception in thread "Thread-0" java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:481)
at java.lang.Integer.parseInt(Integer.java:527)
at Connection.run(TCPServer.java:52)

Answer:

There are a number of problems in the way you are doing this. The biggest is that you all calling read(byte[]) on your Socket, but never checking how many bytes have actually been read. You are also not reading the bytes that are supposedly being sent with the message after the header.

You might want to have a look at DataFetcher, my solution to dealing with (socket) input streams

Question:

Can someone please explain to me why byte[] buf on the server does not contain 100,000 1s as sent by the client? At around the 60k byte mark the values stay all zeros. I admit I haven't worked much with Streams or TCP so I'm sure I've done something wrong; I just don't understanding what it is. I've tried this with both InputStream and BufferedInputStream on the server with the same result. Thank you to anyone/everyone who can take the time to explain this to me!

Client code is pretty simple:

    byte[] msgBytes = new byte[100000];
    for (int i=0;i<msgBytes.length;i++){
        msgBytes[i] = 1;
    }
    Socket sck = null;
    try {
        sck = new Socket("my.server.net", 1234);
        OutputStream outStream = sck.getOutputStream();
        outStream.write(msgBytes);
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        try {
            sck.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Likewise, server code is also simple:

    ServerSocket srvSock = new ServerSocket(1234);
    Socket sock = srvSock.accept();
    returnAddress = sock.getInetAddress();
    returnPort = sock.getPort();
    BufferedInputStream iStream = new BufferedInputStream(sock.getInputStream(), 100000);
    byte[] buf = new byte[100000];
    iStream.read(buf);

Answer:

Try substituting BufferedInputStream with DataInputStream and use readFully, so your server code should became:

ServerSocket srvSock = new ServerSocket(1234);
Socket sock = srvSock.accept();
returnAddress = sock.getInetAddress();
returnPort = sock.getPort();
DataInputStream iStream = new DataInputStream(sock.getInputStream());
byte[] buf = new byte[100000];
iStream.readFully(buf);

Question:

Hello I write to simple tcp send data to netty server demo, Here is Code:

Client.java

package chat;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;

public class Client {

    private Socket socket;
    private InputStream inputStream;
    private OutputStream outputStream;
    private ByteArrayOutputStream byteArrayOutputStream;

    public Client() {
        try {
            socket = new Socket("127.0.0.1", 8081);
            inputStream = socket.getInputStream();
            outputStream = socket.getOutputStream();
            byteArrayOutputStream = new ByteArrayOutputStream(100);
            System.out.println("Start Client...");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void sendMessage(String context) throws IOException {
       Message message = new Message(context);

        pushDataToStream(message.getLength());
        pushDataToStream(message.getCode());
        pushDataToStream(message.getMagic());
        byteArrayOutputStream.write(context.getBytes());
        pushDataToStream(message.getEnd());
        System.out.println(byteArrayOutputStream.size());
        outputStream.write(byteArrayOutputStream.toByteArray());

        byteArrayOutputStream.reset();
        outputStream.flush();
    }

    private void pushDataToStream(int[] data) {
        if (data != null) {
            for (int i : data) {
                byteArrayOutputStream.write(i);
            }
        }
    }

    public static void main(String[] args) throws Exception {
        Client client = new Client();
        client.sendMessage("HELLO, WORLD");
    }
}

Message.java

package chat;

import java.util.Arrays;

public class Message {

    private int[] length;
    private int[] code;
    private int[] magic;
    private String context;
    private int[] end;

    public Message(String context) {
        length = new int[]{calculateLength(context), 0x00, 0x00, 0x00};
        code = new int[]{calculateLength(context), 0x00, 0x00, 0x00};
        magic = new int[]{0xb1, 0x02,0x00, 0x00};
        this.context = context;
        end = new int[]{0x00};
    }

    public int calculateLength(String context) {
        return 4 + 4 + 4 + context.length() + 1;
    }

    public int[] getLength() {
        return length;
    }

    public void setLength(int[] length) {
        this.length = length;
    }

    public int[] getCode() {
        return code;
    }

    public void setCode(int[] code) {
        this.code = code;
    }

    public int[] getMagic() {
        return magic;
    }

    public void setMagic(int[] magic) {
        this.magic = magic;
    }

    public String getContext() {
        return context;
    }

    public void setContext(String context) {
        this.context = context;
    }

    public int[] getEnd() {
        return end;
    }

    public void setEnd(int[] end) {
        this.end = end;
    }

    @Override
    public String toString() {
        return "Message{" +
                "length=" + Arrays.toString(length) +
                ", code=" + Arrays.toString(code) +
                ", magic=" + Arrays.toString(magic) +
                ", context='" + context + '\'' +
                ", end=" + Arrays.toString(end) +
                '}';
    }
}

NettyServer.java

package chat;


import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.ByteToMessageDecoder;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;

import java.util.List;


public class NettyServer {

    private NioEventLoopGroup bossLoopGroup;
    private NioEventLoopGroup workerLoopGroup;
    private ServerBootstrap bootstrap;

    private NettyServer() {
        bossLoopGroup = new NioEventLoopGroup();
        workerLoopGroup = new NioEventLoopGroup();
        bootstrap = new ServerBootstrap();
    }

    public void run() throws Exception {
        bootstrap.group(bossLoopGroup, workerLoopGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .option(ChannelOption.TCP_NODELAY, true)
                .option(ChannelOption.SO_KEEPALIVE, true)
                .childHandler(new NettyServerHandler());

        ChannelFuture future = bootstrap.bind(8081).sync();
        if (future.isSuccess()) {
            System.out.println("Server start..");
        } else {
            System.exit(-1);
        }
    }

    public class NettyServerHandler extends ChannelInitializer<SocketChannel> {
        @Override
        protected void initChannel(SocketChannel socketChannel) throws Exception {
            ChannelPipeline pipeline = socketChannel.pipeline();
            pipeline.addLast(new NettySimpleDecoder());
            pipeline.addLast(new NettyServerProcess());
        }
    }

    public class NettyServerProcess extends SimpleChannelInboundHandler{

        @Override
        protected void channelRead0(ChannelHandlerContext channelHandlerContext, Object o) throws Exception {
        }
    }

    public class NettySimpleDecoder extends ByteToMessageDecoder {

        @Override
        protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf in, List<Object> list) throws Exception {
            if (in.readableBytes() < 4) {
                return;
            }

            int length = in.readInt();
            //The first readInt i think is 25 but i get 419430400 value
            //i want to know why??
            System.out.println("length : " + length);
        }
    }

    public static void main(String[] args) throws  Exception {
        NettyServer nettyServer = new NettyServer();
        nettyServer.run();
    }


}


+-------+--------+------+----------------+--------+
| Length| Code   | Magic| Actual Content | End    |
| 0x000C| 0x000C | 0xb1 | "HELLO, WORLD" | 0x00   |
+-------+--------+------+----------------+--------+

int length = in.readInt(); The first readInt i think is 25 but i get 419430400 value I want to know why??


Answer:

It seems like a big- and little-endian bug.

Most of bit-oriented protocol consider this issue. Different operation system have not equal endianness. So you can use characters-oriented protocol. They do not have not this issue, but will transport more byte.

Example:

Length 25 in four bytes: 00 00 00 19

Little-endian: 19 00 00 00, decimal: 419430400

Big-endian: 00 00 00 19, decimal: 25

Question:

I try to understand how to send several messages over tcp. One of the practices is send message length in first byte to split while byte array into messages. Form theoreticall point of view I can get into troubles if my first byte (or first bytes) is lost, so it become impossible to split all received bytes into messages. Can first byte be lost in pratice?


Answer:

TCP ensures that the stream is transferred unaltered, with no parts skipped, inserted, swapped, mutated, etc. If for some reason, the transport layer cannot transfer the necessary data (for example due to packet loss), the TCP driver in the operating system will try to retransmit the data, and after a few unsuccessful attempts the connection will be shut down.

Question:

I've been trying to get this working for a few days now, but I've had no success.

I want to send a file over socket client/server. The only difference is: I want to send an object that contains the file bytes.

So the client loads a file, reads chunks of 1024 bytes, store them in a object, and send the object to the server. Since the file can be larger than 1024 bytes, I want to send the object repeatedly but with different bytes stored in them (as the buffer reads it). On the server, I want to compose the array of bytes and save it as a file.

The reason I'm using 1024 is because I want to avoid any sort of out of memory error, if the file is, let's say, 4 GB in size.

I tried doing the following on the client:

File file = new File("C:\\test\\test.txt");
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
ObjectOutputStream oos = new ObjectOutputStream(socket.getOutputStream());
byte[] bytes = new byte[1024];

FileTest ft = new FileTest();
ft.setName("Testing");
int counttest = 1;

while (bis.read(bytes) > 0) {
    ft.setCounttest(counttest);
    ft.setBytes(bytes);
    oos.writeObject(ft);
    counttest += 1;
}

On the server:

int bufferSize = socket.getReceiveBufferSize();
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream("C:\\test\\test2.txt"));    
ObjectInputStream ois = new ObjectInputStream(socket.getInputStream());
byte[] bytes = new byte[bufferSize];

while (true) {
    FileTest ft = (FileTest) ois.readObject();
    if (ft != null) {
        System.out.println(ft.getName());
        bos.write(ft.getBytes());
    }
}

So I tested sending a txt file with a sequence of numbers and the test2.txt file produced by the server came out only with the first 1024 chunk of bytes repeated twice. Also, the counttest integer never increases when received in the server.

Any idea how to accomplish this?

Thanks in advance.


Answer:

You are running into the effects of the ObjectOutputStream's attempt to preserve object identity. Repeatedly writing the same object instance will result on the same instance on the receiver's end. This is generally a good thing, but confusing if you are modifying the object on the sender's end and expecting those modifications to show up on the receiver's end.

Thus, you have two issues:

  1. In order to send the data each time, you either need to create a new FileTest instance each time, or use the writeUnshared() method.
  2. Due to this identity preserving behavior, you will need to periodically reset() the ObjectOutputStream in order to keep all these instances from being held forever (and potentially leading to an OOME on the client or server).

Question:

Well, I want to write a simple java client-server-programme, which exchanges byte arrays over tcp-sockets.

/* Server */
public class Server {

private ServerSocket Server = null;
private Socket Client = null;

public static void main(String[] args) {
    Server A = new Server();
    A.runServer();
    A.listenServer();
}


public void runServer() {
    try {
        Server = new ServerSocket(1234);
    }
    catch (Exception e) {
        System.out.println("Server fault: "+ e.getMessage());
        System.exit(-1);
    }       
}

public void listenServer() {
    try {
        while (true) {
            System.out.println("Waiting...");
            Client = Server.accept();
            System.out.println("Got something new");
            readMessage(Client);
        }
    }
    catch (Exception e) {
        System.out.println("Server fault: "+ e.getMessage());
    }
}

public byte [] readMessage (Socket socket) {

    try {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        byte[] buf = new byte[1];
        int len = -1;
        while((len = socket.getInputStream().read(buf))!=-1){
            baos.write(buf, 0, len);
        }
        for (int i=0; i<baos.toByteArray().length; i++) {
            System.out.println(baos.toByteArray()[i]);
        }

        return baos.toByteArray();
    }
    catch (Exception e) {
        System.out.println("Server fault: "+ e.getMessage());
    }

    return null;
}

public void writeMessage (Socket socket, String Message) {
    try {
        PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(socket.getOutputStream()));
        printWriter.print(Message);
        printWriter.flush();
    }
    catch (Exception e) {
        System.out.println("Server fault: "+ e.getMessage());
    }
}
}

/* Client */
public class Client {

public static void main(String[] args) {

    Client B = new Client();
    B.runClient();

}

public void runClient () {
    Socket socket = null;
    try {
        socket = new Socket("127.0.0.1", 1234);
    }
    catch (Exception e) {
        System.out.println("Client fault: "+e.getMessage());
    }

    byte [] Tmp = new byte[10];
    for (int i=0; i<Tmp.length; i++) {
        Tmp[i] = 1;
    }

    writeMessage(socket, Tmp);

    for (int i=0; i<10; i++) {
        byte []  Message = readMessage(socket);
        System.out.println(Message);
    }
}

public void writeMessage (Socket socket, byte [] myByteMessage) {
    try {
        OutputStream out = socket.getOutputStream();
        DataOutputStream dos = new DataOutputStream(out);
        dos.write(myByteMessage, 0, myByteMessage.length);
        PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(socket.getOutputStream()));
        printWriter.print(myByteMessage);
        printWriter.flush();

    }
    catch (Exception e) {
        System.out.println("Could not send data over TCP");
        return;
    }
}

public byte [] readMessage (Socket socket) {

    try {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        byte[] buf = new byte[1];
        int len = -1;
        while((len = socket.getInputStream().read(buf))!=-1){
            baos.write(buf, 0, len);
        }
        for (int i=0; i<baos.toByteArray().length; i++) {
            System.out.println(baos.toByteArray()[i]);
        }

        System.out.println("Test");
        return baos.toByteArray();
    }
    catch (Exception e) {
        System.out.println("Server fault: "+ e.getMessage());
    }

    return null;
}
}

The problem is, that the client send something to the server but the server doesn't receive anything, so he hangs at the readMessage function. On the other hand, the client receive some weird stuff, but not the response from the server.


Answer:

The server receives bytes, but it never leaves the while loop because read() never returns -1. read() returns -1 when the end of the stream is reached. And that happens only when the client closes the socket output stream. Since the client never closes the output stream, the server keeps waiting for the more bytes to come.

Side note: your code is hard to read because you don't respect the standard Java naming conventions: variables start with a lowercase letter.

Question:

I am using the qpid client Java library (version 0.32) to integrate AMQP 1.0.

I have to transfer a byte array (less than 5mb), but this message is never delivered to the subscribers. I recorded the frames via wireshark and the transfer frames are flagged with [TCP Window Full]. The library is probably not dividing the payload. Is the code correct? What do I have to configure?

Broker: Apache Apollo 1.7.1 (default configuration)

pom.xml

    <dependency>
        <groupId>org.apache.geronimo.specs</groupId>
        <artifactId>geronimo-jms_1.1_spec</artifactId>
        <version>1.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.qpid</groupId>
        <artifactId>qpid-amqp-1-0-client-jms</artifactId>
        <version>0.32</version>
    </dependency>
    <dependency>
        <groupId>org.apache.qpid</groupId>
        <artifactId>qpid-amqp-1-0-client</artifactId>
        <version>0.32</version>
    </dependency>
    <dependency>
        <groupId>org.apache.qpid</groupId>
        <artifactId>qpid-amqp-1-0-common</artifactId>
        <version>0.32</version>
    </dependency>

Java code

  ConnectionFactoryImpl amqpFactory = new ConnectionFactoryImpl(...);
  ConnectionImpl connection = amqpFactory.createConnection();
  connection.start();
  SessionImpl session = connection.createSession(...);

  MessageProducerImpl producer = session.createProducer(new TopicImpl("topic://test"));
  BytesMessageImpl bytesMessage = session.createBytesMessage();

  //generate sample data
  StringBuilder s = new StringBuilder();
  for (int i = 0; i < 10000; i++) {
       s.append(UUID.randomUUID().toString());
  }

  bytesMessage.writeBytes(s.toString().getBytes());
  producer.send(bytesMessage);

Answer:

So you have basically landed on every deprecated bit of AMQP software I know of so I would guess that your issue comes from one of the old and unmaintained pieces of software.

ActiveMQ Apollo has been deprecated an unmaintained for several years now, instead you should use ActiveMQ Artemis which is the most active broker in the ActiveMQ project.

For an AMQP 1.0 JMS client you should use the Qpid JMS client which implements the current AMQP -> JMS mapping specification and is actively maintained.

Question:

I have tried everything to get this to work. Basically I have an Android App which receives data from a Python based Server on a local network connection. I can receive Data no problem. However when I attempt to send data back the App crashes and the Python server receives blank data. I have tried several different approaches but non have worked. Here is the Python method I have written to receive the message:

   def checkReply(self):
    reply = "no reply yet"
    self.conn.settimeout(1)
    try:
        test =  self.conn.recv(1024)
    except:
        self.conn.timeout;
        print("I failed to hear this") #Debug to help see if I have heard an incomming message
    try:
        data = test.decode()

        reply = data
    except:
        print("I failed to decode this") #Debug to help see if I could not decode an incomming message
    print(reply)
    self.conn.settimeout(0)

My client on my Android application looks like this:

public class Client extends AsyncTask<Void, Void, Void> {

String dstAddress;
int dstPort;
String response = "No data has been sent yet";
TextView textResponse;
Socket socket = null;


Client(String addr, int port, TextView textResponse) {
    dstAddress = addr;
    dstPort = port;
    this.textResponse = textResponse;
}

@Override
protected Void doInBackground(Void... arg0) {

    Socket socket = null;

    try {
        socket = new Socket(dstAddress, dstPort);

        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(
                1024);
        byte[] buffer = new byte[1024];

        int bytesRead;
        InputStream inputStream = socket.getInputStream();

     /*
      * notice: inputStream.read() will block if no data return
      */
        while ((bytesRead = inputStream.read(buffer)) != -1) {
            byteArrayOutputStream.write(buffer, 0, bytesRead);
            response = byteArrayOutputStream.toString("Ascii");
        }

        byteArrayOutputStream.flush();


    } catch (UnknownHostException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        response = "UnknownHostException: " + e.toString();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        response = "IOException: " + e.toString();
    } finally {
        if (socket != null) {
            try {
                socket.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }
    return null;
}

@Override
protected void onPostExecute(Void result) {
    textResponse.setText(response);
    super.onPostExecute(result);
}

protected String getSite(){
    return response;
}

@RequiresApi(api = Build.VERSION_CODES.KITKAT)
public void returnMsg(){

        try (DataOutputStream outToClient = new DataOutputStream(socket.getOutputStream())) {
            byte[] buf = "hello".getBytes("UTF-8");
            outToClient.writeBytes("Test");
            outToClient.flush();

        } catch (IOException e) {}




}

}

I am having to test on a physical device so I have no log to trace the error message. Any help would be greatly appreciated. Thanks


Answer:

I managed to solve the issue. I needed to flush the buffer before trying to send any data back via the socket

Question:

I have a byte file that contains tcp packets. I want to use sockets to read those packets.

so is it possible to use Sockets to read this file without connection ?


Answer:

FileInputStream mInStream = new FileInputStream("file path").
byte[] buffer = new byte[1024]; 

    // Keep listening to the InputStream 
    while (true) {
        try {
         bytes = mInStream.read(buffer, 0, buffer.length);
        }catch {} 
    }

Question:

I have a client - server program using TCP. Now I want to send different messages of different size. I read that a char in Java is 2 bytes. If I am using UTF-8, can I just send a char and it will be of size 1 byte?

Also which is better to use: DataInputStream and DataOutputStream or BufferedInputStream and BufferedOutputStream ?


Answer:

You want to use an OutputStreamWriter to wrap your OutputStream. That will let you specify an encoding, and write chars instead of bytes. It's the OutputStreamWriter's job to know the encoding, and to convert chars to bytes appropriately. If you have specified that your OutputStreamWriter uses UTF-8, and you write a character that uses one byte in UTF-8, you can expect just one byte to be written to the stream.

try (OutputStreamWriter writer = new OutputStreamWriter(myOutputStream, "UTF-8")) {
    writer.write('x');
}

You can also wrap the OutputStreamWriter in a BufferedWriter, for increased efficiency.

But doing the encoding yourself using getBytes method of the String class is just re-inventing the wheel. Don't do that.

More information about OutputStreamWriter can be found in the Javadoc.