Hot questions for Using Transmission Control Protocol in python

Question:

I am trying to send an image in the form of a byte array from a client application to a server. The server code is written in python while the client is written in java. The image is being transferred correctly however the image saved on the server machine is corrupted.

The following is the code for the server.

import socket
import struct

HOST = "192.168.1.100"
PORT = 9999

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(5)

print('SERVER STARTED RUNNING')

while True:
    client, address = s.accept()
    buf = ''
    while len(buf) < 4:
        buf += client.recv(4 - len(buf))
    size = struct.unpack('!i', buf)[0]
    with open('/home/isaac/Desktop/image.jpg', 'wb') as f:
        while size > 0:
            data = client.recv(1024)
            f.write(data)
            size -= len(data)
    print('Image Saved')
    client.sendall('Image Received')
    client.close()

The following is the source code for the java client:

public static void main(String[] args) throws IOException {
    byte[] array = extractBytes("/home/isaac/Desktop/cat.jpg");
    Socket sock = new Socket(IP_ADDRESS, PORT_NO);
    DataOutputStream dos = new DataOutputStream(sock.getOutputStream());
    System.out.println("Array Length - " + array.length);
    dos.writeInt(array.length);
    dos.write(array);
    BufferedReader reader = new BufferedReader(new InputStreamReader(sock.getInputStream()));
    System.out.println(reader.readLine());
}

Hopefully you can help me. I've tried to google to get my answer but no solution has worked so far.


Answer:

In what sense you get a corrupted image? I've tried it with the following array:

byte[] array = new byte[3000];
for(int i=0; i<array.length; i++) {
    array[i] = (byte)('0' + (i % 10));
}

and I get the same array I sent.

Another thing, just to be sure: the file is less than 2Gb, right?

Question:

I need to implement a TCP Client with Spring Integration, using annotation, without xml conf.

The TCP Server has to send files, and i have to use Spring Integration to handle them, and print them (for now). So, i make a TCP Server with python, but there is no importance for this. Code :

import socket as s

host = ''
port = 2303

co = s.socket(s.AF_INET, s.SOCK_STREAM)
co.bind((host, port))
co.listen(5)
print("Server is listening on port {}".format(port))

conn, addr = co.accept()
print('Connected by', addr)

while True:
    try:
        data = conn.recv(1024)

        if not data: break

        print("Client Says: " + data)
        conn.sendall("Server Says:hi")

    except s.error:
        print("Error Occured.")
        break

print("Closing connection")
conn.close()

And for the client here is the code using spring integration :

import org.apache.log4j.Logger;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.ip.tcp.TcpInboundGateway;
import org.springframework.integration.ip.tcp.connection.TcpNetClientConnectionFactory;
import org.springframework.integration.transformer.ObjectToStringTransformer;
import org.springframework.messaging.MessageChannel;


@Configuration
public class TCPInputChannel {

    private static final Logger LOGGER = Logger.getLogger(TCPInputChannel.class);

    @Bean
    TcpNetClientConnectionFactory clientConnectionFactory() {
        LOGGER.info("create TcpNetClientConnectionFactory");
        TcpNetClientConnectionFactory cf = new TcpNetClientConnectionFactory("localhost", 2303);
        cf.setSingleUse(false);
        cf.setSoTimeout(10000);
        return cf;
    }

    @Bean
    TcpInboundGateway gateway() {
        LOGGER.info("create TcpInboundGateway");
        TcpInboundGateway g = new TcpInboundGateway();
        g.setConnectionFactory(clientConnectionFactory());
        g.setClientMode(true);
        g.setRetryInterval(1000);
        g.setRequestChannel(input());
        return g;
    }

    @Bean
    public MessageChannel input() {
        return new DirectChannel();
    }

    @ServiceActivator(inputChannel = "input", outputChannel = "respString")
    ObjectToStringTransformer stringTransformer() {
        LOGGER.info("create ObjectToStringTransformer");
        ObjectToStringTransformer st = new ObjectToStringTransformer();
        return st;
    }

    @ServiceActivator(inputChannel = "respString")
    public String receive(String recv) {
        LOGGER.info("Recv: " + recv);
        return "echo";
    }

}

When I run it, the server can make a connection, but he can't send messages, and the client don't print anything.


Answer:

The AbstractConnectionFactory uses:

/**
 * Reads data in an InputStream to a byte[]; data must be terminated by \r\n
 * (not included in resulting byte[]).
 * Writes a byte[] to an OutputStream and adds \r\n.
 *
 * @author Gary Russell
 * @since 2.0
 */
public class ByteArrayCrLfSerializer extends AbstractPooledBufferByteArraySerializer {

By default. So, your server should ensure \r\n in the end of package it sends.

Or you can select any appropriate deserializer for your purpose: https://docs.spring.io/spring-integration/reference/html/ip.html#connection-factories

Question:

I'm using this method to send data from a Java client and receiving it on a python server. The data sent is a simple string of length 10 but I'm not able to receive it on server side. It shows a blank output for received data even when the connection is established properly. I've checked this question and so I've reprogrammed my server to first receive 2 bytes and then receive the string but it's not working. Can anyone explain? I've put some comments to explain the output.

Server.py

import socket
import time

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
print "Socket Created..."
s.bind((str(socket.INADDR_ANY), 2609))
print "Socket bound to INADDR_ANY with port 2609..."
s.listen(5)
print "Socket listening for connections..."
print "Waiting for client to connect..."
(c, remote_addr) = s.accept()

print "Connection From" + str(remote_addr) + " accepted"
rec = 'pppppppppppppppppp'

print 'rec = ' + rec               #Properly prints the initialized string
print 'lenght is ' + str(len(rec)) #prints length = 18
print "Receiving Data..."
leng = c.recv(2)
print "Data received..."
print leng                         #Does not print Anything

rec = c.recv(1024)

print rec                          #Does not print Anything

print 'lenght is ' + str(len(str(rec)))  #Length is 0
c.close()
print "Client socket closed..."
s.close()
print "Server Socket closed..."

Output

% python Server.py
Socket Created...
Socket bound to INADDR_ANY with port 2609...
Socket listening for connections...
Waiting for client to connect...
Connection From('192.168.43.7', 39100) accepted
rec = pppppppppppppppppp
lenght is 18
Receiving Data...
Data received...


lenght is 0
Client socket closed...
Server Socket closed...

Answer:

As the rest of your code on the sende side is unknown, I cannot identify the problem for sure, but there may be two possible reasons I can imagine:

Sender side

You use the code as stated in your link. If you do not receive anything, this may indicate that the OutputStream is bufferedand nothing is send, yet, although you called write(...). Try to flush() it and see if you receive something.

Receiver side

In Python 3 recv returns a bytes object as stated in the documentation:

The return value is a bytes object representing the data received.

In Python 2, you get a string, as stated in the documentation:

The return value is a string representing the data received.

Now, as you send your strings from a Java client, which encodes in UTF-8, you should decode the bytes or string again at the receiver side with data.decode('utf-8') to get a Unicode string.

Question:

I have a TCP connection between a Java server and Python Client. when 'q' is sent from any to the other connection must be closed. It works when I send 'q' from python to java. However, when I sent 'q' from java to python it does not work. I tried typecasting but no use.

Java Server:

public class Sender {

public static void main(String[] args) throws IOException {
    String fromclient;

    ServerSocket Server = new ServerSocket(25000);

    System.out.println("TCPServer Waiting for client on port 25000");

    while (true) {
        Socket connected = Server.accept();
        System.out.println(
                " THE CLIENT" + " " + connected.getInetAddress() + ":" + connected.getPort() + " IS CONNECTED ");

        PrintWriter out = new PrintWriter(connected.getOutputStream(), true);
        Scanner sc = new Scanner(System.in);


        while (true) {
            String input = sc.nextLine();
            out.println(input);
        }
    }

}

}

Python Client:

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("localhost", 25000))
while True:
    r, _, _ = select.select([client_socket], [], [])
    if r:
        data = client_socket.recv(4096)
        string_data = data.decode('utf-8')
        print str(string_data) == 'q'
        if str(string_data) == 'q' or string_data == 'Q':
            print ('Connection closed')
            client_socket.close()
            break
        else:
            print (string_data)

The statement print str(string_data) == 'q' always returns false


Answer:

You need to change your Java server code from out.println(input) to out.print(input). Add also an out.flush() to force that the data are send immediately not only after the buffer is filled.

println - would also send the linebreak to the client

With print it works as expected.

edit Some additonal. In such a case it helps on the Python client to print out the bytes in data.

Amend the Python client code as

data = client_socket.recv(4096)
for i in bytearray(data):
    sys.stdout.write(hex(i) + " ")
print

Assumed input on Java is q+ENTER. See below the output on the Pathon client side.

using on Java server side out.println

0x71 0xd 0xa
False
q

using on Java server side out.print

0x71
True
Connection closed

That way it's easy to spot the additional bytes (0xd 0xa) which were sent to the client.

Question:

I have a Python server and a java client. when a string is sent from java to python, it's not complete. Example:

Java Code Sending: Helloooooooooo

Recieved in Python as:

he

lloooooooo

oo

here are the codes

Python server:

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

server_address = ("localhost", 10000)
print 'starting up on %s port %s' % server_address
sock.bind(server_address)

sock.listen(5)

while True:
     print 'waiting for a connection'
     connection, client_address = sock.accept()

     try:
         print 'connection from', client_address

         while True:
             data = connection.recv(1024)
             print 'received: ' + data
             print type(data)
             if data:
                 print 'sending data back to the client'
                 connection.sendall(data)
             else:
                 print 'no more data from', client_address
                 break

     finally:
         connection.close()

Java Client:

package server;

import java.io.*;
import java.net.*;

class Reciever_test {
 public static void main(String args[]) throws Exception {
        String sentence;
        String modifiedSentence;
        BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
    Socket clientSocket = new Socket("localhost", 10000);
    DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
    BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
    while(true)
    {
        sentence = inFromUser.readLine();
        outToServer.writeBytes(sentence + "\n");
        modifiedSentence = inFromServer.readLine();
        System.out.println("FROM SERVER:" + modifiedSentence);
    }

    //clientSocket.close();
}

}


Answer:

In vain of what Klaus D.is saying, the recv guarantees how much data at most it will wait for, it might decide to handle what it has before the size is reached. I used a simple buffer, although I am sure there might be better ways:

import socket
import atexit

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ("localhost", 10000)
print ('starting up on %s port %s' % server_address)
sock.bind(server_address)

sock.listen(5)

def close(connection):
    connection.close()

while True:
     print ('waiting for a connection')
     connection, client_address = sock.accept()
     atexit.register(close, connection)

     try:
         print ('connection from', client_address)

         buffer=[]
         while True:

             data = connection.recv(1024)
             for b in data:
                buffer.append(b)
             print ('received: ' + str(data))
             print (type(data))

             if str(buffer[-1]) == '\n': #if changed to use the buffer to decide when done receiving, the newline is the terminator
                 print ('sending data back to the client')
                 connection.sendall(''.join(buffer))
                 buffer = []
             # else:
             #     print ('no more data from', client_address)
             #     break

     finally:
         close(connection)

The atexit is to close the socket gracefully in case your program crashes

Question:

The python server receives an image file from the android and sends the string "OK" in response. The source code of the python server is as follows:

serverSocket = socket(AF_INET, SOCK_STREAM)

serverSocket.bind(ADDR)
print('bind')

serverSocket.listen(CLIENT_NUM)
print('listen')

while True:
print('waiting...')
try:
    connectionSocket, addr_info = serverSocket.accept()
    print('accept')
    print('--client information--')
    print(connectionSocket)

    img = open("./img.jpg", 'wb')
    while True:
        img_data = connectionSocket.recv(BUFSIZE)
        data = img_data
        if img_data:
            while img_data:
                print("receiving Img...")
                img_data = connectionSocket.recv(BUFSIZE)
                data += img_data
            else:
                break
    img_file = open("img.jpg", "wb")
    print("finish img recv")
    img_file.write(data)
    img_file.close()

    connectionSocket.send("OK".encode())
    connectionSocket.close()
    print('connection closed')

except KeyboardInterrupt:
    sys.exit(0)

The android client sends an image file to the python server and it receive the string "OK" from python server. The source code of python server is as follows:

 public void run() {
    try {
        InetAddress serverAddr = InetAddress.getByName(serverIp);
        socket = new Socket(serverAddr, serverPort);
        try {
            dataOutput = new DataOutputStream(socket.getOutputStream());
            dataInput = new DataInputStream(new FileInputStream(img));

            byte[] buf = new byte[BUF_SIZE];
            int dataLen;
            while ((dataLen = dataInput.read(buf)) != -1) {
                dataOutput.write(buf, 0, dataLen);
                dataOutput.flush();
            }
            BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
            Log.d("Socket", reader.readLine());
        } catch (Exception e) {
            StringWriter sw = new StringWriter();
            e.printStackTrace(new PrintWriter(sw));
            String exceptionAsString = sw.toString();
            Log.e("StackTrace", exceptionAsString);
        } finally {
            try {
                if (dataInput != null)
                    dataInput.close();
                if (dataOutput != null)
                    dataOutput.close();
                if (socket != null)
                    socket.close();
            } catch (IOException e) {
                StringWriter sw = new StringWriter();
                e.printStackTrace(new PrintWriter(sw));
                String exceptionAsString = sw.toString();
                Log.e("StackTrace", exceptionAsString);
            }
        }
    } catch (IOException e) {
        StringWriter sw = new StringWriter();
        e.printStackTrace(new PrintWriter(sw));
        String exceptionAsString = sw.toString();
        Log.e("StackTrace", exceptionAsString);
    }

}

The server cannot

If I delete the two lines below, the server receives the file normally. But if I insert the two lines below, the server doesn't receive the file.

BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
Log.d("Socket", reader.readLine());

How can the android client send an image file to the python server and get a response?


Answer:

I solved this problem. I thought the cause of this problem must have been on the server side, so I modified the code of server and it worked!

img = open("./img.jpg", 'wb')
img_data = connectionSocket.recv(BUFSIZE)
data = img_data
firstPacketLen = len(img_data)
print("receiving Img...")
while len(img_data) > 0:
    img_data = connectionSocket.recv(BUFSIZE)
    data += img_data
    if len(img_data) < firstPacketLen:
        break
print("finish img recv")
img.write(data)
img.close()

connectionSocket.send("OK\r\n".encode())
connectionSocket.shutdown(SHUT_RDWR)
connectionSocket.close()
print('connection closed')