User Tools

Site Tools


Writing /var/lib/dokuwiki/data/meta/teaching/ie0117/proyectos/2013/i/way_to_shop/multi-process_server.meta failed
teaching:ie0117:proyectos:2013:i:way_to_shop:multi-process_server

Writing a multi-process server with python

The target of this page is to show how to create a server, able to support multiple connections and executing different tasks for each one.

Basic concepts

Python includes a module specifically for create a http web server, BaseHTTPServer. You can find more information here. But, in its simplest way, a http server just waits for a request and returns a document, the web page. But we want to create a server that executes a program using the data sent by the client and communicates the right answer.
First we need to create a network socket, which is the way the client communicates with the server. The server will create a special socket, listening for all the connection requests. Once the communication between the sockets (server-client) has been established, the server will create a new process for attending all the requests sent by this client.

Sockets

Sockets is a method of communication between a client program and a server program in a network. A socket is defined as “the endpoint in a connection”. 1)
To create one you first need to choose an address family, the most common is IPv4, python supports IPv6 too. There are other way to indicate an address like UNIX family. We will use the AF_INET (IPv4) family address, which requires a pair of IP address (like 100.30.200.25) and a port. The low number ports are for specific services, for example, the port 80 is used for http comunication. If you are using a port for a non standard service it's a good a idea to choose a four digit port number, like 10123.
Second you have to choose a transport protocol. The most used are Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). The last one is a simple transmission model, used in very simple communications and is exposed to any unreliability. The first is one of the original protocol of IP suite and provides reliable, ordered, error-checked delivery of a stream, we will used this one.

To create a new socket you first need to import the socket module. Later we will need to import the os module too:

import socket

Then we create an object socket using the AF_INET for IPv4 and SOCK_STREAM for TCP.

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Next we need to indicate to which pair of address and port are we going to work in. This variable will be used to connect to the right computer. If you are creating the server and the client in the same computer (yes, it's possible!) you can use 'localhost' instead of the IP address.

sock_address = ('localhost', 10123)

Another example:

sock_address = ('100.10.200.25', 10123)

From now on the code of the client and the server are different. The client's code is shorter and simpler, we will look a it first.

Writing a network client

Right now, you should have a code very similar to this:

import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 10100)

The first instruction will be to connect to the server:

client.connect(server_address)

Now, if there really is a server online, a connections has been established. The next step will be to create a loop to receive data, in this case from the keyboard, and send it to the server.

while True:
	message = raw_input()	
	client.sendall(message)	

Now you have a network client, this is the final code:

#! /usr/bin/env python
 
import socket
 
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('192.168.1.103', 10100)
print "socket created"
client.connect(server_address)
print "connection established, ready to send data"
 
exit = False
while not exit:
	message = raw_input()	 
	client.sendall(message)	
	if message == "exit":
		exit = True

Writing a network server

We have the same to lines of code:

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 10100)

But in this case we don't want to make a connection, instead we configure the socket to listen for connection requests:

server.bind(server_address)
server.listen(1)

The next line blocks the program, waiting for a connection request. Once the connections has been established it creates new socket object: client_socket. This is a normal socket, but only communicates with the client socket in the other side. We will put this instruction inside a loop, so it will be accepting new connections for ever

client_socket, client_address = server.accept()

And here comes the magic. After creating the connection we will do a fork2). This means that the operating system will create a new process for this connection and at the same time will assign a new port, automatically.
Before the code you need a little of theory. Every process has something call PID, Process IDentifier. It is a number used for the kernel to temporarily uniquely identify a process 3). When we ask the kernel to fork our process it return a relative PID, the parent has a PID assigned at the moment we ran the program, but the child process will have a zero (0). We will use this to difference between the child and the parent process.
In the code below we fork the process, if the PID is equal to zero, then we stop the loop and print a notification message. Otherwise it is the parent process and we should keep listening for new connections. The os._exit(1) is needed to close the process without calling up the python cleanup handlers.

wait_connection = True
while wait_connection:  
  client_socket, client_address = server.accept()  
  newproc = os.fork()  
  if newproc == 0:	
    wait_connection = False    
    print 'A connection has been established'
    os._exit(1)

Now you have a network server, this is the final code:

#! /usr/bin/env python
 
import socket, os
 
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 10100)
server.bind(server_address)
server.listen(1)
print "server created and configured"
 
wait_connection = True
while wait_connection:	
  print "Online and waiting for new requests"	
  client_socket, client_address = server.accept()  
  newproc = os.fork()  
  if newproc == 0:	
    wait_connection = False     
    print 'A connection has been established'
    os._exit(1)

Doing something useful

At this moment the only thing our server is able to do is to print a message every time a client request a new connection. The final goal is to use the data sent by the client in a process inside the server.

First we need to receive and save the message sent by the client. For this we wait for an incoming message. The argument of the next function indicates the maximum amount of data to be received.

msg = client_socket.recv(4096)

As perhaps you noticed , the communication between server and client is made through strings of characters. This means that every message should contain the functions to be executed and all the needed information. This implies the design of a communication protocol.

The simplest way to make understandable for the computer information sent by client is to split every part of the message with a very uncommon character, like <>. To achieve this we could use the string.split(<>) python function and saving the data in a new list:

instruction = []
instruction = msg.split('<>')   

A simple task could be to print a message in the server screen. for this we create a function call printf, then the client has to send message like this: printf<>hello!!. So the instruction will be printf and the information hello!!.

Possibly you want to do something more complex, so it's a good idea to create a decide function, to choose a specific function between all implemented functions in you server.

Lets implement the printf function in our server. It is a good idea to create a module for the function, so the server and the functions code will be no mixed in the same file. In this file, the decide function will receive the client socket, and inside a loop it will wait for an incoming message. Once it receive the new message, it will read the instruction, which is in the position [0] of the instruction list. Then will call the right function.

def decide(client_socket):	
	while True:
		msg = client_socket.recv(4096)		
		instruction = []		
		instruction = msg.split('<>')
		command = instruction[0]			
		if command == 'printf':
			printf(instruction)									

def printf(msg_list):
	print msg_list[1]

Remember to import decide from the module functions in the server code:

from functions import decide

And that's all. Now first run the server and then the client, write something in the client terminal, like printf<>Salute from the client! and it will appear in the server terminal. Here is the functions code:

#! /usr/bin/env python
 
import socket
 
def decide(client_socket):	
	while True:
		msg = client_socket.recv(4096)		
		instruction = []		
		instruction = msg.split('<>')
		command = instruction[0]			
		if command == 'printf':
			printf(instruction)									
 
def printf(msg_list):
	print msg_list[1]

Running the programs

If you are running the server and the client in the same machine you can keep the server address as 'localhost', otherwise you have to change the address to the right IP. To discover the address of the computer you are going to use as server open a terminal and type:

sudo ifconfig

The answer will be similar to this:

eth0      Link encap:Ethernet  HWaddr 00:26:9e:10:17:c0  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:45 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

wlan0     Link encap:Ethernet  HWaddr 00:25:56:18:15:c9  
          inet addr:192.168.1.103  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::225:56ff:fe18:15c9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:12821 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13473 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:7776475 (7.4 MiB)  TX bytes:4296056 (4.0 MiB)

If you have a wire connection then look in eth0 area, if your connection is wireless check up wlan0. In this example the computer is a laptop and the address is 192.168.1.103. This is the address assigned by the router. So, this is not the real world wide internet address of the computer. If you want computers outside the local network connecting to your server you have to configure the router to redirect the messages to the right port.

Next you have to change the execution permission of the files. For this go to the directory in which you have the files and type:

sudo chmod +x server.py
sudo chmod +x client.py

For running the programs just open two terminals, one for each one, and type:

in the "server" terminal
./server.py

in the "client" terminal
./client.py

You better run the server first or the client won't find the server and the program will end with an error.

Back to project
UP

2)
This instructions works on gnu/linux and possible any other unix-like operating system.
teaching/ie0117/proyectos/2013/i/way_to_shop/multi-process_server.txt · Last modified: 2022/09/20 00:08 (external edit)