CM10228 / Programming Ib:   Lecture 11


How to Network


-I  Testing, Scaling & Iterative Design 2

  1. What was a reasonable plan for getting the AI bot working?
    1. Make one that wanders randomly, AND tries to exit if it stands on the E.
      1. Walk through why this would work.  Brownian motion.
    2. Improve that by looking & not walking into walls.
    3. Improve that by trying to search systematically or remember where the E was & how much gold you have.
  2. What is a reasonable plan for testing the AI?
    1. Hints:
      1. title of this subsection
      2. the answer is NOT doing all three steps above & *then* testing.
  3. Everyone should be able to get good marks on the test plan!  Just think about it first!
    1. Same about commenting.  Where do comments go?
      1. every method & class gets commented
      2. any tricky algorithms get commented
    2. Still, there is also the idea of satisficing
      1. Need to decide what is worth your time.
      2. Want to use anytime algorithms in your life as much as possible
        1. have some solution available whenever the time runs out,
        2. the more time you get the better

I.  Buffering

  1. A buffer is a piece of memory that holds a local copy of something that you really want to read or write to somewhere else.
  2. The reason you have them is so that you don't waste too much time waiting on slow processes, like writing or reading from a disk or network.
  3. You have probably seen a little bar representing a buffer fill up when you run a video from the Internet, e.g. on quicktime. 
    1. For quicktime, when the buffer seems to be full enough that you can probably watch to the end of the movie, it lets you start seeing the movie.
    2. But if the network gets unexpectly worse / slower, then the movie will stop & you will have to wait until the buffer refills.
  4. Because writing is often buffered, if you want to make sure that what you have written has gone to the screen, you have to give a command to flush the buffer (see code below).
    1. This is also a big issue if you add printlines to debug a program.  Be sure to flush them right away, or the program may crash before the message you wanted to print get sent to the screen, even if the printline was executed succesfully.

II. Clients for the date & echo services:

  1. Note most of the below doesn't work these days, because services have been turned off.
    1. This is due to security concerns.
    2. I've left the below in for you to see anyway as example clients, it's just that you aren't going to get to run them & try it yourself.
    3. Unless you download (or write) a service server yourself and run it on your own machine!
  2. In order to make clients and servers more clear to you, here are some examples based on standard services.
  3. The example servers are standard internet services, which are provided by at least unix / linux computers.
    1. Standard services have standard port numbers.
    2. On unix machines (including mac OS X) you can find these in /etc/services
    3. There is also an internet site detailing all standard port-number assignments.
    4. From that page:
      The port numbers are divided into three ranges: the Well Known Ports,
      the Registered Ports, and the Dynamic and/or Private Ports.

      The Well Known Ports are those from 0 through 1023.

      DCCP Well Known ports SHOULD NOT be used without IANA registration.
      The registration procedure is defined in [RFC4340], Section 19.9.

      The Registered Ports are those from 1024 through 49151

      DCCP Registered ports SHOULD NOT be used without IANA registration.
      The registration procedure is defined in [RFC4340], Section 19.9.

      The Dynamic and/or Private Ports are those from 49152 through 65535
    5. So when you pick a port for your own application, pick one bigger than 49152!
  4. The example client is a program called telnet, which is a generic client which can connect to any service. 
    1. The thing is, you have to type in the side of the protocol that the server expects.
    2. That's because telnet (being generic) won't know about it.
    3. So you are kind of being a fake client, and telnet is just helping you out.
  5. However, for security reasons, many of these services are by default turned off these days.
    1. The below (in purple) is for hackers only!!
    2. If you do this stuff & your box gets hacked, don't blame me!  You need to undo it again before you plug back into the internet!
    3. If you have installed linux on your own computer, you might want to turn these on & play with them. 
      1. However, note that you should probably turn them back off again before plugging your computer back into the internet if you do!
      2. Note that this has to be your own linux box, because you need superuser (root) access to do this.
        1.  If you don't know what this means don't worry -- there are other examples below that don't require this, that you can run, like the linelength server.
        2. But do still have a look at what the servers do, so you can understand the code of what a client does that talks to it.
        3. I demonstrate how to do this below in the main (non purple) lecture notes.
    4. Mac OS X used to be almost linux.  the below worked on 10.3, but I left my 10.3 laptop in Maastsricht so couldn't demo this for you, sorry.
      1. To get this working there, I had to edit /etc/xinetd.d/{daytime,echo} to set disable from yes to no.
      2. Then I had to restart xinetd:  Vigor11:/etc root# service xinetd restart
        No such service xinetd
      3. Too linux / too easy!
      4. ps -auxw | egrep xine
        root       285   0.0  0.0    27484     44  ??  Ss    5Feb07   0:00.01 xinetd -inetd_compat -pidfile /var/run/xinetd.pid
        Vigor11:~ root# kill -HUP 285
        Vigor11:~ root# ps -auxw | egrep xine
        root       285   0.0  0.0    27484    300  ??  Ss    5Feb07   0:00.04 xinetd -inetd_compat -pidfile /var/run/xinetd.pid
        root      7034   0.0  0.0    18644    208 std  R+    8:56PM   0:00.00 egrep xine
        Vigor11:~ root# kill -KILL 285
        Vigor11:~ root# ps -auxw | egrep xine
        root      7036   0.0  0.0    18644    208 std  R+    8:56PM   0:00.00 egrep xine
        Vigor11:~ root#
        Vigor11:~ root# /System/Library/StartupItems/IPServices/IPServices start
        Starting internet services
        Vigor11:~ root# ps -auxw | egrep xine
        root      7040   0.0  0.1    27484    704  ??  Ss    8:57PM   0:00.02 xinetd -inetd_compat -pidfile /var/run/xinetd.pid
        root      7042   0.0  0.0    18644    208 std  R+    8:57PM   0:00.00 egrep xine
        Vigor11:~ root#
      5. You should never really have to kill -KILL anything... it means "don't clean up your children, just DIE".
        1. As with last week's discussion of death, the unix utility "kill" just sends a signal, and the first argument is which signal it sends.  The second is the process ID.

    5. Since there are a lot of different actors in these examples I'm using colour coding.
      1. Sorry if you are colour blind.  You can try looking at the html source to see which colours I used!
      2. Green is me.
      3. Blue is the telnet program.
      4. Reds & oranges are the servers (there are different ones.)
      5. Black is the unix shell.  (the operating system)
    6. Here I am turning the services on & talking to them on for my linux box:
      [root@sydney etc]# telnet localhost 7
      Trying 127.0.0.1...
      telnet: connect to address 127.0.0.1: Connection refused
      [root@sydney etc]# /sbin/chkconfig daytime on
      [root@sydney etc]# telnet localhost 15
      Trying 127.0.0.1...
      telnet: connect to address 127.0.0.1: Connection refused
      [root@sydney etc]# less services # this let me see what the different services were & find the right number
      [root@sydney etc]# telnet localhost 13
      Trying 127.0.0.1...
      Connected to localhost.
      Escape character is '^]'.
      17 MAR 2004 21:50:15 GMT
      Connection closed by foreign host.
      [root@sydney etc]# less services
      [root@sydney etc]# /sbin/chkconfig echo on
      [root@sydney etc]# telnet localhost 7
      Trying 127.0.0.1...
      Connected to localhost.
      Escape character is '^]'.
      hi
      hi
      golf
      golf
      ^]
  6. Notice a couple of things here:
    1. I've tapped into the server with a program called "telnet".  
      1. Telnet basically just lets you talk to ports directly. 
      2. It used to be used to login for sessions, but it's too insecure.  Nowadays we use ssh instead.
      3. I'll be showing you how you can use telnet to fake internet / application protocols in a couple weeks (assuming *that* hasn't been shut down).
    2. Mostly because of security, but also because in class I won't be networked, I'm talking to my own machine as if it's a machine on the internet.
      1. Your own machine can be accessed with the special IP address 127.0.0.1
      2. it's DNS name is "localhost" (as well as any other name you've given it.)
      3. Again, I'll probably explain DNS & how the internet routes packets in two weeks.
    3. I happen to be in the /etc directory, so the file that tells you the right port number for things is just "services".
    4. I have to be root to turn the services on.
  7. The first server, daytime (port 13), kicked me out after giving me the information it gives.
    1. I didn't have to type anything at all.
    2. very simple protocol.
  8. The second server, echo (port 7) responded to what I typed. 
    1. But there's no way to quit it!
    2. To quit I had to talk to telnet directly, so I had to type the telnet escape command, ^].
  9. Assuming you can find a computer that will let you talk to its daytime server, then you can run this program (which is taken with only a slight modification from Object-Oriented Programming with Java by David Barnes, Chapter 19 (on Networking!):
    import java.net.*;
    import java.io.*;

    // Contact the daytime server running on hostname.
    public class DaytimeClient {
    // Contact the server at the appropriate port and set
    // the socket attribute for the run() method to use.
    public DaytimeClient(String hostname) throws Exception {
    // The well-known port of the TCP daytime service.
    final int DaytimePort = 13;
    try{
    socket = new Socket(hostname,DaytimePort);
    }
    catch(UnknownHostException e){
    throw new Exception("Unknown host: "+e.getMessage());
    }
    catch(IOException e){
    throw new Exception("IOException on socket creation: "+e.getMessage());
    }
    }

    // Obtain the time of day from the daytime server.
    public void run() {
    Socket daytime = getSocket();
    try{
    // Get the stream used by the server to send data.
    InputStream inStream = daytime.getInputStream();
    // Wrap the stream in a BufferedReader.
    BufferedReader reader = new BufferedReader(
    new InputStreamReader(inStream));

    // The server will only send a single line.
    String response = reader.readLine();
    System.out.println(response);
    // Close the connection now it is finished with.
    daytime.close();
    }
    catch(IOException e){
    System.err.println(e.getMessage());
    }
    }

    protected Socket getSocket(){
    return socket;
    }

    private final Socket socket;

    public static void main(String[] args) {
    if(args.length == 1){
    try{
    DaytimeClient client = new DaytimeClient(args[0]);
    client.run();
    }
    catch(Exception e){
    System.err.println("Exception: "+e.getMessage());
    }
    }
    else{
    System.err.println("You must supply the name of a host to contact.");
    }
    }
    }

  10. Here's one that talks to echo...

    /* author david barnes & joanna bryson
    *
    */
    import java.net.*;
    import java.io.*;


    // Contact the echo server running on hostname.
    class TCPEchoClient {
    // Contact the server at the appropriate port and set
    // the socket attribute for the run() method to use.
    public TCPEchoClient(String hostname) throws Exception {
    try{
    // The well-known port of the TCP echo service.
    final int EchoPort = 7;
    socket = new Socket(hostname,EchoPort);
    }
    catch(UnknownHostException e){
    throw new Exception("Unknown host: "+e.getMessage());
    }
    catch(IOException e){
    throw new Exception("IOException on socket creation: "
    +e.getMessage());
    }
    }

    protected Socket getSocket(){
    return socket;
    }

    private final Socket socket;


    public static void main(String[] args) {

    try {
    TCPEchoClient eClient = new TCPEchoClient("localhost");

    // now you know stdin is really a socket too!
    BufferedReader stdin =
    new BufferedReader(new InputStreamReader(System.in));
    String myEchoString;

    System.out.print("Welcome to the echo client. The prompt looks like >>."
    + "\n >> ");

    // Wrap the input stream in a BufferedReader.
    BufferedReader reader = new BufferedReader(
    new InputStreamReader(eClient.getSocket().getInputStream()));
    // Wrap the output stream in a BufferedWriter.
    BufferedWriter writer = new BufferedWriter(
    new OutputStreamWriter(eClient.getSocket().getOutputStream()));

    // read a string from stdin & send it to echo
    while ((myEchoString=stdin.readLine()) != null) {
    System.out.println("Sending: "+myEchoString);

    writer.write(myEchoString);
    writer.newLine();
    // Make sure the data is flushed to the stream.
    writer.flush();

    // Read the response.
    String response = reader.readLine();
    System.out.println("The response is: "+response);
    System.out.print("\n >> ");
    } // while strings
    // Close the connection
    eClient.getSocket().close();
    }
    catch(IOException ioe){
    ioe.printStackTrace();
    }
    catch(Exception e) {
    e.printStackTrace();
    }

    } // main()

    } // class TCPEchoClient
  11. Sorry I didn't color those creatively, but if you look at them in emacs (or eclipse) it should color them for you.
    1. If your BUCS unix server emacs isn't making pretty colors for you, you should put this in your .emacs file
      ; stick colour in old versions of emacs --

      ;; Are we running XEmacs or Emacs?
      (defvar running-xemacs (string-match "XEmacs\\|Lucid" emacs-version))

      ;; Turn on font-lock mode for Emacs
      (cond ((not running-xemacs)
      (global-font-lock-mode t)
      ))

      ; -- ( colour stuff from Adam Dziedzic ) --
    2. Notice:  emacs is written in lisp!
    3. You might want to look at ~cssjjb/.emacs for more random stuff

III. Servers

  1. There are a couple differences with servers:
    1. They have to start running first.
    2. They need an extra port -- one to listen for clients.
  2. But once a server has made the connection to the client, it's really just the same.
  3. Here is yet more code from David Barnes -- this time I left the main as he wrote it (in another class) though I really don't see the advantage... (so no comment claiming I'm a coauthor!)
  4. The server class
    import java.net.*;
    import java.io.*;

    // A simple server that accepts a client connection
    // and sends back the length of the strings it
    // receives from the client.
    class LineLengthServer {
    public LineLengthServer(int port) throws Exception {
    try{
    // Listen on the given port.
    serverSocket = new ServerSocket(port);
    }
    catch(BindException e){
    throw new Exception("Failed to create a server socket: "+
    e.getMessage());
    }
    }

    // Read strings and return their length (as a String)
    // to the client.
    public void run() throws Exception {
    ServerSocket serverSocket = getServerSocket();
    Socket clientSocket = null;
    try{
    System.out.println("Listening for a client on port: "+
    serverSocket.getLocalPort());
    // Wait for a client to make contact.
    clientSocket = serverSocket.accept();
    // Contact ...
    System.out.println("A client has arrived.");

    // Wrap the input stream in a BufferedReader.
    BufferedReader reader = new BufferedReader(
    new InputStreamReader(clientSocket.getInputStream()));
    // Wrap the output stream in a BufferedWriter.
    BufferedWriter writer = new BufferedWriter(
    new OutputStreamWriter(clientSocket.getOutputStream()));

    // Read lines until the client terminates.
    String request = reader.readLine();
    while(request != null){
    // Write the length of the line as a String.
    writer.write(String.valueOf(request.length()));
    writer.newLine();
    writer.flush();
    request = reader.readLine();
    }
    }
    catch(IOException e){
    throw new Exception("IOException talking to the client: "+
    e.getMessage());
    }
    finally{
    if(clientSocket != null){
    System.out.println("The client has gone.");
    // Close the socket to the client.
    clientSocket.close();
    }
    }
    serverSocket.close();
    }

    protected ServerSocket getServerSocket(){
    return serverSocket;
    }

    // The socket on which the listening is done.
    private final ServerSocket serverSocket;
    }
  5. The "main" class
            // An example of a simple server that reads lines from
    // clients and sends back the length of each line
    // it receives.
    public class LineLengthServerMain {
    public static void main(String[] args){
    try{
    // An arbitrary port number to listen on. This should be
    // larger than 1024 for user-written servers.
    final int port = 24101;
    LineLengthServer server = new LineLengthServer(port);
    server.run();
    }
    catch(Exception e){
    System.err.println("Exception: "+e.getMessage());
    }
    }
    }

  6. Running it from linux
    [joanna@sydney Networks]$ javac LineLengthServerMain.java
    [joanna@sydney Networks]$ java LineLengthServer
    Exception in thread "main" java.lang.NoSuchMethodError: main
    [joanna@sydney Networks]$ java LineLengthServerMain
    Listening for a client on port: 24101
    ^Z
    [2]+ Stopped java LineLengthServerMain
    [joanna@sydney Networks]$ bg
    [2]+ java LineLengthServerMain &
    [joanna@sydney Networks]$ telnet localhost 24101
    Trying 127.0.0.1...
    A client has arrived.
    Connected to localhost.
    Escape character is '^]'.
    hi there
    8
    have you ever thought
    21
    this isn't really measuring the line length...
    46
    Ah ha!
    6
    ^]

    telnet> .
    ?Invalid command
    telnet> quit
    Connection closed.
    The client has gone.
    [2]+ Done java LineLengthServerMain
    [joanna@sydney Networks]$
    1. Notice I used telnet as a client again (see colour code mentioned earlier).
    2. And also, I suspended the server process (with ^Z) and then backgrounded it by typing bg
      1. This made it start running run, but leaving me STDIN to use with telnet.
      2. You can also type fg and get right back where you were.
      3. You could also start the server in the background in the first place by typing an & afterwards like this:  j2bryson$ java LineLengthServerMain &
      4. Note both programs now shared STDOUT!
    3. Consequently, both programs were printing to the screen at the same time -- telent I made blue, the server red, and what I typed green.
    4. I also forgot the telnet protocol / commands -- I thought you killed it with a .
      1. mail quites with a .
      2. I got it right the second guess though.
      3. Don't be afraid to hack!
  7. If you want to make a server accept more than one connection, just put a while loop in so it looks again (see the outlined server back in section I.)
  8. If you want to have the server accept new clients while it's still servicing old ones, you need to use threads!
    1. You need this for coursework two.
  9. Here's an example of that, again from Barnes (hey, he writes good examples! This is from RandomServiceListener)
        public void listen() throws Exception {
    ServerSocket serverSocket = getServerSocket();
    // Wait up to 30 minutes for new clients.
    final int timeToWait = 1*60*1000;
    boolean keepWaiting = true;

    while(keepWaiting){
    try{
    // Wait for a client to make contact.
    serverSocket.setSoTimeout(timeToWait);
    Socket clientSocket = serverSocket.accept();
    // Let a separate Thread handle it.
    new Thread(new RandomService(clientSocket)).start();
    }
    catch(InterruptedIOException e){
    // We timed out waiting for a client.
    keepWaiting = false;
    }
    catch(IOException e){
    throw new Exception("IOException: "+ e.getMessage());
    }
    }
    // No more clients, so close the socket.
    serverSocket.close();
    }
    1. Notice not only the threading, but the fact it handles interupts, so it can eventually be killed!
    2. It also sets a timeout so that it will effectively interrupt itself if it's been waiting too long (as well as accepting terminate signals, e.g. from the keyboard.)

IV.  Finding out what your local machine's address is:

  1. Another useful program potentially for your coursework, again from David Barnes (the bit that matters is in blue):
    // Print out Internet address information about the local machine.
    import java.net.*;

    public class PrintInetDetails {
    public static void main(String[] args){
    if(args.length == 0){
    try{
    InetAddress myDetails = InetAddress.getLocalHost();
    System.out.println("The local host is called: "+
    myDetails.getHostName()+
    " and its address is: "+
    myDetails.getHostAddress());
    }
    catch(UnknownHostException e){
    System.err.println("Unknown host: "+e.getMessage());
    }
    }
    else{
    for(int i = 0; i < args.length; i++){
    try{
    InetAddress details = InetAddress.getByName(args[i]);
    System.out.println("The host is called: "+
    details.getHostName()+
    " and its address is: "+
    details.getHostAddress());
    }
    catch(UnknownHostException e){
    System.err.println("Unknown host: "+e.getMessage());
    }
    }
    }
    }
    }

  2. When/if I show it to you running on my laptop, it only says lamely the standard localhost address, but here's what it did on my desktop when I first wrote these lecture notes (alas, ai.mit.edu is no more!):
    [jjb@jjb Networks]$ java PrintInetDetails
    The local host is called: jjb.cs.bath.ac.uk and its address is: 172.16.33.1
    [jjb@jjb Networks]$ java PrintInetDetails horsepower.ai.mit.edu soggy-fibers.ai.mit.edu
    The host is called: horsepower.ai.mit.edu and its address is: 128.52.37.26
    The host is called: soggy-fibers.ai.mit.edu and its address is: 128.52.32.78
    [jjb@jjb Networks]$ java PrintInetDetails wjh.harvard.edu www.google.com
    The host is called: wjh.harvard.edu and its address is: 140.247.94.249
    The host is called: www.google.com and its address is: 66.102.11.99
  3. Notice that the two MIT AI lab addresses are very similar -- again, more on that in the protocols lecture.

V. Summary

  1. Mostly a lot of examples of clients & servers
  2. A little bit of a feel for how to use unix/linux, and
  3. how telnet can be used to fake one side of a client.

page author: Joanna Bryson
13 March 2012