Project 2 (Web Proxy Server)

Due Date

Wednesday, October 04, 2017 @ 11:59pm

A Note on Collaboration

You are NOT permitted to work together on this programming assignment.
Each student must submit their own work.
If you use core ideas from anywhere or anyone (besides textbook or instructor), you must acknowledge this in your report.
You may NOT copy code from anywhere or anyone!
(Except the textbook (e.g., Section 2.7.2) and Linux Howtos: C/C++ -> Sockets Tutorial .)

Introduction

In this assignment you are asked to build a Web proxy server that is capable of delivering Web content on behalf of a remote Web server. When a user's browser is configured to connect via a proxy server, her browser establishes a connection to the proxy server's machine and forwards its complete request to the proxy server rather than the "real" Web server. The proxy server accepts user's HTTP requests and forwards them to their final destinations---essentially it introduces an extra "hop" between the user's browser (or the client) and the Web server. There are several reasons why people want to use a proxy server instead of connecting to the remote Web server directly: 1) users want to anonymize themselves from the Web server; 2) a corporation might want to monitor or restrict employees' Web surfing; 3) a proxy can be used to locally cache data in order to reduce the amount of global traffic. Serving multiple clients simultaneously, while not critical to the functionality of a Web proxy server, is critical to its performance. Therefore, allowing the server to process multiple simultaneous requests in parallel is a must-have feature for a practical server.

The Details

The proxy server acts as an HTTP server to client browsers while it acts as an HTTP client to the "real" Web server. The protocol specification for version 1.1 of HTTP is defined in RFC 2616. While this document is quite long, you are asked to implement a very small part of the complete protocol, as explained below. In particular, your server only needs to understand the http: scheme; if the request contains any other scheme (example, file:, ftp:, etc.), it should be rejected. We do try to be as specific as possible in this assignment description in order to reduce your need to refer to the lengthy protocol specification, but you should be prepared to do so occasionally.

Upon startup, your proxy server should read a configuration file (specified on the command line), open a TCP socket on a port indicated in the configuration file, make a list of the blocked websites, and start accepting requests. An HTTP-compliant Web server supports HEAD, POST, and GET methods. Your proxy server only needs to support the GET method. In addition, we require that you support one non-standard method for ease of grading: QUIT. When you get a QUIT request, your server should exit immediately. To serve each request, we first need to parse the request line and headers sent by the client. Since we will support only the GET method, the relevant information is the Web page name in the request line. A request for the proxy server may look like this:
     GET http://www.foo.com/mumbo.html HTTP/1.1
     Connection: close
     ... ...
        
The requested Web page name contains the Web server name www.foo.com and the requested file on that server /mumbo.html. In this case, your proxy server should refer the list containing the blocked websites, and if not blocked, it should make a TCP connection to Web server www.foo.com at the default port 80 (or the port specified in the URL, for example, port 1234 if the URL is www.foo.com:1234/mumbo.html ) and ask for file /mumbo.html by sending the following request:
    GET /mumbo.html HTTP/1.1\r\nConnection: close\r\n\r\n

After sending a request to the "real" Web server, an HTTP response including the requested file will be received at the proxy server. The proxy server should then forward the content to the client. There are three parts in an HTTP response message: the status line, the response headers, and the entity body. The status line and response headers are terminated by an empty line (or an extra "\r\n"). In short, an HTTP response message may look like the following:

     HTTP/1.1 200 OK
     Date: Thu, 24 Jan 2013 15:29:52 GMT
     Content-Type: text/html
     Server: Joe's Web proxy server
     Connection: close
    
     ... ... <content of the main.html>
          
The status line and most of the header fields should be forwarded to the client without modification.

The server should be able to handle multiple simultaneous service requests in parallel. There are three common ways to handle muliple connections concurrently: (i) multiple threads, where the server listens at a fixed port on the main thread. When it receives a TCP connection request, it sets up a TCP connection socket and services the request in a separate thread; (ii) multiple processes, forking a child process for each connection instead; and (iii) using the select (2) system call to perform asynchronous operations.

The "Connection: close" setting in the above examples serves an important role---disabling the persistent connection feature in HTTP 1.1. A persistent connection allows multiple Web requests to flow through a single TCP connection. However, the support for persistent connections is somewhat challenging and it is not required for this assignment. Note that when you test your proxy server through a standard Web browser, the "Connection: close" setting will likely not be in the HTTP request. Your proxy server can insert this setting into the request before forwarding it to the Web server so that the persistent connection is turned off.

The Configuration File

A configuration file consists of a sequence of lines containing comments or commands. Comments are initiated by a pound sign (#) and extended to the end of the line. There are two clauses that can appear in a configuration file:
  • port n: If present, this clause specifies the TCP port number your server should listen on. If omitted, the default http port (80) should be used.
  • block pattern: This clause specifies that all URLS matching pattern are blocked - an attempt to access them should return an error message.
Sample configuration file:
   # proxy_config --- sample HTTP proxy configuration file
   #
   # This clause defines the port where proxy should listen on.
   port 8080

   # A list of blocked URLS goes here
   block www.nytimes.com
   block www.facebook.com

   

Program Skeleton

Here we provide a hint on what the program skeleton may look like. You are, however, not required to design your program this way.

     Main routine {
         Parse the configuration file; 
         Create a server socket listening on the specified port;
         For each incoming client socket connection {
             Spawn a worker thread / fork off a child process / use select to handle the connection;
             Wait for/handle the next connection;
         }
     }
    
     Handling any connection {
         Read the request line and header fields until two consecutive new lines;
         (Note that a new line can be a single "\n" or a character pair "\r\n".)
         Examine the first line (request line);
         If the request method is not "GET" {
             Return an error HTTP response with the status code "HTTP_BAD_METHOD";
         }
         If the requested site is marked as block in the configuration file {
             Return error 403 (Forbidden)
         }
         Make TCP connection to the "real" Web server;
         Send over an HTTP request;
         Receive the server's response;
         Close the TCP connection to the server;
         Send the server's response back to the client;
         Close the connection socket to the client.
     }
      

Testing

If you develop your program on your home computer, it is imperative to test it on the department network before turn-in. Below we provide some information about how to test your program.

For testing, you can configure a Web browser to use your proxy server. For instance, you can configure Firefox in the following way: 1) Click "Preferences"'; 2) Choose "Advanced" Tab; 3) Choose "Network" Sub-Tab; 4) Click "Settings" for Connection; 5) Click "Manual proxy configuration" and fill the server name (or address) as well as its port number for "HTTP Proxy". Make sure your proxy server is running and now all HTTP requests from your browser are served via the proxy server!

You can also test your server using the telnet utility. If your proxy server runs at port 8080 of cycle1.csug.rochester.edu, you can do

     [yourself@foobar] telnet cycle1.csug.rochester.edu 8080<enter>
     GET http://www.google.com HTTP/1.1<enter>
     Connection: close<enter>
     <enter>
     ... ... <response from your server, including status line, header, and the requested file>
     [yourself@foobar]
      
You can also try this directly with a Web server by specifying the default port number 80. For instance, you can do
     [yourself@foobar] telnet www.google.com 80<enter>
     GET / HTTP/1.1<enter>
     Connection: close<enter>
     <enter>
     ... ...
     [yourself@foobar]
      

Persistent Connections (extra credit)

While we recommended that you disable HTTP 1.1's persistent connection for an easier implementation of the proxy server, you can choose to add this support and we may reward you with some extra credit. We are not going to provide much information about how to do this. If you elect to take up this challenge, you are expected to solve the problems that arise on your own. One such problem, for instance, is how to determine the end of a response message in a persistent connection. This is very easy for non-persistent connections (with Connection: close setting)---a response message ends when the server closes the TCP connection.

Submission

You need to submit a tar file containing your source files, a makefile, and a README file. You are allowed to use only C/C++ for this project. your program should take one parameter (the name of the configuration file) on startup. You should also name your executable ProxyServer. You should be able to launch your server using "ProxyServer <configuraton file>".

The README file should be in plain text format, or be a PDF generated from your document source. It should contain a description of your design, and explain what is and what is not realized in your implementation. A makefile is strongly recommended. You may also specify the build command in the README file. Name your directory project1. Tar the directory and submit it on BlackBoard by 11:59 pm on Oct 4.

 
   tar -cvf project2.tar project2/*
   
Multiple submissions are allowed, but only the last one will be graded.

Grading Guideline

  • 50%: properly forward Web pages from remote servers to browsers with error handling.
  • 20%: supports multiple client connections simultaneously.
  • 20%: properly support blocking with appropriate error codes;
  • 10%: a clear README file, clarity of your source code, and completeness of your comments.
  • 20% extra credit: support persistent HTTP connections. Please explain your implementation in the README file.