ThreadCraft: Building a Proxy Server

Lets just launch right into it. What's a proxy server? A proxy server is an intermediate server that acts as a gateway between the user and the target server. It is employed for various purposes:

• Improve security
• Secure internet activity
• Balance traffic to target
• Rate limit traffic
• Save bandwidth by caching

And in this case, we'll be applying a specific rule to priorities client requests to target.

High Level Overview

We've illustrated the working of our proxy server below. The key features of our server are:

1. Multi-Threaded with thread safety
2. Uses producer(listener)-consumer(worker) semantics - a well known construct in concurrency
3. There is a thread-safe priority queue (heap) which serves as the central structure for storing, organizing and delivering requests.
4. Every producer has a port on which it listens for incoming requests.
5. On receiving a request, it typically adds it to the priority queue. The only exception being the /GetJobs path for which it will simply pop the request from the queue and return it to the client.
6. On the other side, consumers pull requests on the priority queue and send it to our target server. The target server sends an appropriate response back to the consumer which forwards it to the correct client.

That's all there is really. The devil however, is in the details.

The Producer (Listener Thread)

While we won't dive into the details (the flowchart is self-explanatory!), we do wish to draw attention that we do a global lock on the ENTIRE priority queue and send a signal when we've filled it successfully so that we can wake one of our consumers. Although a better approach could exist with respect to locking a specific segment, given that the priority queue will eventually have to organize ALL its data, it wouldn't be worth attempting such an endeavor.

The Consumer (Worker Thread)

Consumer is much simpler as the flow chart illustrates. Again we'll leave it to you to figure out the finer details.

Serve Forever

Finally, we discuss the serve_forever function of our proxy server. To begin with, we initialize our priority queue. Next, we spin up num_listener number of producers using pthread_create and passing the list of port and server file descriptors followed by spinning up num_worker number of consumer threads. We now proceed to join (pthread_join) the producer threads thus created followed by the consumer threads.

Conclusion

We now have a multi-threaded proxy server that can serve requests based on a priority rule we defined. Clearly we can extend this concept to perform numerous other functions such as rate-limiting, user authentication, load balancing, etc. We have a much more robust system on our hands now!