CSAIL Web server architecture

There are five main CSAIL Web servers: courses, groups, people, projects, and www. In addition, there are a number of special-purpose Web servers, among them legacy, mit-only, redirects, and webstats. As www is the Lab's official face on the Web, it has a different configuration that will not be described here. All the others run Apache 2.2 on 64-bit CSAIL Debian GNU/Linux.

These servers share a common configuration layout, based on the standard layout for the Debian Apache 2.2 package. They are all also implemented as virtual machines, so that the service may easily be transferred from one physical server to another as resource requirements and hardware reliability dictate. In addition, all of the servers except mit-only and webstats serve multiple virtual hosts. The functions of the servers break down as follows:

Server Description
courses academic courses which have storage in /afs/csail/proj/courses/6.xxx; supports both CSAIL and MIT certificates for authentication
groups research groups which have storage in /afs/csail/group/*
people Lab member and alumni home pages and anything else which is served from a user home directory, such as ~/public_html
projects anything which has storage in /afs/csail/proj/* except academic courses
legacy old, unmaintained sites belonging to former groups and discontinued projects; scripting restricted for security; includes www.ai.mit.edu
mit-only projects which, for administrative reasons, must use MIT and not CSAIL certificates for authentication
redirects specialized configuration which only serves redirects; used to remap discontinued URI namespace to current servers, and for active virtual hosts which have a large number of aliases
webstats Web server logs

Languages and tools available on each web server

The courses, groups, people, projects, and mit-only servers all belong to the configuration class WEB_SERVER_FEATUREFUL and should have nearly-identical configuration including support for PHP5, FastCGI, and popular Web and database framework packages for Perl, PHP, Python, and Ruby. The other servers generally do not support server-side scripting beyond the standard mechanisms included with the Perl, Python, and Ruby packages in Debian.


  • upload size limits
    • HTTPS - 16M. This is a hard limit involving many points in the path from your browser, SSL negotiation, and Apache. PHP and MediaWiki have similar limits. [Occasionally the SSL renegotiation gods may smile upon you and let a larger upload through, but don't let that fool you.]
    • HTTP - 128M (or lower). Various components (such as those listed above) may cap your effective file size limit much lower.
  • script running time, memory, and temp files
    • Generally, be nice to your labmates. If your webpage or script is causing a major performance or security problem, TIG will be forced to take it offline temporarily.
    • Don't use web scripts for heavy computation. If you're not sure, or want tips on how to make a script less resource intensive, please ask help@csail.mit.edu
    • Make sure that your scripts either tell automated search engine robots to stay away, or handle multiple requests for every possible URL (including ?query=string;component=words) faster than the robots can follow new URLs.

Logs and debugging

To debug errors in your web scripts, please investigate what accesses and errors are displayed in the Web server's error logs. Specifically, any time your browser shows "500 Internal Server Error", this means that a more informative error message has been written to the server's error log.

-- GarrettWollman - 26 Sep 2007
Topic revision: 09 Jul 2012, ArthurProkosch

MIT Computer Science and Artificial Intelligence Laboratory


  • About CSAIL
  • Research
  • News + Events
  • Resources
  • People

This site is powered by Foswiki MIT: Massachusetts Institute of Technology