CSAIL Web server architecture
There are five main CSAIL Web servers: courses, groups, people,
projects, and www. In addition, there are a number of
special-purpose Web servers, among them legacy, mit-only,
redirects, and webstats. As www is the Lab’s official face on the
Web, it has a different configuration that will not be described here.
All the others run Apache 2.2 on 64-bit CSAIL Debian GNU/Linux.
These servers share a common configuration layout, based on the standard
layout for the Debian Apache 2.2 package. They are all also implemented
as virtual machines, so that the service may easily be transferred from
one physical server to another as resource requirements and hardware
reliability dictate. In addition, all of the servers except mit-only
and webstats serve multiple virtual hosts. The functions of the
servers break down as follows:
| Server | Description |
|---|---|
courses |
academic courses which have storage in =/afs/csail/proj/courses/6.xxx=; supports both CSAIL OIDC and MIT certificates for authentication |
groups |
research groups which have storage in /afs/csail/group/* |
people |
Lab member and alumni home pages and anything else which is served from a user home directory, such as ~/public_html |
projects |
anything which has storage in /afs/csail/proj/* except academic courses |
legacy |
old, unmaintained sites belonging to former groups and discontinued projects; scripting restricted for security; includes www.ai.mit.edu |
mit-only |
projects which, for administrative reasons, must use MIT and not CSAIL OIDC for authentication |
redirects |
specialized configuration which only serves redirects; used to remap discontinued URI namespace to current servers, and for active virtual hosts which have a large number of aliases |
webstats |
Web server logs |
There’s also a data server available for distributing data sets from NFS.
Languages and tools available on each web server
The courses, groups, people, projects, and mit-only servers
all belong to the configuration class WEB_SERVER_FEATUREFUL and should
have nearly-identical configuration including support for PHP5, FastCGI,
and popular Web and database framework packages for Perl, PHP, Python,
and Ruby. The other servers generally do not support server-side
scripting beyond the standard mechanisms included with the Perl, Python,
and Ruby packages in Debian.
Limits
- Upload size limits
- HTTPS - 16M. This is a hard limit involving many points in the path from your browser, SSL negotiation, and Apache. (Occasionally the SSL renegotiation gods may smile upon you and let a larger upload through, but don’t let that fool you.)
- HTTP - 128M (or lower). Various components (such as those listed above) may cap your effective file size limit much lower.
- Script running time, memory, and temp files
- Generally, be nice to your labmates. If your webpage or script is causing a major performance or security problem, TIG will be forced to take it offline temporarily.
- Don’t use web scripts for heavy computation. If you’re not sure, or want tips on how to make a script less resource intensive, please ask help@csail.mit.edu
- Make sure that your scripts either tell automated search engine robots to
stay away, or handle multiple requests for every possible URL
(including
?query=string;component=words) faster than the robots can follow new URLs.
Logs and debugging
To debug errors in your web scripts, please investigate what accesses and errors are displayed in the Web server’s error logs. Specifically, any time your browser shows “500 Internal Server Error”, this means that a more informative error message has been written to the server’s error log.