Web server cluster logs

Web server cluster logs

Web server logs for the CSAIL Web cluster are available to Lab members by logging in to the host tig-web-syslog. (That host has local home directories rather than using AFS home directories, so you won’t have your normal shell config files availabe

The latest web cluster logs (access and error logs combined) are on that host in /var/log/remote-syslog.log. That file collects logs of all virtual domains served by TIG’s web server cluster, which means there’s a lot to wade through. Your best bet is probably to grep for a partial URL or for the domain name (“virtual host” in Apache terms) you’re interested in. For instance, if you’re debugging problems with a CGI conference-registration script at https://myconference.csail.mit.edu/register.cgi you could grep for all or part of that URL with a command like

grep myconference.csail /var/log/remote-syslog.log

That might not show resulting errors, but at least it will show you the timestamps of all the accesses to that URL. If you wanted to show the five following lines after each match, you could use something like

grep -A5 myconference.csail.mit.edu/register.cgi /var/log/remote-syslog.log

(Besides custom domains, you can also search for accesses to the standard shared static webserver domain names; for instance, a command like

grep people.csail.mit.edu/myusername

would find accesses to your personal web site, and a command like

grep groups.csail.mit.edu/mygroup/about.html

might find accesses to your research group’s “about” page (assuming you don’t have your own custom domain defined – most research groups do).

We recognize that accessing web cluster logs in this way is a little clumsy and we hope to improve this facility in the future.