TWiki > TIG > SoftwareTopics > UsingSubversionAtCSAIL

Using Subversion at CSAIL

What is Subversion?

According to the Subversion home page, it is "a compelling replacement for CVS." Subversion is a version control system that was built with CVS users in mind, but it addresses many of the limitations and annoyances of CVS. It allows you to manage versioning on directories, for example, where CVS does not. Subversion has module-based support for integration with Apache2, and can leverage its authentication and access control models to allow people who don't have local user accounts to collaborate with local users.

The topic of version control is outside the scope of this document, as is the nitty-gritty underpinnings of Subversion. For more info on these topics, please read the SVN Book.

How do I install Subversion?

To do anything useful with Subversion, you need to have the subversion package installed on your workstation.
  • Subversion is installed by default on all CSAIL Debian workstations.
  • For Windows, TortoiseSVN is recommended, which integrates into Windows Explorer (so that checkout, update, and similar operations are accessible by right-clicking a file or folder).
  • Mac OS provides subversion via XCode / Developer Tools; it should already be in /usr/bin/svn. We recommend new users use SCPlugin which integrates into the Mac OS Finder. ZigVersion is also available (a standalone application).
  • Ubuntu 8.10 and 9.04 provide svn 1.5 through the subversion package (may need to be manually selected).
  • Other OSes: see http://subversion.tigris.org/getting.html. We recommend version 1.5 where possible (1.6 must be handled carefully to prevent locking out 1.5 users.)

How do I create a Subversion repository at CSAIL, populate it, and check out a working copy?

1. Determine where to put your repositories.

All repositories should be created in the appropriate group, project, or personal directory in the CSAIL AFS cell. If your repository is for use across your entire research group, we suggest that you create a directory called REPOS in the top-level of your group's directory, e.g., /afs/csail.mit.edu/group/tig/REPOS. If your repository is for more personal projects, keeping it in REPOS in your home directory makes sense.

2. Create the empty repository structure.

In the following example, we will create an empty repository at /afs/csail.mit.edu/group/tig/REPOS/test_repo. This is done on svn.csail.mit.edu to ensure that the respository is created using the correct svn version. 

csailuser@somehost:~$ ssh svn.csail.mit.edu
[...]
csailuser@svn:~$ cd /afs/csail/group/tig
csailuser@svn:/afs/csail/group/tig$ mkdir -p REPOS

  • Verify that AFS permissions are set correctly using fs la REPOS now. Update them as necessary. If you skip this step, you will need to recursively set permissions on your repository directory and all its subdirectories later. 

csailuser@svn:/afs/csail/group/tig$ cd REPOS/
csailuser@svn:/afs/csail/group/tig/REPOS$ ls
csailuser@svn:/afs/csail/group/tig/REPOS$ svnadmin create --fs-type=fsfs ./test_repo
csailuser@svn:/afs/csail/group/tig/REPOS$ ls
test_repo
csailuser@svn:/afs/csail/group/tig/REPOS$ cd test_repo/
csailuser@svn:/afs/csail/group/tig/REPOS/test_repo$ ls
README.txt  conf  dav  db  format  hooks  locks
csailuser@svn:/afs/csail/group/tig/REPOS/test_repo$

To ensure performance and reliability of your repository, you must:

  • use the --fstype=fsfs argument to svnadmin
  • access your repository using svn+ssh://svn.csail.mit.edu/afs/csail/... syntax, or using https://
  • only use direct file:// access on CSAIL Debian systems that have an identical version of svn as svn.csail.mit.edu does (1.5.1 as of this writing; this works as of August 2009 but is not guaranteed).

3. Import stuff into the repository.

Let's say you have an existing codebase tree in your home directory, in a subdirectory called project_x. You will need to do an initial import of your code into Subversion's format. The following example shows us importing that code base into the newly created repository. 

csailuser@somehost:~/Work$ ls
project_x
csailuser@somehost:~/Work$ svn import ./project_x/ svn+ssh://svn.csail.mit.edu/afs/csail/group/tig/REPOS/test_repo/ --message "initial import of project_x"
Adding         project_x/foo
Adding         project_x/bar
Adding         project_x/pub
Adding         project_x/pub/beer
Adding         project_x/baz

Committed revision 1.
csailuser@somehost:~/Work$

The svn import command above tells Subversion to take the contents of your local directory "project_x" and import it into the new, empty repository. Note that the path to the repository is a svn+ssh URL. This directs your local svn client to first ssh to svn.csail.mit.edu, and then use its copy of svn to safely perform the modifications.

4. Make sure you can checkout a copy of the repository.

Once you have created the repository, you will likely never deal directly with it again. All modifications to the repository are done by way of edits you make to local copies (called "working copies"). To get a working copy, you just need to check one out. The following example shows how to do that. 

csailuser@somehost:~$ mkdir checkouts
csailuser@somehost:~$ cd checkouts
csailuser@somehost:~/checkouts$ ls
csailuser@somehost:~/checkouts$ svn checkout svn+ssh://svn.csail.mit.edu/afs/csail.mit.edu/group/tig/REPOS/test_repo
A  test_repo/foo
A  test_repo/bar
A  test_repo/pub
A  test_repo/pub/beer
A  test_repo/baz
Checked out revision 1.
csailuser@somehost:~/checkouts$

We now have a local working copy of the repository, and we can share the URL of the repository with our group mates so they can also check out a copy and begin working.

How would I do the same steps on Mac OS or Ubuntu?

The above steps assume that the local machine "somehost" is running CSAIL Debian, but very few modifications are necessary for Mac OS or Ubuntu. Ensure that:
  • Kerberos is installed properly (Mac, Ubuntu) and that you have valid Kerberos tickets
  • ssh is configured to forward Kerberos credentials (bottom of the appropriate Kerberos page above)
  • Your local username is the same as your CSAIL username. If you must keep them distinct, you can change ssh svn.csail.mit.edu to ssh yourcsailusername@svn.csail.mit.edu, and change svn+ssh://svn.csail.mit.edu/... to svn+ssh://yourcsailusername@svn.csail.mit.edu/..., but you'll lose some flexibility.

How would I do the same steps on Windows?

Ensure that: Then, perform step 2 via PuTTY, and step 3 via TortoiseSVN.

How do I change a working copy and update the repository?

There's no particular magic to editing a working copy; just use your favorite editor and hack away. When you're done, you will want to commit your changes. in the following example, we've edited the file pub/beer, and want to push the change to the repository. 

csailuser@somehost:~/checkouts/test_repo/pub$ svn commit -m "added more beer"
Sending        pub/beer
Transmitting file data .
Committed revision 2.
csailuser@somehost:~/checkouts/test_repo/pub$

How do I make sure I have the latest version of files in my working copy?

Use svn status -u to show all edits (local and remote) that are pending, or just svn update to refresh the local copy from the repository. 

csailuser@somehost:~/checkouts/test_repo$ svn update
U  bar
Updated to revision 3.

Hmmm, someone else went to the bar. Now I have an updated version of that file, which has version number 3.

How do I do (whatever) in Subversion?

That's not really what this document is for, but the SVN Book will probably have the answers you are looking for.

If not, and you are a member of the CSAIL community, please join the CSAIL svn-users mailing list. If you don't know how to join a CSAIL mailing list, ask one of us or one of your group mates.

The rest of this document is only related to issues that are particular to our implementation here at CSAIL and will probably be of little use to anyone else.

Other access methods

So far, we've only seen how to use Subversion with the svn+ssh: protocol, which works for collaborators who all have CSAIL accounts. However, this isn't always adequate, especially if you need to collaborate with people outside of CSAIL. There are four different ways to access a Subversion repository:

  • Via the file:/// "protocol"; (can cause race conditions and svn client version incompabilities)
  • Via the svn proprietary client/server protocol; (does not allow sane access control)
  • Via a combination of the SVN program and an SSH tunnel, called svn+ssh; (as seen above)
  • Via Apache2 DAV, using the https:// protocol.

Setting up https access

If you need to collaborate with someone who does not have a CSAIL account, you will need to use the https access method for Subversion, which makes the repository accessable as a WebDAV share. Once configured properly, your repository will be accessible from a url of the form https://svn.csail.mit.edu/coolNameGoesHere Unlike the other two access methods, this gives you the ability to, and requires that you, manage your own usernames and passwords. These exist in a simple flat file which is made with the htpasswd command. See AuthUserFile for more information. You also have the ability to define fairly complex and finely-grained access control based on the directory in the repository. See the Subversion Book for more information.

Once you've created your htpasswd file (and also an access control file, if so desired) you must give the SVN WebDAV? server access to your repository. Change to the directory containing your repository and run 

find . -type d -exec fs sa {} svn rlidwk \;

Note: even if the svn user already has appropriate access permissions, please make sure that your effective afs permissions to the repository directory (either through an AFS group or directly) are all (rlidwka). If they are not, the final step (web script creating the mapping) will fail.

Make sure that the htpasswd and optional access control files are in a location in AFS accessible by svn but not necessarily by other users.

Finally, to set up and enable the SVN WebDAV mapping, go to https://svn.csail.mit.edu:1443/admin/admin.cgi. Fill in your desired location and the path to your SVN repository. Type the path to your htpasswd and/or access control files in the field labeled "Password File", "ACL File". Be sure to click "Use Authentication" if you want the server to check usernames and passwords! Once done, click commit and then try out your repository.

Important note about SVN repositories on network filesystems

Older versions of Subversion used BerkeleyDB as their back-end storage mechanism. Newer versions (1.6) use SQLite. Neither mixes well with network filesystems like AFS or NFS, and their use on these filesystems could result in data loss. We strongly recommend that you convert any repositories with the BerkeleyDB backend to the FSFS backend, and that you avoid creating repositories using client versions >= 1.6. You can find out which backend you're using by examining the db/fs-type file in your repository. If it contains the string bdb, you're using the BerkeleyDB format. Instructions for converting a bdb repository to fsfs are given in the following example:

  • correct any errors in place 
svnadmin recover /afs/csail.mit.edu/group/rvsn/papers

  • dump all svn actions to a log 
svnadmin dump /afs/csail.mit.edu/group/rvsn/papers > svn.dump

  • move existing repos out of the way 
cd /afs/csail.mit.edu/group/rvsn/
mv papers papers.bdb

  • recreate repos; default type is FSFS 
svnadmin create papers

  • replay the log 
svnadmin load /afs/csail.mit.edu/group/rvsn/papers < svn.dump

  • if everything worked 
rm svn.dump
rm -rf papers.bdb