This chapter, which was written by Gonzalo Tornaria (and typeset by David Joyner), explains how to use the revision control system used by the SAGE developers. If you are a SAGE user not interested in becoming a SAGE developer then you will have no need to worry about this.
Homepage: <http://darcs.net/>
The ``official" manual: <http://darcs.net/manual/>
The wiki (``the community-maintained site for all things darcs"):
<http://darcs.net/DarcsWiki>
If you are debian based try:
$ sudo apt-get install darcs
<http://darcs.net/DarcsWiki/CategoryBinaries>
for links to binaries and instructions for different systems (Windows,
Mac OS X, several Linux, BSD, Solaris, AIX).
No need! If you have a ``darcs" binary that works, you are all set.
Try darcs --version
to see which version you have.
The man page is nice, but you don't really need it; darcs
alone will
give you a list of the available commands, and darcs COMMAND --help
will give you detailed help on a particular command.
For more help, check the manual or the wiki.
$ sage -upgrade [...]
$ cd $SAGE_ROOT/devel/ $ darcs get http://modular.math.washington.edu/sage/dist/src/sage-darcs/ Copying patch 65 of 65... done! Applying patch 65 of 65... done. Finished getting.
sage-darcs
directory
you just got to $SAGE_ROOT/devel/sage
$ ln -snf sage-darcs sage
sage-darcs/install
executable:
$ chmod +x sage-darcs/install
$ sage -br [...] sage:
You can change whatever you want in this ``repo" you have now in
$SAGE_ROOT/devel/sage-darcs
Moreover, if you record your changes as you work, you can selectively rollback, unrecord, or even get rid of some of your changes.
Let's try an example:
$ cd $SAGE_ROOT/devel/sage-darcs
print "hello world"
./sage/all.py
$ darcs what -ls M ./sage/all.py +1
$ darcs what { hunk ./sage/all.py 232 +print "hello world" }
$ darcs diff -u diff -rN -u -u old-sage-darcs/sage/all.py new-sage-darcs/sage/all.py --- old-sage-darcs/sage/all.py 2006-03-11 19:43:42.002107800 -0500 +++ new-sage-darcs/sage/all.py 2006-03-11 19:43:42.290064024 -0500 @@ -229,3 +229,4 @@ from IPython.iplib import InteractiveShell InteractiveShell.exit = _quit_sage_ +print "hello world"
What is going on? Quoting from the manual:
In a single DARCS repository you can think of two ``versions" of
the source tree. They are called the working and pristine trees.
Working is your normal source tree, with or without DARCS
alongside. The only thing that makes it part of a DARCS repository
is the _darcs
directory in its root. Pristine is the recorded
state of the source tree. The pristine tree is constructed from
groups of changes, called patches (some other version control
systems use the term changeset instead of patch).5.1 Darcs will
create and store these patches based on the changes you make in
working.
What you have just changed is your ``working" tree. The ``pristine" tree has not changed, and the diff above is the difference going from ``pristine" to ``working".
Keep in mind that the revision control is done on the ``pristine" tree, so if you do a zillion changes to your ``working" tree they will get all confused (you can still undo them one by one, but they are a zillion, and you won't remember which changes go together!).
When you have a nice set of changes that you might want to preserve, you should ``record" them. This updates the ``pristine" tree with your changes, in the form of a new patch.
$ darcs record [...] What is your email address? Gonzalo Tornaria <tornaria@math.utexas.edu> hunk ./sage/all.py 232 +print "hello world" Shall I record this patch? (1/?) [ynWsfqadjkc], or ? for help: y What is the patch name? test patch -- hello world Do you want to add a long comment? [yn] n Finished recording patch 'test patch -- hello world'
$ darcs what -ls No changes!
$ darcs changes -p hello Sat Mar 11 20:01:46 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * test patch -- hello world
$ darcs changes [... a LOT of changes! ...]
$ darcs changes --from-tag sage Sat Mar 11 20:01:46 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * test patch -- hello world Sat Mar 11 05:06:20 EST 2006 William Stein <wstein@gmail.com> * change factorial to use PARI C libraries "n!"; (40 times faster) Fri Mar 10 17:10:49 EST 2006 William Stein <wstein@math.washington.edu> * fix to matrices -- det only cached if immutable Fri Mar 10 01:24:37 EST 2006 William Stein <wstein@math.washington.edu> tagged sage-1.1.0
There are two commands related to building: ``sage -b" rebuilds this tree, while ``sage -br" rebuilds and runs the freshly built version.
$ sage -br [...] hello world sage:
To interchange patches, rather than using a P2P network, DARCS lets
you create a text file called a ``patch bundle", which contains patches
together with some context information. The patch bundles can then be
sent by whatever means (typically email, but you can print the bundle,
use a pigeon carrier, and have the bundle OCR'd at the other end, cf.
<http://www.faqs.org/rfcs/rfc2549.html>
for some fun):
$ darcs send -o hello.darcs Creating patch to "http://modular.math.washington.edu/sage/dist/src/sage-darcs/"... Sat Mar 11 20:01:46 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * test patch -- hello world Shall I send this patch? (1/1) [ynWvpxqadjk], or ? for help: y
Notice that ``Creating patch to ..." remembers and uses the repo from where we pulled last time as a base. Indeed, when you choose a patch to send, DARCS will bundle all the depending patches that are not already in the base.
Your patch bundle will be good for anybody who is up-to-date with the same base as you; thus you should base your patch bundles in public repositories, and preferably the main SAGE repo, so that anybody can pull missing dependencies.
[aside:
If your system has a running sendmail configured to send mail to the outer world, you can send a patch directly from SAGE (if you don't, you might want to try ``msmtp", which is a pass-through from your system to some smtp server of your choice). If so:
$ darcs send Creating patch to "http://modular.math.washington.edu/sage/dist/src/sage-darcs/"... Sat Mar 11 20:01:46 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * test patch -- hello world Shall I send this patch? (1/1) [ynWvpxqadjk], or ? for help: y What is the target email address? wstein@math.washington.edu Successfully sent patch bundle to: wstein@math.washington.edu
Enough of SAGE greeting the world every time! Let's get rid of that patch:
$ darcs unpull -p hello Sat Mar 11 20:01:46 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * test patch -- hello world Shall I unpull this patch? (1/?) [ynWvpxqadjkc], or ? for help: y Finished unpulling.
Remember to ``sage -b" to rebuild SAGE.
Beware: unpull will eliminate the patch from both ``working" and
``pristine", meaning that the patch will be gone. This is completely
safe with patches that you have pulled from a stable repo, because you
can always pull them again. If you create a patch in your local repo,
don't send it to anyone, and then unpull it, your patch is gone
forever (actually, if you made a mistake, you might be able to recover
from the file in _darcs/patches
, which is not deleted, but this is
tricky).
In our case, we saved a patch bundle, so that we can (and will) recover the patch from the patch bundle.
The recipient of the patch bundle can apply it with ``darcs apply"; since we unpulled the patch, we can play the role of recipient:
$ darcs apply hello.darcs Finished applying...
--repodir=$SAGE_ROOT/devel/sage-darcs
.
Remark: applying a patch bundle is ``safe" in the sense that darcs will refuse to apply a patch that would produce a conflict.
If you apply a patch bundle that someone sent you, and you decide you don't want it, you just ``unpull" it as above.
The above works because we made a symlink to the working repo from
$SAGE_ROOT/devel/sage
, which is the default for building SAGE.
Be warned that if you upgrade with ``sage -upgrade" to a new version of
SAGE, the symlink will be ``stolen", and (annoyingly) the repo will be
moved to $SAGE_ROOT/devel/old/sage-darcs
. You'll have to move your
repo back and redo the symlink if you want to use that.
Not upgrading is not an option. Even if you stay up-to-date via the DARCS repo, eventually you will need to upgrade some other package.
As an alternative, here is what I normally do:
$ sage -update [...]
$ cd $SAGE_ROOT/spkg/standard $ ls sage-* sage-1.1.0.spkg
$ cd $SAGE_ROOT/spkg/installed $ mv sage-* sage-1.1.0
$ sage -update-build [...]
As SAGE development goes on, new patches will be added to the main repo. Presumably this will happen more often than releases, so that you can track development and be in the bleeding edge.
$ darcs pull Pulling from "http://modular.math.washington.edu/sage/dist/src/sage-darcs/"... No remote changes to pull in!
$ darcs unpull -p factorial [... interaction ...]
$ darcs pull Pulling from "http://modular.math.washington.edu/sage/dist/src/sage-darcs/"... Sat Mar 11 05:06:20 EST 2006 William Stein <wstein@math.washington.edu> * change factorial to use PARI C libraries "n!"; (40 times faster) Shall I pull this patch? (1/1) [ynWvpxqadjk], or ? for help: y Finished pulling and applying.
If you see that there are a lot of patches, you don't care what they are and want to stop the interaction and pull them all, type ``a", or invoke pull with the ``-a" option as in ``darcs pull -a"
Remember that upgrading the SAGE library this way may require upgrading some of the other packages in the SAGE distribution. You definitely need to upgrade the SAGE distribution every once in a while (see discussion above).
WARNING: pulling changes can produce conflicts. Although this is perfectly safe for your recorded changes, this might be unsafe for unrecorded changes!
As a rule of thumb, you should never pull with unrecorded changes. You don't have to organize your changes before pulling, you might just record a temporary patch with something like
$ darcs record -lam "TEMPORARY patch"
$ darcs unrecord -p "TEMPORARY patch"
If you have some custom patches in your repo, when you pull from the main repo you may find some conflicts. This means that your patches modify the same line as some of the patches you are pulling from the main repo.
If a conflict occurs, DARCS explains that there was a conflict in
certain files, and the relevant source files are *still* changed with
text inserted pointed to the conflicts (look for v v v
and
^ ^ ^
).
You then edit the source file to explain what you really want.
When you have a resolution for the conflict, you record a new patch; this new patch will depend on the two conflicting patches, and so long as the merge patch is pulled together with the conflict, no complain will raise.
Again: if two patches A and B have a conflict, and you pull both together, the conflict will be raised. You solve the conflict, and record a patch M which merges A and B.
Now say another person has a repo including patch A. If this person pulls patch B alone, there will be a conflict. But if patches B and M are pulled together, no conflict will arise!
If you add new files in your working repository, the commands won't see them by default. Some commands have a ``-l" option for this, e.g. (always in the repo dir)
$ touch newfile $ darcs what -s No changes! $ darcs what -ls a ./hello.darcs a ./newfile
$ darcs record No changes! $ darcs record -l addfile ./hello.darcs Shall I record this patch? (1/?) [ynWsfqadjkc], or ? for help: q Record cancelled.
Alternatively, you can be explicit about newfiles with:
$ darcs add newfile $ darcs what -s A ./newfile $ darcs what -ls A ./newfile a ./hello.darcs
Warning: this doesn't add the file to the ``pristine" repository, it just schedules for addition, this will need to be recorded as any other patch!)
We can explicitly remove files from revision control:
$ darcs remove newfile $ ls newfile newfile
Remove should be called when you want to remove a file from your project, but don't actually want to delete the file. Otherwise just delete the file or directory, and DARCS will notice that it has been removed. Be aware that the file WILL be deleted from any other copy of the repo to which you later apply the patch.(from ``darcs remove -help'')
Indeed:
$ rm -f new $ darcs what -s R ./new
Life saving device:
$ darcs revert hunk ./new 1 -darcs whatsnew |less Shall I revert this patch? (1/?) [ynWsfqadjkc], or ? for help: y Do you really want to revert these changes? y Finished reverting. $ ls new new $ darcs what -s No changes!
Unlike remove, this needs to be done with DARCS, so that revision control knows the file was renamed (this will allow DARCS to commute patches past the rename!)
$ darcs mv notes.txt notes.text $ ls notes.* notes.text $ darcs what -s ./notes.txt -> ./notes.text $ darcs record -am "moving notes.txt to notes.text" Finished recording patch 'moving notes.txt to notes.text' echo add some text >> notes.text $ darcs what -s M ./notes.text +1 $ darcs record -am "add some text to notes.text" Finished recording patch 'add some text to notes.text'
$ darcs send -o addtext.darcs -p add Creating patch to "http://modular.math.washington.edu/sage/dist/src/sage-darcs/"... Sat Mar 11 22:27:40 EST 2006 Gonzalo Tornaria <tornaria@math.utexas.edu> * add some text to notes.text Shall I send this patch? (1/?) [ynWvpxqadjkc], or ? for help: y
$ darcs unpull -p notes.text [... interaction ...] $ ls notes.* notes.txt $ cat notes.txt darcs diff --from-patch "mpz_pylong: add notices"
$ darcs apply addtext.darcs Finished applying... $ cat notes.txt darcs diff --from-patch "mpz_pylong: add notices" add some text
There is a configuration directory for DARCS, where you can override
some defaults. In linux this is in ~/.darcs
. Some examples:
~/.darcs/author
:
in this file you put your email address; then you won't be asked
about it anymore. Otherwise you will be asked once for each new
repository.
~/.darcs/defaults
:
in this file you put default options for DARCS commands, for
example:
diff diff-opts -up
--diff-opts=-up
" is assumed as a default.
Another example, if you have ``msmtp" installed and configured:
send sendmail-command msmtp %t %<
~/.darcs/boring
:
in this file you put your local version of what files to ignore (for -l), For example, you could add a line like:
\.darcs$
Before:
$ darcs what -ls a ./addtext.darcs a ./hello.darcs
$ darcs what -ls No changes!
Use the ``--mark-conflicts
'' option for this.
As above, you should NOT have
any unrecorded changes in your repo. You then resolve the conflicts as
above.
Alternatively, if you are the main repo maintainer, you can bounce the patch bundle to its author, and ask him to pull the main repo and merge the patch. It should be easier for the patch author to understand how to fix the conflicts.
this reverts ``working" to the state of ``pristine" (i.e. to the last recorded version of the tree).
you have *one* chance to undo a revert, but if you pull, unpull, apply, record, unrecord, etc., your chance is gone.
this is the inverse of record. Like unpull, it takes a patch out of your ``pristine" tree, but you KEEP the changes in your ``working" tree. Presumably you will change things a bit and record another patch.
this is like doing ``darcs unrecord" followed by ``darcs record". You don't get a chance to change the patch name, description, or dependencies, but the timestamp is changed (so the you obtain a new patch which has nothing to do with the old one).
this applies an ``UNDO" patch to your ``pristine" tree, but keeps the changes in your ``working" tree. In a sense, this is the same as ``darcs unrecord", but the patch stays in the repo, so you can go back to the patch some other time...
if you messed up while resolving a conflict, this will mark the conflicts again. BEWARE: you will loose all unrecorded changes!!
_darcs
included). This makes you a public repository and others can pull
patches from there! (as easy as 1-2-3)
Quote from the manual:
Keep in mind that the patches are your history, so deleting them with unrecord makes it impossible to track what changes you really made. Redoing the patches is how you ``cover the tracks''. On the other hand, it can be a very convenient way to manage and organize changes while you try them out in your private repository. When all is ready for shipping, the changes can be reorganized in what seems as useful and impressive patches. Use it with care.
All patches are global, so don't ever replace an already ``shipped'' patch in this way! If an erroneous patch is deleted and replaced with a better one, you have to replace it in all repositories that have a copy of it. This may not be feasible, unless it's all private repositories. If other developers have already made patches or tags in their repositories that depend on the old patch, things will get complicated.
In short: once you start distributing a patch, you should not change it anymore. Instead of using amend-record, you should record a NEW patch that amends the old one. Otherwise, you'll start to have headaches. For instance, if someone bases some patches in your old patch, and you base some patches in your new patch, you'll have a hard time merging...
In particular, once a patch goes into the main repo, it should never go out for any reason.
A developer can have several trees to avoid problems with this, e.g.:
[ It might be possible to unify the last two if testing for SAGE can be implemented. Indeed, it is possible to have DARCS run a test before approving a patch that is applied with ``darcs apply". If the test fails, the patch bundle is bounced together with the output of the test.
This would be really nice; one could send (signed) patches by email to ``darcs apply" (e.g via procmail), and have them be tested in the background, and only applied if all tests pass... that's peace of mind :-)
The difficulty is that SAGE is not self-contained; there isn't right now an easy way to build/run SAGE from a temporary directory without touching the whole SAGE environment.
IF we had some way to ``build and run, and run the tests" from an
alternate directory, and without installing anything to $SAGE_ROOT
,
this would be possible (and very cool).
]
See About this document... for information on suggesting changes.