Math 581d: October 13, 2010 Reminder: turn on screencast recorder. Reminder of general goal of this class: 1. Understand how math software (mainly sage) is very useful for research math. 2. Be able to understand Sage *deeply*. REMARK: It is impossible to understand Magma, Maple, Mathematica or Matlab deeply in the sense I mean above, because they are closed source. (Unless, of course you work for one of those companies; closed source means only they have the power that comes with deep understanding.) Example of 1: computation in live math research from my inbox today: https://mail.google.com/mail/?shva=1#mbox/12b966bbb7abab3b Rest of today will address 2. TODAY: Python -- the center of sage Language of Sage is (almost) Python. Assuming you have some familiarity with Python, give an overview of the main important features of the language, and resources for learning more. In case you know nothing about Python, it's a popular free open source language with no company pushing it (like say Sun/Oracle and Java or Microsoft and .NET), though many many companies use it heavily, and fund its development. From python.org right now: "Python is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python and see almost immediate gains in productivity and lower maintenance costs." * "Work more quickly": means that you get stuff done instead of fighting with the language and environment for silly reasons * "Integrate your systems": means Python is particular useful at creating big systems out of possibly messy collection of software tools. * "Maintenance costs": Python code is more likely to be readable and hackable. Topics: * How Python is used in Sage - Watch out: the Sage preparser - The Sage library - Python components of Sage distribution Go to sage-4.5.3/local/lib/python/site-packages/ and type "ls"; interesting packages include: - pycrypto -- cryptosystems - cython -- python compiler - ipython -- interactive shell - jinja2 -- templating - moinmoin -- a wiki - PIL -- python imaging library (a "programmable photoshop") - pygments -- html highlighting - sqlalchemy -- abstracts interface to most SQL databases - sphinx -- ReST documentation system for python - twisted -- network framework; everything from web apps to email to ssh servers - ZODB -- Zope Object database - arpack -- some sparse linear algebra? - cvxopt -- convex optimization - delaunay -- ??? (maybe installed by scipy?) - docutils -- related to python documentation - easy-install.sh -- you can do easy_install foobar to install most python packages foobar - see http://pypi.python.org/pypi/ - gd.py -- very quickly draw png images with lines, arcs, etc. - jabberbot -- (part of moinmoin) - matplotlib -- canonical python 2d graphics library - mpmath -- floating point math functions - numpy -- numerical python n-dimensional arrays - pexpect -- control command-line subprocesses - polybori -- ?! not sure what this does here?! - pylab -- matlab clone interface for matplotlib - rpy2 -- fast interface to R - sage -- the Sage library itself (huge) - sagenb -- the Sage notebook - sagetex -- allows you to embed Sage in latex documents - scipy -- canonical numerical computing library for python - setuptools -- package for distributing and working with standalone python packages - sympy -- symbolic calculus library - weave -- embed C/C++ code directly in Python - zope.interfaces, etc. -- dependency I think twisted needs; specifies api of class * Resources for learning Python *deeply* - I think the best reference on Python is: Python in a Nutshell: http://oreilly.com/catalog/9780596001889 - Best "jump in" books: Python tutorial: http://docs.python.org/tutorial/ Dive into Python: http://diveintopython.org/ - Official Python docs (http://docs.python.org/): - Python C/API guide: http://docs.python.org/c-api/index.html The Python C/API book is absolutely critical to become familiar with if you want to understand Python deeply!! - (massive) Standard Library and module guide: http://docs.python.org/library/index.html - etc. - The Source code of Python itself - Download, extract, and look around: http://python.org/download/ - LICENSE - README, especially the section at the end "Distribution structure" Main stuff listed there: Doc/ Documentation sources (reStructuredText) - for first few years of Sage, Doc/ was a big *Latex* document, with latex2html (non-GPL-compatible license!) and crazy *perl* scripts needed to build the docs. Now things are much, much better. Sphinx and ReST. Learn much more here: http://sphinx.pocoo.org/intro.html Many Python projects have switched to using Sphinx. Grammar/ Input for the parser generator Parser/ The parser and tokenizer and their input handling Python/ The byte-compiler and interpreter - if you are a CS major, you'll get very excited looking at the files in these directory. Include/ Public header files - these are important when you are writing code that embeds python, or uses the Python C/API guide (see above), e.g., possibly when using Cython. They basically define how you can think of Python itself as a "C program". For example, look at "Include/object.h" to see what a Python object really is. I've seen repeatedly (at Sage Days, etc.) that the people who are able to look at this sort of thing are much more powerful programmers. Obviously, you have to know C to fully read object.h, though the big comment at the top is in English. Learn C! (Note: Python does not use C++ at all.) Lib/ Python library modules - Python has a slogan: "batteries included" This is where some of the batteries are. E.g., Look at say Lib/csv.py. Ever wonder how Python parses csv files: you can easily find out! This directory contains a large amount of *good quality* working Python code, which is worth looking at. Ever wonder how Python does "regular expressions"? Check out Lib/re.py and the files it imports, such as Lib/sre_parse.py. Modules/ Implementation of most built-in modules - More batteries are here. These are the parets that aren't implemented in Python. E.g., look at Modules/_json.c which is a bunch of C code, written against the Python/C API, which implements code needed for fast parsing of JSON format data (Javascript Object Notation). There is corresponding code in Lib/json/ that provides a nice Python-level interface to this. Objects/ Implementation of most built-in object types - The core of Python itself, e.g., where things like ints, floats, dictionaries, lists, sets, etc., are implemented. Has to be very fast. E.g., take a look at dictobject.c, which is an important and well documented module. Much of python is built on top of an optimized implementation of dictionaries. configure Configuration shell script (GNU autoconf output) - Yes, Python just uses a standard "./configure; makez" autoconf build system. NOW: - build Python from source live! - modify something, rebuild, and see the change. Yes, we are in control. * Next -- main Python *programming* features (explain each with an example): - functions: def foo(arg1, arg2, arg3='hello'): ... return ... def foo(*args, **kwds): .... - modules directories, __init__.py - classes class Integer(RingElement): def __init__(self, x): .... - decorators @foo def f(n): return n*n - list comprehension and generator expressions [a*a for a in range(10)] (a*a for a in range(10)) * Python 3 versus Python 2...