CSCI 435 S07 Compiler Design


Python Syntax    PLY

4-20-2007 We're coming down to the wire in Compiler Design -- today I'm handing out some guidance for Presentation Day to help make your Presentations more effective.  With only one week to go, it is important to make the progress necessary to be successful.  All three teams seem to need a little catching up to get done in a fully satisfactory way.  So plan to put in some extra hours to make it happen.  There will be no separate After Action Report day since May 2nd turns out to be Study Day.

4-2-2007 RUBY, INFORM, and LOGO Teams presented brief status reports today.  Next major review will be the Wednesday after Easter Break.  The minor review scheduled for Wednesday April 4th is cancelled in favor of getting more work done on the projects.  Teams are encouraged to get together and work during that time on their Lexer/Parser tasks.

3-30-2007 Well we better be finished with GRAMMAR because the clock is running.  Monday is April 2nd and May 2nd is the last day of class when your Projects should be complete and your Presentations Given.  May 2nd will be the After Action Report Meeting.  Reports today went well.  Read Josh's when he submitted by email.








3-27-2007 Monday's Meeting (not surprisingly) showed relatively little progress between Friday and Monday.  I was disappointed that the teams had not rising to the direction of producing significant Planning Forward Reports.   The schedules so far provided tend to reflect an inadequate understanding of the problems being addressed and so are limited in usefulness.  The point I made in class is that Lexers inherently precede Parsers so the fact that some plans reversed the two was an indication of inadequate understanding.  There are three KEY COMPONENTS: GRAMMAR, LEXER/PARSER, BACKEND.  Plans that cannot be tracked are not plans at all.  STATUS: RUBY(working on ANTLR parser but ran into difficulty, their plan showed 30 March completion on Parser, but the plan confuses Lexers and Parsers), INFORM(plan is to create a mapper that takes INFORM game output and interactively builds a game map.  Currently don't have a GRAMMAR.  Plan seems to implicitly constrain Games to put out parseable messages, but it is unclear how this is to be accomplished.  Plan to complete GRAMMAR by Monday was stated.  Difficulty with Perl code found as an illustration by Megan is that it is long (50 pages or so).  Presumably something more complete will emerge shortly.), LOGO(Most complete input: Kittleberger provided a WORK LOG and an updated plan with reporting.  Next MILESTONE completiong of ReadFromConsole class to be finished on April 2nd.  Randall provided a slightly revised version of his plan with a schedule.  The April 2 MILESTONE is to complete his PLY parser.)  The meetings for the balance of the week will be short.  You should prepare brief reports.  The next MAJOR meeting will be on April 2nd.  A complete report on project status should be provided at that time.  Remember we only have about four weeks left.  You should provide a "blow by blow" plan that gets you to where you need to be.

3-21-2007 Friday we should be finishing up GRAMMAR definitions.  On Monday I expect Planning Forward Reports.  My own impressions are 1) LOGO is reasonably well defined and constrained, 2) RUBY is moving forward with progress on GRAMMAR and some decisions about the parser, but may run into problems ahead, and 3) Inform is a bit iffy on the GRAMMAR since they are uncertain about the range of output from Inform 6 subset/games which would be the input to their mapper.  The Friday meeting should stabilize things.  LOGO is on a learning curve for Eclipse see . PLANNING FORWARD REPORTS should include a schedule of expected events with Milestones through the end of the Semester culminating in the Project Review.

3-19-2007 Timbers, Satchell and Baker were MIA.  The LOGO team seems to be most focused and read to at least write trial code.  Both the Inform and Ruby teams are still floundering a little trying to narrow the scope to something they feel comfortable with.  Each have found some interesting on-line documents to read and study.  It is essential that focusing is completed ASAP.  I suggested that Team that have members that are not producing fire the non-producing members.  There is no place for the unwilling on teams that are trying to get things done.

3-12-2007 We had our first Seminar/Program Management Format Class today as a Round Table reviewing the Plans. The plans as handed in were mostly inadequate, so next time we'll do an updated WALKTHROUGH. 

Some key elements are obviously: 1) the LANGUAGE, with its GRAMMAR (you should be able to give some details on this), 2) the LEXER/PARSER that will process the language, 3) the BACKEND -- what comes out of this thing?  And how are you going to make that happen.  You should have at least some idea in each of these areas.

Do some research!-- (that means look stuff up and learn enough about it so you actually sound like you know what you are talking about.)

If you want to form teams, by all means do so.  However, you need to partition the tasking so I know what each team member is responsible for and you have to avoid "finger pointing" -- I want to see progress.  You should bring tangible progress to the class.  If you want me to cover something or help in some way, please let me know.  I've available as a "consultant" -- as well as wearing my "program manager" hat.

2-28-2007 We'll continue with Bottom-Up Parsing today but I thought I'd include the content of the email I sent out.


I was exchanging email with Bobby Kittleberger about planning and thought it might be fun to send you all some thoughts.  You may not have given much thought to planning but it is an important discipline in its own right that ought to be taught more fully in college than it usually is.  Good planning is what makes complex activities go well.  Without it you often only get chaos.  So what is a good plan?  In a real environment it ought to provide subsidiarity -- that is scope for local originality and creativity.  So it ought not to overdefine but guide.  By the time one gets down to an individual however, the plan should be very detailed, but it should be the individual's response to a more general set of objectives decending from above.  A couple of quotes from my Levels of Study hand-out:
“Never tell people how to do things.  Tell them what to do and they will surprise you with their ingenuity.”  - George Patton
“Hell, there are no rules here – we’re trying to accomplish something.”- Thomas Edison

“To manage a systems effectively, you might focus on the interaction of the parts rather than their behavior taken separately.” - Russell Ackoff

“If it’s a good idea, go ahead and do it. It is much easier to apologize than it is to get permission.” - Admiral Grace Hooper

Keeping those in mind a generic plan is a response to a need and consists, in broad terms, of:
1. Description of the Project / Objectives / and Approach.  The Objectives are what is to be accomplished and the Approach is how it is to be accomplished.  Objectives rarely change much while the approach responses to vicissitudes.
2. WBS (Work Breakdown Structure) is the term that the Air Force, I think it was, coined to describe the hierarchical decomposition of Objectives into Work into Tasks, etc.  This mostly applies to large tasks but can apply to any work done to achieve objectives. Look at WBS They have a nice illustration of a Bicycle.  I tend to call that kind of thing a PBS (Product Breakdown Structure) and then build a WBS behind it.
3. Work is then described in the form of Tasks.  A real-task is one which emits some observable work product.  That doesn't mean that work can't be intangible, only that intangible work, like thinking about the problem, cannot be observed and what cannot be observed is difficult to oversee or manage or even tell if it is happening at all.  So generally we talk about work in terms of observable events which are usually tied to Deliverables.  So I tend to equate Work = Production of Deliverables = Observable Events = Milestones.

I started to illustrate the idea of a Plan to Bobby with the following BNF'ish breakdown -- it isn't finished and I don't want to be too structured but I do want to get the idea across that a plan isn't something slapped together.  It is a carefully crafted, fully thought out, description of the work to be done, and then it is used as a vehicle to track progress and it is MAINTAINED -- so that the plan changes to always reflect reality.

A Partial Cut at a Planning BNF:

plan = description + { task } + { milestone } + schedule
task = task_description + task_deliverable_description + scope_estimate + [ milestone ] + [risk_assessment]
task_description = task_header + [task_body]
task_header = numeric + task_summary_statement    // ex. 1.1.1 Create a plan
task_summary_statement = task_verb + task_deliverable
task_body = how-to_text  # terminal description of the effort should be brief and is optional if task_header is adequate.
task_deliverable_description = what-is-it_text # terminal description of deliverable should key to specification if possible
scope_estimate = {resources_required}
resources_required = time + knowledge + skill-set + money + tools #Time is the most important and the others should be present only if needed to clarify scope

... This is not done of course but it gives you a little idea of what planning is all about.  To get more information you can look at Microsoft Project, at PERT and CPM and other on-line resources.
CPM  Critical Path Method
PERT  Program Evaluation and Review Technique
Gantt Charts Simple time chart which is very helpful for small projects and useful to summarize large projects.

Try to create a professional plan for your project.  Careful thought at the beginning will ensure a better product at the end.

Regards, Ray

2-26-2007 Today we started Chapter 5 Bottom-Up Parsing but we also looked at the little calc example -- one page of code that produces a little 4 function calculator for integers.  An interesting exercise would be expanding the grammar to do floating point numbers.  That should not be too hard.  You can probably just change the regex slightly and then cast to a floating point number instead of an int.  Anyone want to try it?

2-23-2007 Class Project Concept Presentations were given.  No one turned their slides in on time, but some were more prompt then others.  The integrated slides are on the R: Drive in the presentation in the Lectures folder titled: CSCI435W5C3StudentProjectConcepts.ppt  A number of interesting ideas were presented.  The key to a successful PROJECT is trimming it down to a MANAGEABLE SCOPE.  ASSIGNMENT: A Project Plan is due next Friday in written form.  It should be from three to five pages in scope.  It can be longer, but only as a White Paper attachment.  The Project plan should contain at least three sections: 1) Description of the Project.  This should include the objectives and Deliverables.  Deliverables from a generic point of view should be a small language grammar, a lexer and parser and a Interpreter/Translator that makes the thing actually run.  There should also be an example program that runs and shows off the project.  2) Milestones which are observables schedule throughout the next nominally eight or nine weeks that get you from here to there.  and 3) Schedule which is the time and sequence data on the milestones.  After we finish the next couple of weeks of class we will go into SEMINAR MODE which will be meeting three times a week to talk about our progress.  It will be a round table format and each person will talk about ISSUES and ACCOMPLISHMENTS.  An ISSUE is a problem one is encountering which needs to be solved and an ACCOMPLISHMENT is something that you have done that moves you in a significant way towards successful completion of a MILESTONE.

What is a milestone?  A Milestone is an observable accomplishment that demonstrates tangible progress towards your goal.  A logical sequence of milestones might be: a. Formal definition of  your grammar, b. Successful operation of a lexer on sample code, c. Successful operation of your parser generating example ASTs, d. A working interpreter/translater.

The final payoff will be PROJECT REPORTING DAY when you turn in all your deliverables.  This will be hard to distinguish from the last day of class.  If anything here is unclear you can get clarification by sending me email or asking questions in class.  YOU SHOULD HAVE FUN WITH THIS.  This is a baby model of the kind of thing you may be spending your life doing.


2-21-2007 ASSIGNMENT: Using PLY write a Scanner for TINY, see the Tokens for the TINY language on page 75 of your textbook.

2-20-2007 Yesterday we looked at BNF and EBNF.  Students are working on a little language ideas for their presentations.  The PPT slides are due Thursday for presentation in class on Friday.  Meanwhile I want them to build a Lexer for TINY using PLY.

2-16-2006 We talked about the plan of the course -- the first half focusing on FUNDAMENTALS: Lexing, Parsing, ASTs, Decorating the AST, Code Generation and the second half will focus on a PROJECT for some sort of Little Language.  I talked about possibilities Inform or DarkBASIC or something else and I mentioned some examples.  Then I launched off on a "rant" about getting turned on and involved -- you have to participate.  If you don't participate you not only don't learn anything but you are developing a bad habit which will only hurt you in later life.  I give you permission to get in my face!  How else will you learn.  Now for some links -- I've been giving out papers -- I wonder if any of you have been reading them?  Inform 7  Inform 6 Wikipedia On Inform  DarkBASIC  Check-it out!  Find some Little Language ideas.  They can be small parts of larger languages or I think more interesting, unique and useful little languages.  Doing some Google Searches on language topics may stir your juices and get some ideas cooking.  DEVELOP A PRESENTATION FOR NEXT TIME!

2-7-2007 PLY handout and note that PLY is on my website at PLY  What we will be doing is working to get the calculator example to work.  This might take a while and we will be reading the code to understand what is going on.  Note that the code is also on the R: drive for the course.  Also you might want to check out Inform, an interactive fiction game language.  One project I thought might be fun was designing a miniature version of Inform or a similar game language.  With Python it is possible to link in graphics and maybe (I'm not sure how exactly at this point) sound.  We just have to explore the available libraries.  PIL -- Python Imaging Library is the image manipulation.  There are also PyOpenGL bindings and Py has built in support for Tcl/Tk which lets you do graphics.  So there are a lot of possibilities for the serious student.

2-5-2007 Planning to give a lot of handouts in class today.  Assignment is to read the Dr. Dobbs Journal paper and do the following short exercise for Wednesday's Class:

Consider the regular expressions:
	a, a.b, a?b, a*b, a+b, a\\.b, a\\+b, .*ab
and the following strings:
   a, b, Xab, aXb, abX, ab, aXXXb, a.b
Make an 8x8 matrix with the regular expressions as the row labels and the
strings as the column labels and put a + in each row,col combination that 
is a match to its respective regular expression. (turn in next time)

2-2-2007 We did a Python workshop today using two of the on-line tutorials.  The experience should give you some idea of the flixibility of the Python string library and the Python re (regular expression) library.  If you were unable to complete the workshop, you should work on it in your room on your personal computer.  Regular expressions are heavily used in parsers as are string functions.  The next thing we'll start looking at is the content of Chapter 2. So read over Chapter two paying special attention to regular expressions and Finite State Machines.  You should have seen Finite State Machines in some form in CSCI 225.

1-29-07 I put support materials on the R: drive at CSCI435S07.  Currently it contains 2 Word documents with Links on Python and on parsers in a syntax like <link> // <comment on link>.  The homework for Wednesday is 1) Read Chapter 1 and go to the exercises and pick two that interest you and do them with the thought of briefing them on Wednesday.  2) Send me an email with the exercises you have picked to do.  You may be picked to brief one of those on Wednesday and maybe not.  You can create slides or chalk talk from notes.  A suggestion: A very good study technique (at least it works very well for me) is to Synopsize the Book when I study.  I create notes in a format like <concept> <concept explanation/exposition><example/illustration> and work my way through the book that way.  Sometimes I just open Powerpoint and make slides from the book as I study it so when I'm done I have a slide presentation.  That might be a suggestion.