|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
Start of topic | Skip to actions
This page contains the scribe notes for the Fall 2009 COMP 617 seminar. The scribe should add a new heading and section below on this page, or attach a separate file to this page. The heading or the file should then be linked from the course schedule.
Notational conventionsWe decided to drop the Blackboard font on the wiki.
08/24/2009: Introduction to Exact Real Arithmetic (part 1)Speaker: Walid Scribe: CherifIntroductionThis semester we plan to focus on Exact Real Arithmetic (ERA). Floating point arithmetic is not exact. When you make complex numerical computations, you can get some significant imprecisions. Things that we need to deal with include:
Goals
Possible representationsRepresentations of real numbers between 0 and 1.
B = Bool = {0,1} = {T,F}
Real1 = (N -> B)
<0 1 0 1 1 0 ...>
Real2 = B x (1 -> Real2)
Real3 = 1 -> (B x Real3)
Real4 = B x N x Real4Bool is not needed. We need the set of integers Z instead of the set of natural numbers N. We need the list to be a lazy list (by delaying Real4)
Real4++ = Z x (1 -> Real4++)
Real-1/-2 = Dedekind cut (I & II) Homework (for Wednesday, bring to class as a printout)
08/26/2009: Introduction to Exact Real Arithmetic (part 2)Speaker: Walid Scribe: Cherif
Meaning: Real -> R
B = {T,F} = {0,1} = 2 (blackboard 2)
N = {0,1,2,...}
Reviewing Representation 1
Real1 = N -> B Example: 0.01101001....
if x=0.9999... then 10x=9.9999... and 9x=9 (by subtracting 10x-x) however 9x being equal to 9 means that x = 1.0which shows that x=0.9999... and 1.0 are equivalent so there are no canonical representation for reals.
(Real,0,1,+,-,*,/,exp,log,d/dx,integration,...)
add: (N->B) -> (N->B) -> (N->B) =RealxReal->Real
Homework (for Friday)
8/28/09: Survey of Number Inaccuracy Problems and SolutionsSpeaker: Walid Scribe: TravisThings to Cover
Remarks on Errors Caused By Inaccurate Real Number Representation
Ways to Deal With Errors
Homework (for Monday)
8/31/2009: Stream Programming in OCamlSpeaker: Jun Scribe: MarisaStream programming in OCAMLType declarations, general form: type name = T of type | T of type One of the homework representations was type r2 = R2 of bool * (unit-->r2) Star refers to pairing; multiple pairings can be used. type iorb = int | bool (It's either an integer or a Boolean, but need explicit tag to say which.) The tag to distinguish is: type iorb = I of int | B of bool In OCaml, all types are lost when put into an executable because it is statically typed. Explicit tags that are put in, however, stay. Computation done primarily with pattern matching, such as:match x with | I I --> I (I + 1) | B b --> B (not b) ;;The integer line is wrapped in the same tag (I). Instead of having an infinite suffix, have just one bit (first bit is actually computed), then next bit is “promised” to calculate on demand. Unit means type that has no interesting value, so it is used to represent uninteresting arguments or return values. (Cannot declare a function with no arguments, so use unit for the uninteresting.) ( ) is an object, but it is the uninteresting object. f: unit  r2 f ( )In Java for example, there is actually a type Void that represents the null value, but that isn’t the case here. Ex: let f (r: r2) = (*type annotation on the function*) match r with | R2 (b, f') match f'( ) with R2 (b', _) --> b'(Underscore is a keyword.) Example: R2 (a,a) is a syntax error, so R2 (,) would be used. Note: Doesn't quite work like a lazy sequence in terms of pattern matching. If we want to get the nth bit: (keyword rec is recursive function) Note: From Scheme, the difference between let and let rec depends on whether f can be used in its own definition (which would be a recursive definition).
let rec f r n =
match n with
| 0 --> match r with
| R2 (b,_) --> b
| _ --> match r with
| R2 (_,g) -->
match g ( ) with
f(g( )) (n')
The first bit is discarded in the underscore of (_, g), and then g( ) gives rest of sequence.
Conceptually, when representing a list the tail of the list is an empty list (canonical). However, for an infinite list, there is no tail. Instead, there is a function in between each cell that takes a unit. This serves as a cutoff so that there is no tail. This ends up being delayed computation ("promise" to compute).
Example: writing addition
let rec plus r s = match r, s with (*this r,s is the same as creating a pair and matching the pair*) | R2 (b,f), R2 (c,g) --> want to return something of type R2 let (carry, rest) = plus ' (f( )) (g( )) let carry1 = (carry 1, R2(b+c+carry2, rest) ;;-get the carry from the rest of the bits let plus r s = match plus ' r s with |(_, sum) --> sum (*parens act as tags here*)For adding, for example: 0.01010.. + 0.00101.. = 0.01.. no carry yet. As long as keep getting one and not a carry, need to check the next bit to see if there is a carry. Thus, want to precompute carry just by looking at the first bit.
let rec carry r s
match r,s with
|R2 (b,f), R2(c,g) --> (*bool arith. here*)
match b + c with
| (cry,0) --> cry
| (cry,1) --> cry + carry (f( )) (g( ))
9/2/2009: Functions Representing StreamsSpeaker: Walid Scribe: MathiasWalid's Comments About Notes
Comparison of Collection Types
Homework on IsomorphismProve that B ~= 2)
where B = {T,F} and 2) = {1,2}.
Def: A ~= B iff there exist f, g, such that
for all a in A, g(f(a)) = a, and
for all b in B, f(g(b)) = b.
Addition of Exact Reals Implemented Using FunctionsNotation:Meaning function: [|X|] = Meaning(X)Definition: Real = N -> B (Real, +, ...)(Reals are functions from natural numbers to Booleans)
[| + |] : R x R -> R
plus = + : Real x Real -> Real
: (N -> B) x (N -> B) -> (N -> B)
\___a__/ \___b__/ \result/
OCaml:
let rec plus (a, b) = let result n =
... a n ...
... b n ...
... plus (rest a) (rest b) ...
let rec because the function calls itself, i.e. it is recursive. plus is supposed to add the digits from a and b, and take care of the carry. We don't have a way to return the carry separately, though. Instead, when adding the n'th digit of a and the n'th digit of b, we return the carry of that operation. The n'th digit of the sum actually is the (n+1)'th digit.
[| A |] + [| B |]
[| A + B |] = -----------------
2
We push everything back. Since we move the digits to the right, this has the effect of dividing the sum by two.
plus "n'th digit of a" and "n'th digit of b" returns the carry of a_n + b_n
We can do a case analysis on the n'th digits of a and b:
let rec plus (a, b) = let result n =
match (a n, ab n) with
| (0, 0) -> 0
| (0, 1) -> (plus (rest a) (rest b)) n
| (1, 0) -> (plus (rest a) (rest b)) n
| (1, 1) -> 1
What does rest do? It shifts digits to the left.
(rest a) nis the same as a (n + 1)Therefore: let rest a = let result n = a (n + 1) in result SummaryFunctions are powerful, but it may be easier to just use streams. For example, how do we make sure there is no infinite recursion? Homework and further discussion were left for Friday.9/4/2009: Acumen and ERASpeaker: Walid Scribe: Marisa Handout points: standing research questions as a guide for the semester -Need to have real-world goals in mind that can be solved by the work we do. What are results of numerical inaccuracy (natural disasters, etc.) that could be prevented by solving the issue with exact real arithmetic? For floating point, computations are considered to be done in constant time. Essentially, each computation is one step no matter what the input is. For ERA, this is not the case. Rather, the user may wish to see the result to an arbitrary number of digits, and the cost may be large. (Cannot guarantee the next bit in a certain amount of time.) Each digit that results in the computation can be as big as the time it takes for the whole computation. This is the price of ERA's arbitrary precision. We're not building a calculator. We are solving bigger problems. We will use Acumen for example problems and compare to other implementations. We will build things on top of Acumen and beneath Acumen to explore this. Where can we win with ERA? -Exact answers to things that previously had inexact answers or no answers. -Maybe, get exact answers and run faster as well. How would this double advantage be possible? Ideas: Plausible, but only on a large scale; over a long series of computations. This would be where the advantage would come in. Floating point might end up spending more on controlling error, and ERA wouldn't need to. Counterpoint: More accurate algorithms wouldn't necessarily run faster. (Eliminating error control won't necessarily speed time.) Because era operators are smarter, they will be more expensive. More control overhead to achieve their flexibility. Automatic storage managmt vs manual storage mangmt: there are cases where manual is much better than automatic. Huge policy overheads. For any computation oyu do in era, you use ordinary arithmetic. How could speed be improved? There are reasons we can expect era to be faster. Numerical codes are written by hand. There is a limit to writing things efficiently by hand--it takes time. If you are able to do things at a higher level, more structure, you have more control with more opps for optimization. Automates more things. Also, you don't necessarily know that every computation is necessary for final result. (padding with extra bits to make sure there aren't too few bits) There could be cases like multiplying by zero and that takes time, when with era, you don't need to do that. (Get only the very necessary work done, so save extra time.) Note: We are talking about two kinds of speed here: designing algorithms, and then actual effective computation. Last point on handout refers to things that we need to understand. If you have a stream rep, how do you know that .999 = 1? Could take arbitrary number of steps to determine. So, we say that equality or comparison are undecidable. How does this affect our ability to compute? Redundancy, incrementality, and fairness are also things that need to be considered for their impact on computation. Strictness analysis and abstract interpretation can be applied to ERA to our advantage. The research questions here are dynamic. As we go forward, keep the questions in mind as a guide, and address them. Additions can be made to the list. Implementation: of addition: using rep. N-->B and indexing from 0(handout notes) for zero == <.0000...> defined as let zero n = false one == <.111...> let one n = true representing half: half == <.1000...> let half n = match n with | 0 --> true (see handout notes)We can define one, zero, half, first, and rest. Case analysis for determining first carry: A concise way to find second digit (first after carry) is an 'if then else' involving same cases (if d1=d2 then d3 is the digit, if not then the opposite of d3 is the digit). Extend this logic to say what the carry is (first digit). This 'hardware' representation exists because real numbers exist, essentially. (It is how the computer figures out if it's adding two numbers, on the most basic level.) Code in handout has a function plus that does what the d1, d2, d3 method does. Converting float to these reps, and vice versa. Generating pi: spend some time figuring it out on your own then look up what the research is on generating pi. Think about how to do subtraction and pi for next time. Keep this representation in mind (the d1, d2, d3) because it doens't show up in code. There is a lot of number shuffling in the current code, because of N--> B. Are there other reps that would make it ieasier to think and write about? (Consider.) 9/9/2009: MultiplicationSpeaker: Walid Scribe: Henry
f n= if n=0 then false else if n=1 then true else f(n-2)
g n= true
[[B]]]=1 [[A]]= 1/4+ 1/16+1/64+1/252...
[[AXB]]= [[A]]X[[B]] = 1/3
9/11/2009: Notes on cost estimation:Speaker: Walid Scribe: CherifThe fib function:fib n = if n = 0 then 0 if n = 1 then 1 else fib (n-1) + fib (n-2) Estimating the cost for various inputs:fib 0 = if 0 = 0 then 0 ... = if true then 0 ... = 0If we count the number of recursive calls, then cost is 0, if we count all the operations, then the cost is 3 fib 1 = if 1 = 0 then 0 else if 1 = 1 ... = if false then 0 else if 1 = 1 ... = if 1 = 1 then 1 = if true then 1 = 1Cost is 5 fib 2 = if ... = if ... = else ... = if ... = else .. = = fib (2-1) + fib (2-2)cost is the cost of computing 2-1 (which is 1) + cost of computing 2-2 (which is 1) and the cost of computing fib 1 (which is 5) and fib 0 (which is 3) and then adding them (which is 1) + the cost of the overhead to reach the recursive calls (which is 4) Cost is 15 fib 3Cost is 27 fib nCost is ? Cost table:
9/14/2009 Polishing WritingSpeaker: Walid Scribe: JunStandardized structuresWhat are the patterns that we see in newspapers?
The survey reportWhat are the necessary sections?
9/16/2009 Complexity of MultiplicationSpeaker: Walid Scribe: Alexandre Chapoutot The subject of the seminar is the complexity of the operation of multiplication between binary streams. By complexity, we mean the amount of work to get the nth digit of A * B. This complexity is expressed by a function f of n.ClaimsFollowing the different study of the participants we have these claims:
Operations to countThe main point into the study of the complexity is to know what we are looking at. In other words, what are we counting:
AssignmentRedo the study of the complexity with such things in mind. In particular, making some experimentation and making time analysis. To do that, we can use MetaOcaml? which offers a convenient way to make time analysis see for example:
AnalysisThe study of the complexity brings a deeper understanding the multiplication algorithm. The following operation introduces two stream addition if the element (a.c) is put in front of the stream (B.D).
a B
x c D
-------
0 (a.c)
0 0 (a.D)
0 0 (c.B)
0 0 (B.D)
The question is what happen following the different values of the
digits of streams to add. We do a case analysis and we study the
values of the complexity following the number of stream additions, the
number of Boolean operations and the number of recursive calls.
9/18/2009 Complexity of Multiplication (part2)Conclusions From Last TimeExamining a certain digit is a linear time operator, because it is constructed by a sequence of 'rests'. This is true no matter what number we are trying to get a number from.Examining Multiplication of 0x0Do we have to do work for all previous digits? Yes: because of the way the algorithm is designed, you start at the beginning of the number and look at one digit at a time till you get to the digit you desire. Is it possible to write a version of multiplication that doesn't require us to look at all of the previous digits? No. They can all influence the answer. A fascinating question is what is the lower bound for computation time for multiplication (with the best possible algorithm). We conclude that 0x0 is O(n^2).Examining Multiplication of 1x0 (or 0x1).We only have on addition. We will assume it costs constant time. C(n) = A(n+2) + C(n+2) + n. We decide that this formula doesn't actually accurately represent multiplication, because we need to do n additions for each n'th digit. Walid says that every time we introduce an addition, addition must be done for every future digit of the multiplication. A consensus that this is true isn't reached. But this would give us O(n^3) complexity.9/21/2009 Complexity of Multiplication (part3)Speaker: Walid Scribe: Marisa Assignment for next class: Bring final version of survey papers and of complexity analysis. For complexity analysis homework, do the hand derivation. Class will divide into groups of 2 to do collaborative revisions to the surveys, keeping notes about the collaborative process at the end of each iteration of the paper. Groups will merge into groups of 4, then 2 groups, then finally the class will collaborate on one final version. Space guidelines (page count) are loose guidelines--more space may be used if necessary. Complexity analysis: The issues with performance: -delayed cost on remaining streams ("rest" operation) -Do we need previous digits? -How many digits do we need after n? (present digit) These issues are of concern in defining any operation. At this level of defining the real numbers, the number of digits needed becomes a notion of sensitivity analysis. Sensitivity of a function is defined as df(x)/dx. For example, a straight horizontal line has no sensitivity to input. (It will not ask for digits of input in order to give output, because output is always the same.) A sensitive function has a high rate of change. The smaller the interval of x for which we want to know the value of the function y(x), (with x and y axes), the more digits we need. (This approach does not deal with discontinuities. It is for continuous and differentiable functions.) For multiplication: to compute x*y, extract head of x and rest of x, and head of y and rest of y. Just extracting each head is cost of n, because doing that requires doing "rest" operation, which is more expensive based on current position. Need to do the derivation by hand (full computation, tracing the evaluation) for the hw, in order to see how the program works. Eventually, timings will test the clock speed. Ultimately both speed and space are important for this programming. 0/0 case: linear-->quadratic 01/10 case: quadratic-->cubic (The intuition for this is that addition is linear.) 1/1 case: cubicAdditional notes:The right thing to do when analyzing the complexity of multiplication and addition is using higher order complexity functions. So for example we want to get the complexity of addition as function is inputs (where inputs themselves are functions). Is memoization helpful? can it reduce the cost of getting any of the bits of an input to a constant?9/23/2009 Complexity of AdditionSpeaker: Walid Scribe: Henry Recapping: In multiplication, getting the nth digit has n cost (as we take rest of rest of rest... of n). The Question: are the additions done dependent on n as well, or independent of it?Reexamining addition
a B
+ c D
-----------
d1 d2 R
Where:
d1=0 if a=c=0 d1=1 if a=c=1 d1=0 if a!=c, head of B+D=0 d1=1 if a!=c, head of B+D=1 -- otherwise, this doesn't get looked up d2=h if a=c d2=!h if a!=cNegating h would require us to look inside B+D, otherwise, h remains uncomputed Optimization for d1: d1=a if a=c d2: essentially, if a=c then B+D (remains uncomputed) else Toggle (B+D) (gets the first digit of B+D and flips its value) Definition of Toggle
let toggle f=
let h n=
if n=0 then
not (f(0))
else f(n)
in h;
Cheap: returning a function. Costly when pulling out digits.
Looking up d1 has constant cost if the comparison comes out right, possibly explosive in cost if comparisons don't work out
Cost formulationIgnoring explosive case, looking up second digit has constant cost when it occurs, but could cause more costs down the road Let's call the cost of addition A(0)A F, G(0)= F(0)+G(0)+ 1 A F, G(n+1)= F(0)+G(0)+1+A R(F) R(G) (n) R(G)(n)= G(n+1)+1[Edwin Westbrook Formulation] Even if we shift and have the first thing that we're looking at as the nth position, as per Alexandre's formulation, we still have extra work. Suggestion: Have recurrences on constant functions, remove operators on F and G in the second parts, simply have: A F, G (n+1) =A F, G(0) +A RF, RG, (n) 10/05/2009 Complexity of Addition (continued)Speaker: Walid Scribe: CherifAddition Revised
a B
+ c D
---------------
d1 d2 R
Where:
d2 = a=c?d:not d d1 = a=c?a:d d R = B+D Notation, Higher order function, and CarryingNote that:f(x,y,z)is the same as: f x y (z)It is also possible to have x,y as a subscript for f(z) to mean the same thing.
These things have different names like:
f (n,m) takes in a pair and returns an answer. The same task can be achieved using a function F (n) (m) which takes a value and returns a new function waiting for a second value to return an answer. See the example below:
f (n,m) = n + m F (n) = let h (m) = n + m in hGoing from f to F is called Curry(ing) or Shonfinkel(ing). Going in the inverse direction (from F to f) is called uncurrying.
Back to AdditionWe will useA(F,G,n) for the Cost of computing the nth bit while adding two reals whose digit lookups have costs F(n) and G(n).
The cost of getting the 0th digit is:
A(F,G,0) = F(0)+G(0)+1 //Lookup a, lookup c, evaluating if conditionThe cost of getting the 1st digit is:
A(F,G,1) = F(0)+G(0)+1+A(F+,G+,0)
= A(F,G,0) + A(F+,G+,0)
//Lookup a, lookup c, evaluating if condition, recurse on rests
The cost of getting the 2nd digit is:
A(F,G,2) = A(F+,G+,1)+1 //The last +1 is because we need to check that n != 0 and n != 1In general: A(F,G,n+2)=A(F+,G+,n+1)+1Let's see if this works for n = 5 A(F,G,5) =1+A(F+,G+,4) =1+1+A(F++,G++,3) =1+1+1+1+A(F++++,G++++,1) =4+F++++(0)+G++++(0)+1+A(F+5,G+5,0) =5+F+4(0)+G+4(0)+1+F+5(0)+G+5(0)But: F+(n)=F(n+1)+1therefore: A(F,G,5)= 6+F(4)+4+G(4)+4+F(5)+5+G(5)+5So we can generalize: A(F,G,n) =n+1+F(n-1)+G(n-1)+n-1+n-1+F(n)+G(n)+n+n =5n-1+F(n)+G(n)+F(n-1)+G(n-1)We can change addition from linear to quadratic (assuming F and G are constants) if we are strict about computing d1 and d2 instead of jumping directly to the rest.
10/7/2009 Acumen PosterThe overall structure is good. Descriptions of problem, solution stands out. "Acument saves time" is a good advertisement. Pictures illustrating what a CPS is? (IOW, we have no intro.) Pressure on real estate. Don't focus on PhyDL? in particular but should present acumen as a whole. Pendulum might be good. Edwin thinks bus controller illustrates the CPS idea well. It might be a bit too detailed. Turn examples into "modeling in Acumen". Best if we can pick diverse 3 examples. Don't write Acumen ASCII code, but put the pretty-printed code. Include the ASCII code in a tiny box. Marcie's example is nice because it demonstrates a bunch of ideas like Lagrangian, partial diff. Nice if we can write a discrete controller for Marcie's example. A simple controller that tries to stabilize it should be fine. Goal is to reduce total energy. Any other examples? Marcie's example showcases recent progress. We need other systems. Bouncing ball is potentially a good example. Maybe get mechies to help us write the equations. What's the controller part? We don't need controller because bouncing ball already has discrete transitions when it bounces. Tricky thing is that derivatives wouldn't be continuous. Old Acumen solves for highest-order derivative (which may be discontinuous). RIDL can discretely change any variable upon an event. We could illustrate interactivity (the haptic stuff). We might have to tweak title to say just (virtual) physical systems instead of CPS. We don't have to get into the hybrid aspect. Marcie' example is hybrid, interactive exmaple is not but includes discrete observations. Bouncing ball and juggling will have to wait for next year. You can remove the words like "Examples" or "Case Studies". Follow the style at the top of page and that should give you more space. Have big eye-catching pictures with a short detailed description in small text. Three pictures: stablizing a mechanical system (pendulum); implementing haptic feedback system; modeling complex oil-exploration robots. Include "Problem" with the pictures box. Acumen code right below them (connect code with corresponding examples). Small foot notes like "Expressive Mathematical Language", "Capturing Discrete Events", and "Modular Specification" or "Powerful Abstraction Mechanisms". Include the flow diagram of design (the one that depicts the iterative nature) in the Problem part.10/9/2009 Creating Reading ListReading list: -Pursuit of real answers -read from 2 perspectives: learning what's there, and also revising it to send it as a journal paper -if there are performance analyses or exercises that can be done in seminar, note them -MSB Arithmetic paper (Most significant bit) -BBP/BPP Paper on computing digits of pi -Corky's paper: Fortran and ERA -Edalat's paper on ERA integration -Journal paper by Edalat, Computing with real numbers A helpful perspective to take on the implementation is circuits. In the pursuit paper, it is proved that for any output of demand, only need a fixed amount of look-ahead. It would be interesting to see if we can eliminate buffering. We can have a multitude of implementation strategies as long as output is the same. An idea for a seminal paper is to compare different strategies. For each paper, everyone will write a one-page response: summary of work, strong points, and weak points. Strong points--what information can we, and other people, build on out of this paper? Weak points--what are limitations in the work, and what opportunities might there be for us to jump in and add to this work? Everyone will share their responses and discuss. Do one paper every 2 lectures. Someone is assigned to read the paper the first lecture, then they give a talk to help others with the notation, before everyone reads it. (Not discussing content, however.) The reader will write a one-pager about the notation used in the paper. Scheduling: first reader of each paper Pursuit--Marisa Pi--Henry MSB--Cherif Corky's paper--Edwin Edalat's integration paper--Jun Edalat's Computing with real numbers10/23/2009 Acumen Paper Abstract and Reading ListSpeakers: Cherif and Eddy Scribe: MathiasAcumen Paper AbstractEvery abstract should answer the following seven questions:
Reading List
Topic Actions: Edit | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r34 < r33 < r32 < r31 < r30 | More topic actions
Webs: Main | TWiki | Africa | CPSX | EmbeddedSystems | Gpce | Houston | International | K12 | MetaOCaml | MulticoreOCR | ProgrammingLanguages | RAP | RIDL | Sandbox | SpeechClub | Teaching | Texbot | WG211 Web Actions:  Teaching Web
 Create New Topic
 Index
 Search
 Changes
 Notifications
 Statistics
 Preferences
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Log In
This work is licensed under a Creative Commons Attribution 2.5 License. Please follow our citation guidelines.