Update A added to spec 14-Nov-2001

My spec for a $batch() method: http://iron.cx/cheetah/batch_spec.html. I'm currently implementing this. Please offer comments and suggestions below. -- MikeOrr - 13 Nov 2001

---

It would be easier to add things contextually if the specification was here. -- IanBicking - 13 Nov 2001

I would do that, except the document is large and I don't want to edit it through the web. - MikeOrr - 13 Nov 2001

---

You shouldn't need #set $start = $getVar('request.field("start")', 0), since it should be equivalent to #set $start = $getVar(request.field("start", 0))). -- IanBicking - 13 Nov 2001

OK, but it sidesteps the underlying defect (that it's clumsy to protect from NameMapper.NotFound_ error) rather than solving it. -- MikeOrr - 13 Nov 2001

---

It might be useful if you could do a batch over a dictionary -- unfortunately, I can't decide if that should mean iterating over the items(), keys(), or values()... they all would make sense. So I've talked myself out of that one. We should be sure you can do #for (key, value), b in $batch... in Cheetah, so that you can use items nicely. -- IanBicking - 13 Nov 2001

This is iterating over a dictionary, not iterating over a list of dictionaries (which is already planned). Does Python 2.2 allow iterating over a dictionary? Certainly Python can iterate over dict.items() and (in 2.2) dict.iteritems(). But this brings up further issues:

• dict.items() and dict.iteritems() unravel to [(key, value), (key. value), ...] Zope has special code to recognize a 2-tuple as a key/value pair. I didn't implement it coz I thought it wasn't necessary. Perhaps we can add it for phase 2.
• But if we did that, what would the key be? If we leave it as a tuple, $r.1 and $r.2 would fail because NameMapper doesn't recognize subscripts as keys (an earlier version did but Tavis removed that feature as unnecessary).
• We could do some magic to convert (key, value) to {'key': key, 'value': value} at batch creation time, which could be used normally: $r.key, $r.value . Would this be worth the overhead? It would also mean that $batch() would construct artificial records rather than returning the original (tuple) record.
• $batch() was not intended to iterate over a list of simple items, as dict.keys(), dict.values(), dict.iterkeys() and dict.itervalues() are. I suppose you could, and then $r would be the entire record. The statistics functions would fail because there would be no fields for them to look in. Perhaps we could use $b.sum() instead of the expected $b.sum("field") to operate on the record value itself rather than on a subvalue ($r.myfield). Is this a worthwhile feature?

• MikeOrr - 13 Nov 2001

There has been some talk about iterating over dictionaries in for loops in Python. I don't know how it was resolved.

As far as the result of .items(), if Cheetah support nesting tuples on the left-hand side of assignment then it won't be a problem. Right now I always do #for $key, $value in $items, in this case you'd just have to be able to do #for ($key, $value), $b in $batch -- this is possible in Python. -- IanBicking - 13 Nov 2001

---

A sortBy argument might be spiffy, which would just do a ``l.sort(lambda a, b, k=sortBy: cmp(a[sortBy], b[sortBy])= -- this is not as easy as it should be in Python, but very common to need.

-- IanBicking - 13 Nov 2001

I left out sorting thinking that should be precalculated before $batch() is called. But I didn't think about dynamic "click on this column name to sort by this column" tables. But you're right, we should support those kind of tables. But I'm not sure putting this intelligence in the batch is necessary. The batch is recreated at each request (I hadn't thought about caching between requests until now), and the calling template can take care of the 'sortBy' query parameter and do the sorting, no? Is there any advantage to teaching $batch() to do it? We'd need a set of methods like .query().

-- MikeOrr - 13 Nov 2001

---

I don't think Batch should be a mixin/servlet -- at least not the base class. The base class should be just a normal class, that takes the list as part of its constructor, and the .batch() method should work off that list.

You could then make a subclass that operates as a mixin, and would have the added feature of creating links. If you add a prefix option to the constructor -- that would be used before all GET variables -- then you could even mixin multiple batches. It would be weird -- you'd have to do =#from Batch import Batch as Batch2= or something. Maybe there's something else there -- I think the Mixin will be generally more complicated and will be subject to more change, which is why the base Batch class shouldn't be part of it.

-- IanBicking - 13 Nov 2001

I prefer the current mixin interface for now. Maybe you can convince me later. Because you don't need a public Batch object: all you need is a function returning a list for the #for loop to operate on. Having the user initialize a Batch object just means clutter in the template. There may be an advantage in terms of persistent batches, but it's too early to consider this because of the complicated issues involved. What if the underlying data has changed at the next request? what about thread synchronization when updating the persistent origList? I'm not sure the Batch module should try to manage this.

The module will have one public function, and many helper functions (internal but may be used standalone), and several helper classes. E.g. $r.stats.sum("field") and $r.batch.stats.sum("field") will each use a separate Stats instance. An IndexFormat class handles .index/number/[Ll]etter/[Rr]oman in each place that is offered.

-- MikeOrr - 13 Nov 2001

---

Anyway, I think those two ideas -- straight batching, and a web interface to that batch -- can be comfortably seperated.

-- IanBicking - 13 Nov 2001

I'm all for modularity. I want standalone components, with Cheetah and Webware features on top of them. Or in cases where Cheetah knowledge is encoded at the lowest level, the module will provide a fallback implementation if Cheetah is not available (if ImportError) or if Webware is not available (if self.respond() raises AttributeError).

-- MikeOrr - 13 Nov 2001

Well, I think we can isolate a basic set of things Batch would do that wouldn't require it being a mixin. Then a subclass can do all the rest of the stuff. I think this seperation will probably be comfortable. Things like =.query()= would only be implemented in the mixin. Things like table generation would be yet another subclass, adding yet more functionality. -- IanBicking - 13 Nov 2001

---

It would be nice to have automatic table generation -- maybe with clickable headers to indicate sorting. So that should also be kept in mind -- at least the hooks should be there for a subclass to do so.

-- IanBicking - 13 Nov 2001

How can table generation and clickable sort headers be expressed in the spec? If you can provide a spec and/or examples of what you want, I'll consider it. I agree hooks would be good, but I don't know what kind of hooks or where they should go.

-- MikeOrr - 13 Nov 2001

---

I agree with the .even/.odd working off .number -- people who like 0-based indexes will also probably be comfortable with using %2.

-- IanBicking - 13 Nov 2001

---

length seems right to me -- len is a function, not a method.

-- IanBicking - 13 Nov 2001

I wavered, but decided .length() is better because people won't mistake it for the builtin function.

-- MikeOrr - 13 Nov 2001

---

__len__ should also be defined, so that you could use len() as well.

-- IanBicking - 13 Nov 2001

What would you call len() on in this context? Assuming there is no public Batch object, there is nothing except the list the #for is iterating over -- which you do not have available to call len() on. But the length of the original list is available anyway in $b.length, and the length of the batch is in $b.batch.length. And the length of the record -- there is no length -- because $batch() is currently suited for lists of instances or lists of dictionaries, not lists of lists.

-- MikeOrr - 13 Nov 2001

Now that I think about it, it doesn't make sense. -- IanBicking - 13 Nov 2001

---

.percentOfTotal_ should maybe take another argument that it would append. By default it would be "%". This way you get 10% or N/A, but not N/A%. But you also would want to be able to do math with it, so if the appending argument is set to None it should give back a number. -- IanBicking - 13 Nov 2001

Good idea. I've added it to the spec. I'm unsure whether $suffix should default to '%' or None.

I'm open to adding features like .percentOfTotal_() that reduce clutter in the template. Especially if they replace an entire if-else block, as the $b.prev.query() enhancements do. -- MikeOrr - 13 Nov 2001

---

It's a little confusing that .batch creates the batch, and then .batch refers to the entirety (as in $b.batch.index ). $b.batch should be renamed to something else, though I can't think of anything off the top of my head. You mention .all ?

-- IanBicking - 13 Nov 2001

$b.all.*, if implemented at all, would refer to the entire origList. $b.batch.* refers to the current batch. I considered calling it $b.page.*, but decided "batch" was more consistent. But I can change it if too many ppl find it confusing. - MikeOrr - 13 Nov 2001

I like .page better than .batch. Even if isn't always actually a "page", it's a good metaphor. -- IanBicking - 13 Nov 2001

---

Median should be implemented thus:

def median(lis):
    lis = lis[:]
    lis.sort()
    return lis[int(len(lis)/2)]

I've forgotten how variance and standardDeviation would be implemented, but I can look that up if you want. Tavis might remember these more quickly, though, since I imagine he's using this stuff a lot in his thesis.

-- IanBicking - 13 Nov 2001