#sed directive

[Proposal revised 10 Nov 2001 -- MikeOrr]

#sed would be like #filter but operate on on the final output, line by line, just before Cheetah returns it to the caller. (#filter operates on $placeholder values.) The filters in Cheetah.Filters are interchangeable between #filter and #sed, although some filters make sense more with one than with the other. In particular, the Strip filter should work equally well in either case. (Strip strips leading/trailing whitespace on each line but does not join lines, which was the request that originally inspired #sed.)

Unlike #filter, which always uses a default filter if no other filter is active, #sed would not call a filter in the default case. It would be redundant, since in #sed's case (but not in #filter's), the default filter would return the string unchanged. Initially, and after '#end sed', '#sed None' or '#sed Sed' (all interchangeable), the template would execute:

self._currentSed = None

(or the equivalent). After '#sed SomeFilter', the code would be:

self._currentSed = SomeFilter_(self)

Then just before returning the output, split the output into lines with trailing newline, and feed each line through the filter. For example, using code from Strip:

# Assuming string 's' contains all the lines to be sed'ded.
sed = self._currentSed
if sed is None:
     return s # Return the input unchanged.
result = StringIO_()
start = 0 # The current line will be s[start:end]
while 1: # Loop through each line.
     end = s.find('\n', start) # Find the next newline.
     if end == -1: # If no more newlines.
              break
     chunk = s[start:end+1] # Include the trailing newline.
     chunk = sed(chunk)
     result.write(chunk)
     start = end + 2 # Skip to first char after trailing newline.
# Write the unfinished portion after the last newline, if any.
chunk = s[start:]
chunk = sed(chunk)
result.write(chunk)
return result.getvalue()

#sed is named after the Unix 'sed' command, of course.

-- MikeOrr - 10 Nov 2001

---

The name '#sed' means mean nothing to many web designers, even those who work on Unix systems. If web designers are intended to be included among Cheetah's users, a less jargony name may be better.

-- Hamish Lawson 09 Nov 2001

Suggestions? I proposed #output-filter, but BDFL Tavis rejected it because he wants the term "output filter" to apply to #filter. #sed is an appropriate name because out of all the commands on all platforms (AFAIK), 'sed' is the only one that edits files line by line in the manner #sed does.

-- MikeOrr - 09 Nov 2001

'sed' is short for "stream editor", so the name is not totally arcane. Other suggestions?

-- TavisRudd - 09 Nov 2001

---

It might be possible to implement this behaviour without any extensions to Cheetah, by using a combination of #def and #filter. Once we've worked out the behaviour and the implementation then we can address the syntactic sugar issue.

#def stuffToSed_
blah blah blah
#end def
#filter sed
${stuffToSed_, [argsToSed_]}
#end filter ## (which, I think we should implement!!)

Or even simpler:

#def stuffToSed_
blah blah blah
#end def
$sed($stuffToSed_, [argsToSed_])

where sed is a method of the servlet.

-- TavisRudd - 09 Nov 2001

My only objection is the fact that all strippable lines have to be wrapped in a #def method. This would be inconvenient if a large section is to be sed'ded (or even the entire template), especially if the section contains embedded #def's and #block's. Also, it means an extra $theSedSection_ placeholder that would not normally be there, since this is "regular template text" and not a "value" per se.

-- MikeOrr - 10 Nov 2001

One of the things people praise PHP and SkunkWeb for is output caching -- allowing you to cache an entire portion of output regardless of how it was constructed. (Whereas with #filter, it matters how it was constructred -- it must be something stuffed into a $placeholder value, even if that wouldn't be the most logical way to express it.) As a result, Webware has output caching on its wishlist, in addition to its regular caching. #sed is analogous to output caching, but it's "output editing". A feature that Cheetah would be unique in, and which might provide unexpected benefits in terms of flexibility in the future. At the very least, it will answer the request for a #strip directive. And for those who don't use it, it's completely unobtrusive and low overhead.

I'm not sure what you mean by a StripLines filter. Is that the same as the StripSqueeze filter I just checked in? (It replaces chunks of whitespace with ' ', removing all newlines.)-- MikeOrr - 10 Nov 2001

---

I deleted my other comment, because I was confused.

What I think might be sufficient is an extended function-call syntax. So if you make a function like:

def striplines(s):
     return '\n'.join(map(lambda x: x.strip(), s.split('\n')))

Then with the (imaginary) extended function call syntax, you'd just do:

#call $striplines lots of text... #end call

-- IanBicking - 12 Nov 2001