#| \#\"s in this file would need to be fiddled with. I'll post the common lisp file too.

I wanted to try writing streaming csv processing. I was inspired by pizzapal doing classic AI in guile from PAIP. I'm doing something trivial in common lisp instead. Sisters of Mercy is good music, tangentially. pizzapal is working with xlsx I think, but there are shell utilities to convert those to csv.

Streaming to me means that after an initial delay, data flows out at the same rate it flows in. Hypothetically only READs take time to do.

This is all I want to do:
1. Read list of headers
2. Add or change the headers with 3 in mind
2b. Make an alist of conses often kinda like 

(\"header\" . 
 (let ((last-val nil))
  (lambda (&optional in out) 
   (if (and in out)
    (format out \"~s\" (setf last-val (read in)))
    (values last-val)))))
   
3. call the closures corresponding to the headers from 2 for each row

Input stream string:
(defvar *test-input* \"\"this\" \"that\" \"the\" \"other\"
1 2 3 4
5 6 7 8
9 10 11 12
\")

Output (stream):

\"this:1\" \"that:2\" \"the:3\" \"other:4\" \"rowsum:10\"
\"this:5\" \"that:6\" \"the:7\" \"other:8\" \"rowsum:36\"
\"this:9\" \"that:10\" \"the:11\" \"other:12\" \"rowsum:78\"

Arbitrary situational caveats:
Has to stream cells, no #'read-line
Can't use functions schemes won't have
UTF8 streams dealt with not here.

I have no internet on my usual machines while I very slowly rebuild some gentoo boxens so I have no way of posting this soon

|#

;;;This will be what it sounds like
(defvar *assoclosures*)

(defun read-headings (in) \"
(read-headings in)
(Values (list <first line READs>))

Scheme probably doesn't have #'labels but w/e
\"
 (let ((line (read-line in)))
  (labels ((get-heading (stream list)
            (let ((heading (read stream nil nil)))
             (if heading
              (get-heading stream (push heading list))
              (nreverse list)))))
   (with-input-from-string (str line)
    (get-heading str (list))))))


(defun rw-each (headings in out) \"
(rw-each headings in out)
Like (mapl (lambda (x) (apply (cdar x) (list in out))))

Read (describe 'mapl)
\"
 (when headings
  (apply (cdr (assoc (car headings) *assoclosures* 
                    :test 'string=)) 
        (list in out))
  (rw-each (cdr headings) in out)))

(defun rw-rows (headings in out) \"
(rw-rows headings in out)
peeks for eof before doing #'rw-each
\"
 (let ((peek1 (peek-char t in nil nil))
       (peek2 (peek-char nil in nil nil)))
  (when peek2
   (rw-each (copy-list headings) in out)
   (rw-rows headings in out))))

;;;An alist like 
;;;((\"string\" . (lambda (&optional in out) 'foo)))
(setq *assoclosures* 
 (list
  (cons \"this\"
   (let ((last-val nil))
    (lambda (&optional in out)
     (if (and in out)
      (format out \"\"~a:~a\" \" \"this\" 
             (setf last-val (read in)))
      last-val))))
  (cons \"that\"
   (let ((last-val nil))
    (lambda (&optional in out)
     (if (and in out)
      (format out \"\"~a:~a\" \" \"that\" 
             (setf last-val (read in)))
      last-val))))
  (cons \"the\"
   (let ((last-val nil))
    (lambda (&optional in out)
     (if (and in out)
      (format out \"\"~a:~a\" \" \"the\" 
             (setf last-val (read in)))
      last-val))))
  (cons \"other\"
   (let ((last-val nil))
    (lambda (&optional in out)
     (if (and in out)
      (format out \"\"~a:~a\" \" \"other\" 
             (setf last-val (read in nil nil)))
      last-val))))
  (cons \"rowsum\"
   (let ((last-val nil))
    (lambda (&optional in out)
     (if (and in out)
      (format out \"\"~a:~a\"~%\" \"rowsum\" 
       (setf last-val 
            (reduce '+ 
             (mapcan (lambda (x) 
                      (let ((n (funcall (cdr x)))) 
                       (and n (list n)))) 
              *assoclosures*))))
     last-val))))))
  
;;;Test space delimited READable \"csv\" input.          
(defvar *test-input* \"\"this\" \"that\" \"the\" \"other\" 
1 2 3 4 
5 6 7 8 
9 10 11 12 
\")

;;;Test.
(with-input-from-string (in *test-input*)
 (let ((headings (append (read-headings in) (list \"rowsum\"))))
  (terpri)
  (rw-rows headings in t)))

(terpri)
(quit)