I'd like to make some closures to speed up operations that are
specified at run time, as it is done for regex compilation. So let's
take a *very* simple exmaple:

CL-USER> (defun foo (value)
           #'(lambda (x)
               (eq x value)))
FOO
CL-USER> foo
#<CLOSURE (LAMBDA (X)) {AF278FD}>
CL-USER> (funcall (foo 1) 0)
NIL
CL-USER> (funcall (bar 1) 1)
T

So that's ok. Now let's suppose we would like the invert the test
depending on a predicate value:

CL-USER> (defun foo (value predicate)
           #'(lambda (x)
               (if predicate
                   (eq x value)
                   (not (eq x value)))))
FOO

Or, in order to avoid a test at runtime:

CL-USER> (defun foo (value predicate)
           (if predicate
               #'(lambda (x) (eq x value))
               #'(lambda (x) (not (eq x value)))))
FOO

It looks like a macro would allow to avoid code duplication here

CL-USER> (defmacro maybe (predicate statement)
           (if predicate
               `(,@statement)
               `(not ,statement)))
MAYBE
CL-USER> (macroexpand-1 '(maybe T (eq x value)))
(EQ X VALUE)
T
CL-USER> (macroexpand-1 '(maybe NIL (eq x value)))
(NOT (EQ X VALUE))
T

Now redefine foo:
CL-USER> (defun foo (value predicate)
           #'(lambda (x)
               (maybe predicate (eq x value))))
;[...]
; caught STYLE-WARNING:
;   The variable PREDICATE is defined but never used.
;[...]
STYLE-WARNING: redefining FOO in DEFUN
FOO

Calling the function shows that the variable PREDICATE is indeed
not taken into account.
Now (finally) the questions:

1. Why ?
2. How to use macros for generating the lambdas ?

Many thanks,

:Nicolas

  (defun foo (value)
    (lambda (x)
      (eq x value)))

  (defun predicator (predicate fun)
    (if predicate
        fun
        (complement fun)))

So your second "foo" could be:

  (defun foo-2 (value predicate)
    (predicator predicate (foo value)))

Zach

Not a good idea.

You think you now a macro expansion time the value of predicate, but
you don't. All you know is the code, but not its value.

Try:   `(if ,predicate ...  )

It has two parameters: predicate and statement.

The arguments are predicate and (eq x value)

predicate = predicate

statement = (eq x value)

Remember above

(if predicate ... )   ?

predicate evaluates to the symbol predicate, which is always true.

It is similar to (if 'predicate ...) ....

Because macros work at macroexpansion time, NOT
runtime.

 NE> 2. How to use macros for generating the lambdas ?

You can use macro to generate a code which generates lambda,
for example.

(defun foo (predicate value)
           (eval `(function (lambda (x)
                    (maybe ,predicate (eql x ,value))))))

or

(defun foo (predicate value)
           (compile nil `(lambda (x)
                           (maybe ,predicate (eql x ,value)))))

:Nicolas

You can also say (coerce '(lambda ...) 'function), and let the CL
implementation itself decide.

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

:Nicolas

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

  -T.

 NE> Ok, so I use macro to generate code and then invoke the compiler at
 NE> runtime.

No! You use macro to transform your code at compile time.
Then code you've generated does anything it wants in runtime.

 NE> What is best practice for this: eval or compile ?

In case you _really_ need it, best practice is compile.
But you do not need it in this case.
A rule of thumb: if you think you need eval or compile,
it is higly lickely you're doing it wrong.

Basically, the only case when you need eval is when
code is not known at compile-time.  That is, user
enters it from keyboard, or you recieve it from network,
or generate through GA or something. In other cases,
when you just need to fill in some parameters, you
can do that other way.

 NE>  Do they have similar meaning in this context ?

Yep they do, almost. The only difference is that
eval might or might not compile functions, while compile
almost certainly does. compile is a bit tidier IMHO
because it accepts only lambda rather than arbitrary
piece of code. There is also a third way to do it -- (coerce ... 'function),
it is pretty good too, if you just need a function, not necessarily
compiled.

That's how you can generate lambdas at runtime,
but there is no reason to do this in this case.

Xach have shown how you can do this with lambda
and without any code duplication. That was really the best way.
It is actually a good practice to avoid using macros when
you can just use higher-order functions.

If you absolutely want to use macros, you can do this like
this (using IF at runtime):

(defmacro maybe-lambda (predicate paramlist statement)
      `(if ,predicate
                (lambda ,paramlist ,statement)
                (lambda ,paramlist (not ,statement))))

(defun foo (value predicate)
  (maybe-lambda predicate (x) (eql x value)))

No runtime code generation, but maybe have lost its generality.

Or you can write a more complex macro which is more general.
Your `maybe` macro just maybe  inserts NOT in the beginning
of the statement.
My with-maybe macro can insert NOT into any place within
statement user wants it to.

(defmacro with-maybe (predicate statement)
  `(if ,predicate
    (macrolet ((maybe (statement) statement))
      ,statement)
    (macrolet ((maybe (statement) `(not ,statement)))
      ,statement)))

You use it like this:

(defun foo (value predicate)
  (with-maybe predicate
    (lambda (x) (maybe (eql x value)))))

Macro generates a code which has IF switching between two
branches. Branches are made different by different MAYBE
local macros defined via MACROLET -- in first branch MAYBE is no-op,
so it is vanilla statement. In second branch MAYBE adds NOT to
the statement.

Unfortunately there is no tool one can use to get fully macroexpanded
code (at least I don't know any), but once you'll replace macrolet:ed
maybe:s, you'll get exactly function you had wrote yourself:

(defun foo (value predicate)
       (if predicate
            (lambda (x) (eq x value))
            (lambda (x) (not (eq x value)))))

 NE> (defun foo (predicate value)
 NE>     (eval `(function (lambda (x)
 NE>       (maybe ,predicate (eql x ,value))))))

By the way, if you're generating code anyway, you do not really
need macro, you can do this with ordinary function or just construct
in-place:

 (defun foo (predicate value)
     (eval `(function (lambda (x)
            ,(let ((code `(eql x, value))
               (if predicate
                   code
                   `(not ,code)))))

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Here is a sketch [untested]:

(defclass smart-function (funcallable-standard-object) ()
   (:metaclass funcallable-standard-class))

(defparameter *threshold* 10)

(defun make-smart-function (body)
   (let ((fun (make-instance 'smart-function)))
     (set-funcallable-instance-function
       fun
       (let ((counter 0) (funfun (eval body)))
         (lambda (&rest args)
           (declare (dynamic-extent args)
           (when (> (incf counter) *threshold*)
             (unless (compiled-function-p funfun)
               (setq funfun (compile nil body)))
             (set-funcallable-instance-function fun funfun))
           (apply funfun args))))
     fun))

(defmacro smart-lambda ((&rest args) &body body)
   `(make-smart-function `(lambda ,args ,@body)))

Yes, this requires the CLOS MOP. ;)

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

:Nicolas

What is prohibitive is to compile using C a function that you are
going to use only once, and do it for every use.

The alternative then is to use bytcompiled functions, which do save
some time, but are not as efficient as natively compiled ones. You can
always do this, without much hassle. One cheap way is to coerce a
lambda form to a function type.

Juanjo

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Thanks,
Pillsy

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

(defparameter *threshold* 10)

(defun make-smart-function (body)
   (let ((counter 0) fun funfun)
     (setq funfun (eval body)
           fun (lambda (&rest args)
                 (declare (dynamic-extent args))
                 (when (> (incf counter) *threshold*)
                   (unless (compiled-function-p funfun)
                     (setq funfun (compile nil body))
                     (setq fun funfun)))
                 (apply funfun args)))
     (lambda (&rest args)
       (declare (dynamic-extent args))
       (apply fun args))))

(defmacro smart-lambda ((&rest args) &body body)
   `(make-smart-function (lambda ,args ,@body)))

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Might have better performance. If you use &rest and then apply
I guess most compilers will make a list, that is, you have more consing.

(lambda (&rest args)
   (let ((fn ....))
      (apply fn args)))

But if you're doing it via macro, you can use certain number of parameters
directly:

(lambda (a b c)
   (let ((fn ....))
      (funcall fn a b c)))

Then  there is no consing. There might be some overhead from using closure's
variable and funcall, but I guess funcallable-object's funcall is not
totally overheadless
either. So probably funcallable-object offers only a minuscule advantage in
this case.

Pascal's code went right over your head there. The advantage of the MOP
solution has nothing to do with this. And it has &rest/apply processing buried
in it.

What Pascal's code does is allow the user's function to be called several times
without being compiled. It counts how many calls there are. When the calls
exceed a threshold, then the object compiles the function.

When the function is compiled, the object is able to replace its own closure
with that compiled function. So subsequent calls go directly to that function,
bypassing the threshold check.

You can't do that with a lambda closure, because a lambda closure cannot
replace future calls to itself with a different closure.

;; tested modifications to untested code
;; fixed unbalanced parentheses in declare
;; extra levels of unquote added to backquote
;; class replaced by symbol

(defparameter *threshold* 10)

(defun make-smart-function (body)
  (let ((fun (gensym "SMART-FUNCTION-")))
    (setf
      (symbol-function fun)
      (let ((counter 0) (funfun (eval body)))
        (lambda (&rest args)
          (declare (dynamic-extent args))
          (when (> (incf counter) *threshold*)
            (unless (compiled-function-p funfun)
              (setq funfun (compile nil body)))
            (setf (symbol-function fun) funfun))
          (apply funfun args))))
    fun))

(defmacro smart-lambda ((&rest args) &body body)
  `(make-smart-function `(lambda ,',args ,',@body)))

Test:

[4]> (defparameter x (smart-lambda (a b c) (list a b c)))
X
[5]> (symbol-function x)
#<FUNCTION :LAMBDA (&REST ARGS) (DECLARE (DYNAMIC-EXTENT ARGS))
  (WHEN (> (INCF COUNTER) *THRESHOLD*)
   (UNLESS (COMPILED-FUNCTION-P FUNFUN) (SETQ FUNFUN (COMPILE NIL BODY)))
   (SETF (SYMBOL-FUNCTION FUN) FUNFUN))
  (APPLY FUNFUN ARGS)>
[6]> (funcall x 1 2 3)
(1 2 3)
[7]> (funcall x 1 2 3)
(1 2 3)
[8]> (symbol-function x)
#<FUNCTION :LAMBDA (&REST ARGS) (DECLARE (DYNAMIC-EXTENT ARGS))
  (WHEN (> (INCF COUNTER) *THRESHOLD*)
   (UNLESS (COMPILED-FUNCTION-P FUNFUN) (SETQ FUNFUN (COMPILE NIL BODY)))
   (SETF (SYMBOL-FUNCTION FUN) FUNFUN))
  (APPLY FUNFUN ARGS)>
[9]> *threshold*
10
[10]> (symbol-function x)
#<FUNCTION :LAMBDA (&REST ARGS) (DECLARE (DYNAMIC-EXTENT ARGS))
  (WHEN (> (INCF COUNTER) *THRESHOLD*)
   (UNLESS (COMPILED-FUNCTION-P FUNFUN) (SETQ FUNFUN (COMPILE NIL BODY)))
   (SETF (SYMBOL-FUNCTION FUN) FUNFUN))
  (APPLY FUNFUN ARGS)>
[11]> (funcall x 1 2 3)
(1 2 3)
[12]> (funcall x 1 2 3)
(1 2 3)
[13]> (funcall x 1 2 3)
(1 2 3)
[14]> (funcall x 1 2 3)
(1 2 3)
[15]> (funcall x 1 2 3)
(1 2 3)
[16]> (funcall x 1 2 3)
(1 2 3)
[17]> (funcall x 1 2 3)
(1 2 3)
[18]> (funcall x 1 2 3)
(1 2 3)
[19]> (funcall x 1 2 3)
(1 2 3)
[20]> (funcall x 1 2 3)
(1 2 3)
[21]> (funcall x 1 2 3)
(1 2 3)
[22]> (symbol-function x)
#<COMPILED-FUNCTION NIL>
[23]> (funcall x 1 2 3)
(1 2 3)

(defclass my-fun (funcallable-standard-object)
   ()
   (:metaclass funcallable-standard-class))

(defmethod initialize-instance :after ((f my-fun) &key)
   (set-funcallable-instance-function f (lambda () 42)))

* (setf (symbol-function 'foo) (make-instance 'my-fun))

#<MY-FUN {1002F44C09}>
* (foo)

42
* (setf (symbol-function 'bar) 'foo)
; in: LAMBDA NIL
;     (FUNCALL #'(SETF SYMBOL-FUNCTION) #:NEW671
'CLOSER-COMMON-LISP-USER::BAR)
; --> SB-C::%FUNCALL
; ==>
;   (#<SB-C::GLOBAL-VAR
;      :%SOURCE-NAME (SETF SYMBOL-FUNCTION)
;      :TYPE #<SB-KERNEL:FUN-TYPE (FUNCTION #'SYMBOL #)>
;      :WHERE-FROM :DECLARED
;      :KIND :GLOBAL-FUNCTION {1002F4A281}>
;    #:NEW671 'CLOSER-COMMON-LISP-USER::BAR)
;
; caught WARNING:
;   Asserted type FUNCTION conflicts with derived type
;   (VALUES (MEMBER FOO) &OPTIONAL).
;   See also:
;     The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
;   caught 1 WARNING condition

debugger invoked on a TYPE-ERROR: The value FOO is not of type FUNCTION.

Some Common Lisp implementations accept this, some don't. Those that
don't accept this are right not to do so, according to the HyperSpec. (I
think this is an unnecessary restriction, symbols should be accepted as
symbol-functions, but that's maybe just me... ;)

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

 KK> It's a poor Lisp compiler which conses a list for code which only
 KK> passes the &rest argument down to another function with apply.

Can you please show an example of a compiler which optimizes it?

 KK> You can't do that with a lambda closure, because a lambda closure
 KK> cannot replace future calls to itself with a different closure.

That's why we have not one closure, but two closures, with first
calling the second one -- then we can replace second closure with
a compiled version when we would want to. There is overhead
of indirection, but arguably it is very low.

Here's code by Pascal:

(defparameter *threshold* 10)

(defun make-smart-function (body)
   (let ((counter 0) fun funfun)
     (setq funfun (eval body)
           fun (lambda (&rest args)
                 (declare (dynamic-extent args))
                 (when (> (incf counter) *threshold*)
                   (unless (compiled-function-p funfun)
                     (setq funfun (compile nil body))
                     (setq fun funfun)))
                 (apply funfun args)))
     (lambda (&rest args)
       (declare (dynamic-extent args))
       (apply fun args))))

(defmacro smart-lambda ((&rest args) &body body)
   `(make-smart-function (lambda ,args ,@body)))

Let's see does it go over your head or not.