作者： April & May & June

📤: Symbol has exported from EDITOR package;

❎: Definition is not contained in Editor source code;

Movement

Regular

(point-before point) & (point-after point)

Move the POINT 1 character before / after. Like a shorthand of character-offset N=1 or -1 but slightly faster.

❎(move-to-column point column &optional (line (point-bigline point)))

Move the POINT to the COLUMN of its current regular line.

(move-point-to-offset point offset)

Move the POINT to the OFFSET position of current buffer. Similar with (setf point-position), but must be called with the buffer locked.

❎(FIND-PREVIOUS-CHARACTER POINT CHAR) & (FIND-NEXT-CHARACTER POINT CHAR)

Search character CHAR before / after the POINT. Return non-NIL and move the POINT to the position of that character if found, return NIL otherwise.

The value succesfully returned is a POINT, but it probably is not the same point with POINT and can be invalid, so do not use it.

Pixelwise

(cursorpos-to-point px y window &optional point)

Giving X position in pixels, Y in line-number, relative with WINDOW, return a nearest point of the position. If POINT is a valid editor point, move the POINT to that position instead of return a new point.

Must be called with both window and buffer locked. See Locking Window.

❎(point-to-cursorpos point window)

Return 5 values: The horizontal character offset and vertical character offset of the POINT, relative with the visible area of WINDOW; The character at that point; The editor face at that point; The position of that point at its buffer (= how many characters before it).

❎(point-to-xy-pixels window point)

Return the horizontal and vertical position of POINT in pixels, relative with the visible area of WINDOW, as 2 values.

Search

Some source are inside searchcom.lisp of the Editor source code.

Programmatically search

Use get-search-pattern (or new-search-pattern) and find-pattern to search through buffer programmatically.

(GET-SEARCH-PATTERN STRING DIRECTION &OPTIONAL USER-GIVEN)

Make a new search-pattern using STRING and DIRECTION. DIRECTION can be :forward, :forward-regexp, :backward, :backward-regexp. If DIRECTION is :forward-regexp or :backward-regexp, treat STRING as a LW regular expression and search with the expression. If DIRECTION is not indicate a RegEXP, search string in the way specified by default-search-kind Editor variable (:string-insensitive by default).

Calling new-search-pattern under the hood.

❎(NEW-SEARCH-PATTERN KIND DIRECTION PATTERN &OPTIONAL RESULT-SEARCH-PATTERN)

Return a newly constructed search-pattern.

KIND can be one of :string-sensitive, :string-insensitive and :string-regexp;

DIRECTION can be one of :forward and :backward;

PATTERN can be a string.

❎(FIND-PATTERN POINT SEARCH-PATTERN &OPTIONAL LIMIT)

Find the SEARCH-PATTERN 1 time, starting at POINT. LIMIT should be a point if provided.

If success, POINT will be moved to the beginning (lower position, no matter which direction) of the match, and return the length of the match (a number). Unless NIL and the POINT will not be moved.

(FIND-REGEXP-PATTERN POINT PATTERN FORWARDP TEXT-END &OPTIONAL TEXT-START)

Find regex pattern starting at POINT.

PATTERN must be a LW precompiled regular-expression

TEXT-END should be a point, indicating the search limit.

This function must be called inside a lock. It can be more efficient than find-pattern if you’re reusing the precompiled regex.

The two functions use the same function to handle Regex search under the hood: find-regexp-pattern-from-line and underlying search-for-regexp. *search-pattern-experts* seems contain handlers for find-pattern.

Insertion & Deletion

• 📤❎(INSERT-STRING POINT STRING &OPTIONAL START END)
• 📤❎(INSERT-THINGS POINT &REST THINGS)
  Things can be a sequence of characters & strings.
• 📤❎(INSERT-CHARACTER POINT CHARACTER &OPTIONAL COMBINE-CHAR)
• 📤❎(INSERT-FORM-AT-POINT POINT FORM)
  Insert quoted sexp FORM at point. FORM will be pretty-printed. Used in showing macroexpand / form walking output.
• 📤❎(DELETE-BETWEEN-POINTS START END &OPTIONAL DELETED-STRING)
• ❎(INSERT-SPACES POINT NUM)
  Shorthand for inserting a NUM of spaces
• ❎(DELETE-CHARACTERS POINT &OPTIONAL (N 1))
  Delete N character from POINT. Positive to delete forward, negative to backward.
• ❎(BIGLINE-REPLACE-CHARACTERS BUFFER BIGLINE START STRING STROFF LEN)
  Bigline-related function to replace characters directly in region. STRING should be a buffer-string, STROFF is the starting-offset of STRING, LEN is the length of characters need to be replaced. Must be called inside a lock.
  

Window

Locking Window

It seems that only 2 functions for window locking are reserved:

❎(lock-window-and-call func window)

Lock WINDOW and call FUNC. FUNC is called with one argument that’s the WINDOW itself.

(call-with-window-and-buffer-locked window timeout function arg)

Lock WINDOW, and lock BUFFER as for modification, then call FUNCTION with ARG. The FUNCTION will receive 3 arguments: WINDOW, BUFFER and ARG.

Text Property

The text property of the buffer are constructed by a continuous sequence of text-property-regions, splitted by text-property-point. See the notes at the head of text-properties.lisp of the Editor source code.

The text-property content is a plist that can hold any keys and values. the 'editor::face property is used to implement face facility of LW Editor.

📤Interface Functions

They’re same with Emacs’s corresponding functions. Notes that all points are editor:point.

If modification is non-NIL, the change (of-course include the change of text properties) of the function made will be recorded into buffer’s undo history using record-text-property-undo.

When modifying the value of a property using functions like put-text-property and alter-text-property, make sure you are giving them a fresh-new object, but not the original object modified, e.g. modified list using nconc or delete.

That’s because these functions will merge the text-property-regions automatically, by comparing if two value objects are same using EQ (inside the same-properties-p function). Thus if you give a original object modified, the region of your modification may expand to its neighbours.

• (get-text-property point prop)
• (add-text-properties start end properties &key (test ‘eql) (modification t))
• (remove-text-properties start end properties &key (modification t))
• (remove-text-properties-no-edit start end properties)
  Shorthand of (remove-text-properties start end properties :modification t)
• (set-text-properties start end properties &key (modification t))
• (put-text-property start end property value &key (modification t))
• (put-text-property-no-edit start end property value)
  Shorthand of (put-text-property start end property value :modification t)
• (alter-text-property from to prop func &key (test ‘eql) remove-nil-p (modification t))
• (merge-text-property-list start end property value &key (modification t) (test ‘eql))
• (merge-face-property start end face &key (modification t))
  Shorthand of (merge-text-property-list start end 'editor::face value)
• (text-properties-at point)
• (text-property-any point property value &key (test ‘eql) limit)
• (text-property-not-all point property value &key (test ‘eql) limit no-move)
• (next-property-change point &key limit)
• (next-single-property-change point prop &key limit (test ‘eql) no-move)
• (previous-property-change point &key limit)
• (previous-single-property-change point prop &key limit (test ‘eql))
• (list-text-properties-at &optional (point (current-point)))
  Will pop-up an interface with a list of properties shown

Text Property Region

Functions related with text-property-region.

See text-properties.lisp of Editor source code for more details.

(region-text-property region prop &optional default)

Retrieve the underlying text-property-region-plist of a region.

This is the only function to retrieve text-property-region-plist, since LW has shaked its accessor. To set the plist, use the interface functions above.

(region-text-property-or-default region prop)

Same with region-text-property, but take the *default-text-properties* count as the default-default.

(ensure-text-property-region start end)

Giving editor:points START and END, Return a text-property-region starting at START and ending at or before END. Second value indicates if it exactly ends at END. The third values can be :before, :after or :both, indicating whether the region has been chopped at its two sides.

If START is inside a text-property-region, the region will be chopped to be started at START; If the region is longer than END, chop the region to make it end at END;

If the region is shorter than END, return the early-ended region with second value NIL.

Buffer String

Just like Emacs’s buffer-string.

(make-buffer-string &key %string properties)

Make a buffer string. Properties is a list of (start end plist) representing the text-property-region relative with the %string(constructed by bounded-text-properties-in function).

Example:

(editor::make-buffer-string 
  :%string "test"
  :properties `((0 4 (editor:face ,editor::*highlight-face*))))

(insert-buffer-string point buffer-string)

Insert buffer-string to POINT.

(points-to-buffer-string start end)

Retrieve a buffer-string between START and END. Same with Emacs’s buffer-substring.

Must be called inside buffer or START point locked.

(concatenate-buffer-strings buffer-string1 buffer-string2)

Concatenate two buffer-strings.

Overlays

Much performant than Emacs’s overlay, although lacking (= need hacking) for some functions.

Interface Functions

They’re basically same with Emacs’s corresponding functions.

• 📤(MAKE-OVERLAY START END &KEY START-KIND END-KIND)
  START and END will be copied using copy-point, and the copied points will be used as start & end of the overlay. START-KIND & END-KIND can be one of :before-insert, :after-insert or :temporary (default), specifying the kind of new points. They have same functionality with FRONT-ADVANCE & REAR-ADVANCE options of Emacs’s make-overlay, with :after-insert behaves like advance, :before-insert behaves like not-advance.
• 📤(COPY-OVERLAY OBJECT)
• 📤(MOVE-OVERLAY OVERLAY START END)
• 📤(DELETE-OVERLAY OVERLAY)
• overlay-start, overlay-end, overlay-buffer, overlay-properties
  Accessors of overlay object slots
• 📤(OVERLAY-GET OVERLAY PROP)
• 📤(OVERLAY-PUT OVERLAY PROP VALUE)
• 📤(OVERLAYS-IN BEG END)
• 📤(OVERLAYS-AT POINT)
• (BUFFER-OVERLAYS OBJECT)
  Return a list of all overlays in buffer.
  

Overlay Properties

Properties can be put into overlays using overlay-put, and retrieved using overlay-get.

All Property Names Are Symbols Inside the EDITOR Package (May Not Exported).

• face: Face applying to texts inside the overlay (if they’re visible)
• priority: A number default to 0. Overlays with larger priority can override the same property in other overlays.
• window: If specified with an editor:window, the visual effect of the overlay will only be shown on that window.
• invisible: Contents inside the overlay will be invisible, texts before and after it will visually merged together.
• before-string and after-string: Additional virtural-string shown at the start or end of the overlay. The value can be:
  
  • A string without face
  • A cons that has the string as CAR, and corresponding face as CDR.
  • A virtural-string object created by make-virtual-string
  • A vector that suitable for passing to generate-virtual-string-from-vector
    To put multiple faces on the string, use either generate-virtual-string-from-vector or virtual-string-append-string.
• isearch-open-invisible and isearch-open-invisible-temporary can control the visible state of the overlay when being i-searched. See the searchcoms.lisp and definition-folding.lisp for details and example.
  

Hooks

(Probably most of) Global hooks are defined with docstrings inside the init-editor.lisp of the Editor source code. All of them are worked. Use add-global-hook to add hook object. Hook object can be a function or fbounded symbol. Symbol is recommended as it’s easy to be removed using remove-global-hook.

Buffer-local hooks:

• buffer-after-change-hook, use set-buffer-after-change-hook with ADD parameter non-NIL or NIL to add or remove.
  The parameters applying to hook function are (buffer change-start-offset old-offset new-offset). offsets are numbers representing positions inside buffer.
• buffer-before-command-hooks, use add-to-buffer-before-command-hooks & remove-from-buffer-before-command-hooks to add or removed. Hook added using add-temporary-to-buffer-before-command-hooks will only be triggered once, then it will be removed.
  Parameters applying to hook function are (buffer command-function command-args).

1. Features
2. Advantages
3. Struggles

You may have heard of the famous “Macro” system in Common Lisp and many other Lisp dialects. You cannot realize its magic if you’ve never touched Lisp, as you cannot compare its features to those of competitors elsewhere. This is because of its unique features and smart design under the hood.

Features

Compile-Time Function Execution

The first feature is “compile-time function execution”. Let’s look at the macro in C. It looks like #define name[(args)] [expr]. Even if it can be long, has multiple lines and expressions,

#define MACRO(num, str) ({\ 
            printf("%d", num);\ 
            printf(" is");\ 
            printf(" %s number", str);\ 
            printf("\n");\ 
           }) 

An example from https://www.geeksforgeeks.org/multiline-macros-in-c/

But the idea behind is very simple: just search and replace, no logic computation can be taken during the macro being expanded. So that you cannot do something like generate multiple expressions based on the number of arguments.

(defmacro reset (&rest list-of-vars)
  (cons 'progn
        (loop for var in list-of-vars
              collect `(setq ,var nil))))

; (reset foo bar baz) => (progn (setq foo nil) (setq bar nil) (setq baz nil))

C: ???

A similar feature can be found in Rust’s Macros, and that’s why Rust’s macros get such high praise from its users. But Rust’s macros only support limited operators during expansion — compared with the next unique feature of Lisp macros.

Cooperate with the Environment

Yeah, some languages allow you to do some logic during macro expansion (actually before or in compilation), but what if your requirements go beyond the predefined operators? Can we employ more operators, functions, and even runtime features to yield the expansion?

If you’re sticking to static programming languages, you will say “No” as it’s impossible. Fortunately, Lisp is not a static language and has a powerful dynamic environment. You can use any function, even if you defined one line before, inside the macro definition.

(defmacro slot-value-> (object &rest slots)
  (reduce (lambda (exp slot) `(slot-value ,exp ,slot)) (cdr slots)
          :initial-value `(slot-value ,object ,(first slots))))

; (slot-value-> object 'slot1 'slot2) => (slot-value (slot-value object 'slot1) 'slot2)

One of the macros we wrote and quite often used. It employs a standard Common Lisp function REDUCE, and returns its result immediately during compilation.

How can it be done?

Dive into compiler

Different from languages like C, Rust or Python, which have separate environments between compilation and execution, modern Lisp implementations have a compiler which is running inside its Lisp environment. It means that the environment shares its symbols, variables and functions with the compiler. The compiler can use all of them during compilation, and anything compiled will be returned and loaded into the environment instantly. This unique feature is the basis of the powerful Lisp macros.

The compilation environment inherits from the evaluation environment, and the compilation environment and evaluation environment might be identical. The evaluation environment inherits from the startup environment, and the startup environment and evaluation environment might be identical.

— Compiler Terminology, Common Lisp HyperSpec

Notes: Lisp’s environment is very flexible. You can retrieve and modify the environment using the &environment argument and augment-environment, define special compiler-macro, establish variable bindings during compile time using compiler-let and so on. See the HyperSpec and Common Lisp the Language for more information.

Macros are Functions

Behind its name, Lisp macro is just a special type of function which works with literal arguments instead of values. Anything you can do with a function can also be done with a macro, and any existing functions can become macros, too. You can pack it into a closure, remove it with unintern, or change it dynamically with #'(setf macro-function). Thanks to the wisdom of Lisp pioneers, you can do whatever you want, even inside the compiler. Nothing can suppress the true freedom.

Sometimes, Lispers will use macros instead of inline functions, as they’re more straightforward.

It has been 40 years now (since the publication of Common Lisp the Language), and it is still unique, nothing can be compared with it.

Advantages

Lisp’s macro offers incredible power to extend the language easily, with efficiency remaining after expansion. The most famous (notorious) one may be the LOOP:

(loop for i from 0 below 10 collect i)
=>
(block nil
  (macrolet ((loop-finish () '(go #:end-loop)))
    (let ((i 0) 
          (#:to 10) 
          (#:by 1))
      (let ((#:accumulator (list nil)))
        (declare (type list #:accumulator))
        (let ((#:aux-var #:accumulator))
          (tagbody (progn (if (>= i #:to) (go #:end-loop) nil))
           #:begin-loop nil
                   (setq #:aux-var (loop::loop-set-cdr (the cons #:aux-var) (list i)))
                   (progn (let ((#:temp (+ i #:by))) (setq i #:temp)) (if (>= i #:to) (go #:end-loop) nil))
                   (go #:begin-loop)
           #:end-loop (return-from nil (cdr (the cons #:accumulator)))))))))

Yes… it’s also a macro. I cannot write it longer, or your screen will be blown up by the expansion. Temporary symbols (starting with “#:”) are simplified for reading. (Implementation: Lispworks)

As you can see, there are only basic operators after expansion, which can be transferred to nearly equal amounts of assembly. No run-time compensation. That’s why people say that Lisp is good for “making a new language”.

Notes: In Common Lisp, there’s also Reader Macro that can define how Lisp reads the source code. You can introduce new syntax to Lisp with it.

Also, Lisp-style macros are potent to break the barrier between syntactic expressions and function calls — But it’s meaningless for Lisp itself, since Lisp’s source code (S-expressions) is already a valid data structure in Lisp’s runtime.

Besides, is there anything better than making yourself work less and more comfortable?

Struggles

But macro can also become a barrier sometimes. It makes things difficult for those who rely on “stability”.

One is static code analysis. Macro makes it impossible to analyze Lisp code statically. As we mentioned before, Lisp macros are functions, and you cannot get their results without executing them.

This yields some consequences. For example, Lisp language support is difficult to integrate into modern IDEs, as it’s difficult for Lisp to cope with the Language Server Protocol (LSP). You need additional real-time communication between the source code and the Lisp image running the LSP server. Many helping features cannot be implemented as well, like type inference and code refactoring.

But this doesn’t mean there’s nothing to use. There’s some great works like alive-lsp and scheme-langserver. There’re many things we can do.

Second, too much flexibility can corrupt generative AI, and make it difficult to help with coding. During your development in a certain project, you’ll fund a large set of helper macros and functions, import certain libraries, and finally build a subset language that is specially for your needs. For generative AI, it’s difficult to cope with those new facilities, and you may need special prompts & better models for ideal results.

Besides, complex macros can easily confuse readers who are new to Lisp, as understanding the logic of expansion and all other prerequisites is costly. This is also a reason for the division of Lisp dialects.

Many Common Lisp users will stick to the ANSI standard and only use those widespread utility libraries if possible, especially for some open-source projects. That reduces the difficulty for other members to join in development, making it more durable.

---

April & May, 2025. Home page, Medium