UnixWorld Online: Tutorial: Article No. 009

The vi/ex Editor, Part 9: Take Charge with Macros

By Walter Alan Zintz.

Text-Insertion Macros

As befits an editor with all those built-in metacharacters that operate while you are typing in text, there are two ways to create your own macros for use during text insertion. Both can be useful in the right circumstances, so you'll probably want to put them to work at times. You may not have a choice -- often the .exrc file that you may be given when you get a new Unix shell account has some of these shorthand dodges built in. These two tools have as many similarities as differences, so I will expound them in parallel.

What These Tools Do

Both tools act only when you are in text-insertion submode of screen-editing mode. Nonetheless, the commands that set them up and manage them are line-mode commands like:


:ab ucb University of California at Berkeley
:map! } Control-[k2cc

The two example lines above will set up two shorthand forms that you can use without further preliminaries. The first line provides that whenever you type ``ucb'' as a separate word in your text, the editor will replace it with ``University of California at Berkeley''. It happens right on the spot, and without any special signal from you.

The second line is for use if you frequently discover that what you are typing has become a mess, and that the mess started back on the previous line. With this shorthand form in effect, whenever there is a ``}'' character in what you type in, the editor removes it and instead acts as if you had typed in ``Control-[k2cc''. That is, the Control-[ (generated by the ``Escape'' key on your keyboard) causes the editor to escape from text-insertion to command mode, the ``k'' causes the cursor to move up a line, and the ``2cc'' removes both of the lines involved and puts you back in text-insertion mode, ready to type in a replacement for those lines and continue on with your text insertion. As with the previous tool, this happens as soon as you type in the shorthand form, without any special action by you.

Note that whitespace separates either of these setup commands into three parts. The first part, from the start of the line up to the first stretch of whitespace, is just the command name. Part two, between the first and second stretches of whitespace, is the short form that you will type into the text. The third part, everything following the second stretch of whitespace, is what the editor will insert (and/or execute) when you type in the short form. Only the first two stretches of whitespace are separators -- any later stretches are integral components of part three. And whitespace includes both space characters and tabs, in any mixture.

Working Principles

Now that you've seen what these two tools do, let's consider how they work:

:abbreviate

(Shortest abbreviation is :ab). This tool acts when you type in a certain character or string as a separate word, each end bounded by whitespace, or a punctuation character, or the start or end of a line, or the start or end of an insertion. As soon as the editor sees that the abbreviation is a word by itself, it replaces that abbreviation with the longer word or phrase you have set as equivalent.

As an example, you might have declared ``cat'' as your abbreviation for ``felix domesticus''. Then, wherever you type in a line such as ``the habits of the common cat include'', the editor will promptly change it to read ``the habits of the common felix domesticus include''. But there will be no such change in words that happen to include the string ``cat'' in them, such as ``catamaran'' or ``concatenation''. Be careful with this, because while the word ``catlike'' will not be changed, the word ``cat-like'' will be.

Neither a backslash (\) nor a control-V will quote an abbreviation into a file as itself. Usually, the easiest way to insert an abbreviation into your text is to escape from text-insertion submode (back to command submode) in the middle of typing the abbreviation, then re-enter text-insertion submode and type in the rest of the abbreviation. If your abbreviation is only one character long, though, you must fall back on typing the abbreviation with a letter immediately before or after it, then returning to command submode to erase the unwanted extra letter.

:map!

(No editor-accepted abbreviation). Very similar to the abbreviation tool discussed above, but with three major differences:

  1. The shorthand form defined with this command does not need to be typed into your inserted text as a separate word in order to operate. Even if it is embedded within another word, the short form will disappear and its related text will be entered in its place.
  2. This tool does not simply insert the related text into the file, as the :abbreviate tool does; it acts as though the user had typed in the related text instead of the short form. That is, if there is an escape character in the related text, that escape will put the editor back into command submode, and interpret any following characters as screen-mode commands. (Unless one of those characters returns you to text insertion submode -- then characters following that insert-text command will be all be put into the file, unless and until there is another escape character.) That makes accidentally triggering this tool rather dangerous.
  3. Quoting in a character or string that you've defined as a short form via this command is simple. Type control-V before the metacharacter, or the first character of the metastring, and into the text it goes. Even that may not be required if you are dealing with a metastring, and if the timeout option to the :set command is still in its default state: turned on. In this case, all you need to do is be sure that you take more than one full second to type in the entire metastring, and it will have no meta effect. (This, by the way, is one reason that this tool's metastrings should be short -- so you can depend on being able to type one of them in less than a full second when you do want the metavalue.)

Questions naturally arise regarding these tools. One frequent query is why anyone would want to mess around with :abbreviate when it seems much easier to do a general substitution command when the document is complete. That is, instead of that abbreviation I set up at the start of this explanation, just run a :%s/\<ucb\>/University of California at Berkeley/g command after all the text has been entered. There are several reasons to use Vi's abbreviation feature instead:

Another common question concerns precedence of metacharacters. You can use most of the text-input metacharacters I've discussed previously as short-form names in :map! commands. Suppose you did use control-D as such a short form -- what would happen when you typed control-D at the start of an autoindented line? Would it wipe out the indentation or type in the phrase that :map! has associated with it? Or if you used control-H as a short form? When you subsequently typed a control-H during text entry, would the cursor back up a space, or would type in a stored phrase?

The answer is that the :map! value would prevail. By preceding either character with a control-V, you could type in in as itself, but there would be no way to use the ordinary metavalue of either character. If you were to map control-D followed immediately by another control-D or two consecutive control-H characters or any other doubling of an ordinary metacharacter, the situation would be more complex. You could then type the two control-D or control-Hs within one second to get the mapped text typed in, or you could type control-D or control-H followed by a one-second pause to invoke the ordinary metavalue.

Time for another exercise.
Suppose that you used control-D or control-H as a short form with the :abbreviate command. Or suppose that you used some ordinary character string as both an abbreviation and a mapping short form. (The editor will allow you to do this.) What would happen when you typed in this double-use short form during text insertion? This exercise is straightforward enough that I expect most of you will find the correct answer before you look at my solution.

Two final warnings. Do not try to define a non-alphanumeric character or string as a short form with the :abbreviate command. You probably will be able to do this -- the editor won't object -- but when you try to use this abbreviation, nothing will happen. And with either :abbreviate or :map!, do not put any metacharacter as itself into the long-form string. Even if you manage to get it into the string as itself, it will not go into your text that way.

What if you have forgotten what short forms you have set up, or are uncertain as to whether some may have been set up for you via a .exrc startup file? Well, you can query either tool ju st by giving its setup command without any arguments. Here are examples of those queries, with the responses you might receive from the editor:

:ab
cat   cat   felix domesticus
wolf  wolf  canis lupus


:map! 
{     {     ^[o^I^IThe End^[
}     }     ^[o^I^I-XXX-^[
~     ~     (more to come)^[

Note that each response line has at least three strings of printing characters, separated by whitespace. It's that second string in a line that is the short form; the string that when typed in will be replaced by the last string shown. (Yes, in every example line above the first string is identical to the second, but that isn't always so.) The last string is what will be inserted and/or executed.

So now you know what characters and strings will have to be quoted in when you want to insert them as themselves. And if one or more of those short forms is something you will be typing in so often that you can't spare the time to quote it in each time you use it, you can disable the metavalue for the rest of the present editing session. Just give the command name for the tool that uses this short form but precede it with ``un'', and as the only argument give the short form you want to disable. For example, here are the commands that will disable the first entry in each of the lists above:


:unab cat
:unmap! {

Command-Submode Macros

It's common that a text editor has a facility that lets a user create personalized commands, usually as macros built on existing commands. The Vi/Ex editor has four such facilities -- something for every need. While these facilities don't have the low-level programmability of mock-Lisp, they can accomplish a lot to simplify your editing, and you don't need to learn a programming language to use them.

I'll be discussing each facility (or family) in its own section below, because their structures are quite different. Nonetheless, you can often combine them to go od effect, by using a macro of one type to call a macro of a different type.

:map Macros

This is the editor tool that's closest to what most users think of as a macro facility. It uses the command :map as its setup tool, and the macros it creates operate when the user is in command submode of screen-editing mode. Otherwise it works just the way its very close relative, the :map! tool, works -- which I explained in depth in the first half of this tutorial part, above. Consider the three command lines below:


:map v :!wc -w %Control-M
:unmap v
:map

The first line sets up a macro that does a word count on the file I am editing, as of the last write to storage, whenever I type the letter v from command submode while I am screen editing. The second unsets that macro, so that a v command no longer does anything. The third displays a list of the :map macros that are currently in effect. All this should be transparently plain to readers who understand the :map! tool. Still, there are a few points worth noting that are particularly applicable to the :map side of the family.

Choosing a short-form for :map macros should not be difficult. Half a dozen of the printing ASCII characters and many of the control characters are not used as screen-editing commands or addresses. Hardly any strings of two duplicate characters (such as ``DD'' above) are in use, and most editor versions will let you map such strings. You don't need to avoid duplicating your :map! short forms because the name spaces are completely separate. That is, if you use a particular character or string as a :map short-form and also as a :map! short-form; for example:


:map }} :!wc -w %Control-M
:map! }} Control-[j0R

there is no conflict. The editor will allow both mappings, and will use the correct long-form based on the context; whether you typed }} from command or text-insertion submode. As the first example above shows, your command string can include any of the line-mode commands that can be invoked from screen mode, providing you begin each one with a colon ``:'' as you would when invoking it directly while in screen mode, and quote in a Control-M (the RETURN character) to terminate the command.

Suppose that you ran the two following setup commands, either one first:


:map Q 2dd
:map V 3jQ

The first command clearly provides that the Q command, which ordinarily is the command that takes you out of screen mode and into line mode, does not do that any more. Instead, it now deletes two lines, and you now have no way to leave screen mode without unmapping the ``Q'' character. But what does the new V do?

If you've left the :set command's remap option turned on, its default value, then the V drops down three lines and then deletes that third line and the one following. That is, when it comes to the ``Q'' character in that mapping, it discovers that ``Q'' itself has been mapped, and brings in the mapped value of ``Q''. But if you had previously run a :se noremap command, then the editor would not check for any mappings of the characters within a macro, and would use the standard meaning of ``Q'' when it executed the ``V'' macro. So then typing a ``V'' character would move you down three lines and then put you into line-editing mode. (Yes, that means that while you would no longer be able to execute the Q as itself directly, your macros could still access it!)

Buffer Macros

There are limits to the amount of macro text you can store by mapping it -- not as severe now as with earlier versions of the editor, but still somewhat confining. To remedy that, the editor offers a quite-similar tool with practically unlimited storage. It involves those buffers where you store text pulled from your file, for later reinsertion at various places. Specifically I mean the twenty-six buffers named ``a'' through ``z''.

From screen-editing command sub-mode, you can type an at-sign ``@'' followed by a letter of the alphabet, and the editor will take the contents of the buffer with that letter-name and execute it as a screen-mode command string. For example, if you have ``0d3w'' (without the quotation marks) stored in named-buffer ``k'', then typing @k will delete the first three words on the current line. After you start using this method in your editing session, there's an extra added convenience available: typing @@ will repeat the last such buffer command you ran.

To put a command into a named buffer, get the line or lines of your command into your file one way or another, then delete or yank them into the buffer of your choice, as by:


"p3dd
:ya m

to delete a three-line macro into buffer ``p'' and yank a one-line macro into buffer ``m'', respectively. You need not tell the editor that you regard the contents of a buffer as a command macro until you choose to execute it with a ``@'' command. In fact, you can use a buffer's contents both ways, executing it as a command at one moment and putting it back into your file as text the next.

One important difference from macros created by mapping: if you need a linebreak character in a buffer macro, don't try to quote it in. Instead, type it in the ordinary way, so that it forms a line break between two lines of your macro text. And don't break a line in your macro text for any other reason, because the linebreak characte r that appears there will be treated as a command character by the editor when you execute the buffer contents as an editing command string.

:source Macros

Line-mode commands have a macro tool in this editor, too. Of course you can insert most line-mode commands in the previous two types of macros, but this tool is dedicated entirely to line-mode commands, and can include even commands that can't be run interactively from screen mode via a preceding colon. The only line-mode commands that can't be run with this tool are the visual and open commands. With this tool, you set up your macros by putting their commands into one or more files, then invoke them with command lines like:


:so /u/myname/commands.1

Your command files should contain strictly line-mode commands, one per line unless you separate them within the line by pipe ``|'' characters, and should not have a colon before each command. The other restrictions depend on how you plan to invoke your macro files. Ideally you should give your source commands while you are in line mode -- then the above limitations are all you will face. But if you insist on invoking :source while in screen mode, there are two other limitations:

  1. Only the first line of your command file will execute. Due to the editor restriction against running multi-line line-mode commands while in screen mode, all lines after the first in your command file will be silently discarded.
  2. If your first command is not complete on the first line (for instance, an append is not), even that command will not execute. In this case the failure will not be silent.

Another Exercise.
So if you want to source in command files from within screen mode, it's a very good idea to create one-line command files. But there will be a few cases where multi-line command files will be a worthwhile thing, even when you may be invoking them from screen mode. Here's an easy exercise for you: come up with a specific case in which a command file that you may source in from either line or screen mode should nonetheless have more than one line. Of course there are multiple possibilities here, so don't be disturbed if the solution that occurs to you is not one of those I arbitrarily chose for my answer.

When you really get into sourcing, you'll be pleased to know that :source files can contain commands to call other :source files. This is the basis for truly modular editor scripts, and for a raft of rather tricky maneuvers. It also saves typing when you need to invoke a source file from screen mode, but the list of commands is simply too long to fit on one line: a single line in your initial source file is long enough to call a very large number of other source files, each with a single long line of commands. You will probably find that invoking nested :source files from line mode will turn off line mode's colon prompt, but you can turn it back on again via a :se prompt command.

Write and Read Macros

The Vi/Ex editor has tools for running some or all of the lines in the file you're editing through a program outside the editor, then using the transformed lines to replace the original lines in your file. It can also run a program with any or no input and insert the program's output in your file, or write some or all of your file lines as input to a program that may send its output anywhere.

And where is the macro capability in all this? Well, when you use these tools you are not limited to standard Unix utilities as your outside programs -- your own coding will do just as well. Compiled or scripted, one line or a thousand, in a standard language like C or Perl or in a specialized one such as Snobol; the rule is that if your Unix system will execute it, the editor can pass it over your text.

This tutorial is not going to get into writing these personal text processors, in any language, so I will only be explaining how to send your text in and/or out via editor tools. In the examples below, I will suppose you have a text-processing program named myhack that lives within your searchpath.

[Editor's note: One external program I use frequently reformats paragraphs into nicely looking text blocks that are easier to read. I use the program named reform, published on pages 320-321 in the first edition of the famous book Programming Perl by Larry Wall and Randal L. Schwartz. At first blush you may ask, why use such an external program when I can simply set Vi's wrapmargin variable? Of course, the answer is how do you easily reform paragraphs that are already ragged, say due to the problem Walter posed above (using find and replace to expand abbreviations, instead of expanding abbreviations using the built-in Vi abbreviation macro facility?]

Note that the command to execute the outside program should be typed as you would type it at your shell prompt, because it will be passed to the shell intact except for the addition of input and/or output redirection.

If you want to take some (or all) of the lines out of your file, use them as input to your outside program, then put the resulting output in place of the original lines, you can use either a line-mode or a screen-mode command to do it, as shown below:


:196,254 ! myhack -n6
!L myhack -n6
12!! myhack -n6
!/^CHAPTER/- myhack -n6

The line-mode command can be invoked from line mode, or from screen mode by preceding it with a colon. In either case, you give an address or address range, next the exclamation point, then everything following until you type return is passed to the shell as a command line. The line-mode command must have at least one address because there is no default address for this command. But the whitespace I show before and after the exclamation point is permissible but not necessary; I put it in solely for readability.

Screen-mode command form is the exclamation point as the command name, followed by the target address, then the outside command (with arguments and/or whitespace as would be required or permitted on your shell command line), ending when you hit the escape or return key. As with the c d y commands, you can type two consecutive exclamation points to send just the current line, and use a count to send that number of lines as shown in my third example command. The last example involves an extra escape character -- at the end of a search pattern address, whether / or ? based and including any + or - suffix, you must press the escape key before you start typing the outside command.

You're not limited to just one outside program at a time. You can pipeline two or more together as your shell permits, ordinarily with the ``|'' character. (Because a | character and what follows it will be passed to the shell, this editor command cannot appear in a line-mode command string, including a :global string, unless it is the last command in the string.) The final output of the pipeline is what will go into your file. And you can undo the effect of the outside command or pipeline, putting your file back the way it was, with a u command.

You may not want your text to make a round trip, though. You may want to send your text, as modified by your outside program, off to some other destination, or you may want to pull some text into your file that originated in your outside program, or was taken from some outside source. In these cases, use the line-mode commands that appear below:


:1,.w ! myhack -n6 > nufile
:217r ! myhack -n6 < oldfile

The first command above sends the initial lines from the file you are editing as input to your myhack program, and redirects the output to a file. It does not erase the affected lines from the file you are editing. The second runs your myhack program using the contents of another file as the input, then places the output in the file you are editing, right after line 217.

Both line-mode commands are shown with addresses, but they are not necessary. The default address for a :write command is the entire file; for a :read command, right after the current line. The space character just before the exclamation-point flag after each command is absolutely essential; without it you would get something greatly different from what you expected.

Usually there will be output redirection for the :write ! command, and input redirection for the :read ! command, but not always. For example, you may want to :read ! an outside command that generates a pseudo-random number, using no input at all. When you do need input or output, you can build the necessary redirection into your outside program or you can put the redirection on the command line as shown above, using your own shell's notation.

In The Next Installment of this Tutorial

I'll be putting the techniques I've taught so far to work, showing how to set up the editor for special purposes. Your suggestions on what special purposes to consider are welcome, of course. One purpose that is already in my mind is an arrangement of the editor for computerphobes: very simple, with beginner features such as ``stateless'' editing, and fortified against common user errors.

SIDEBAR: The timeout Function

The :set command's timeout option seems arcane in purpose and tricky to use, at least to some editor users. But it becomes pretty plain when you know why and how it actually works.

Basically, when the timeout option is on (its default state) and you type in a short form you've set up by a :map or :map! command, you must type the entire short form in no more than one second. If you miss that deadline, the editor will ignore the metavalue, and take the characters you've typed at their face value.

This odd requirement serves a purpose; preventing deadlock. As an example, suppose you have defined ``DD'' (without the quotation marks) as a macro via the :map command, and have turned off the timeout option. Now, while editing, you type a plain D command to delete part of a line. When the editor receives this single ``D'' it is uncertain what to do. Are you actually telling it to delete that partial line? Or are you starting to type in your double-D macro? The only way the editor can resolve this question is to wait and see what character you type in next. But if you are waiting to see the result of your deletion before you do any more editing, the mutual wait will last indefinitely. With the timeout option left turned on, the wait will only be a second or so before the editor acts on your D command.

One moral of this story is to leave timeout on unless you have a compelling reason to turn it off, and choose your macro names so that you can easily type them in within the one-second limit. If you are not particularly nimble fingered, or if other people may be using your editor macros, then for practical purposes this means either a single character or two repetitions of one character as in my example above. (Some fussy versions of the editor will refuse to map anything except a single character.)

Another moral is to avoid certain macro names, such as ``jj'' (again, without the quotation marks). The standard address j is one that you might want to type twice in rapid succession, to move directly down two lines without the trouble of reaching away from the central keyboard to hit the 2 key. But the user with a macro named jj had better not move down too quickly via that method, or he/she will accidentally invoke the macro of that name.

Finally, you should realize that the one-second count before timing out is not hair-splittingly accurate. The design of the standard Unix software clock means that the time-out interval may be a little less or somewhat more than precisely one second.

Back to the index