Marks are something that my text editor will have under the hood; they are not a feature that would be directly visible to the user. Specifically, a mark is a chunk of data that is associated with a span of text (typically a single word or phrase) within a file being edited. Marks are not saved with the text as part of the file, but the file, while open, could have thousands of marks associated with various parts of the text. Each mark remains associated with its span even as the text is being edited. Marks would be useful for implementing syntax highlighting, bookmarks, and other features that a modern text editor would be expected to offer.

Each mark belongs to a track, which is conceived as being of the same length as the main text. Each point on a track corresponds to a character position within the main text. Each mark takes up certain points on its track, so each mark corresponds to the characters at the corresponding positions — the mark’s span. Each point on a track is occupied by no more than one mark, so marks on the same track cannot overlap. Thus, if a span of text is to have two marks, the second mark has to belong to a separate track. The editor can generate as many of these separate tracks as needed.

Here is an example involving three tracks of marks:

Track 1: art noun adverb_ verb_ noun art adjective noun
Track 2: U                      U                      PB
Track 3:    O    O       O     O    O   O         O     M
Text:    The girl quickly found Fido the yellowish mutt.<

(Note that the less-than sign at the end of the text represents a newline character.)

Take a look at each track in this example:

• Track 1 marks each word, each contiguous sequence of letters, with its part of speech: article, noun, verb, adjective, or adverb. Each mark in this track takes up the length of its corresponding word.
• Track 2 assigns a class to each character other than a space or a lowercase letter — “U” marks each uppercase letter, “P” marks each punctuation character, and “B” marks the mandatory-break character at the end. In this case, each mark takes up no more than one character.
• Finally, Track 3 marks the characters where the line can be broken on a wordwrapped display: “O” marks optional-break characters like spaces, while “M” marks mandatory-break characters like newlines. Again, each mark corresponds to a single character.

Note that each mark has a name. This name in effect assigns a class or category to the mark’s span. Track 1 marks some words as nouns and others as articles; Track 2 marks some characters as uppercase letters; Track 3 marks each space as an optional-break character to make wordwrap easier to implement. Mark names allow the text editor to treat different spans in the same way if they have the same mark name, but differently if their mark names are different.

(Another way of thinking about this, if you’re familiar with object-oriented programming, is that a track corresponds to an object property (part of speech, character type, line-break type), and a mark corresponds to a value assigned to that property.)

You can see how this would be useful in the bowels of a text editor. Marks would be useful not only for marking the locations of line-break opportunities or of different types of characters; they would also be useful in syntax highlighting (marking the locations of various classes of words or phrases) and for spellchecking-as-you-type.

Marks would likely be most useful for marking up just the text that is about to be displayed on the screen — finding where each line of text can be wrapped, determining the “classes” of words so that the editor knows what font and style to use when drawing the text, finding misspellings, and so on. Marking up the rest of the file would be a time-consuming chore that would generate a lot of data that would then have to be stored somewhere in case it is needed later, so this kind of busywork is best avoided.

Implementation

The text buffer contains nothing but text from the open file. Loading and saving files is simpler if the buffer contains only the text loaded from or saved to files. Search and replace is also simpler if the buffer contains all of the data, and only the data, to be searched. (Syntax highlighting relies on searches in the background, so searches need to be fast, and they are faster when they are simpler.) So marks and tracks are stored elsewhere.

Marks never overlap within a track, so a track could be represented as an array of short records, where each record contains just a mark name (as String) and a span length (as Integer). If the mark name is blank, then the span is just unmarked text. So the data for Track 1 above would look like this:

Name       Length     | (Corresponding
as String  as Integer |  span)
---------  ---------- | ---------------
article        3      | "The"
               1      | " "
noun           4      | "girl"
               1      | " "
adverb         7      | "quickly"
               1      | " "
verb           5      | "found"
               1      | " "
noun           4      | "Fido"
               1      | " "
article        3      | "the"
               1      | " "
adjective      9      | "yellowish"
               1      | " "
noun           4      | "mutt"
               2      | "." + EndOfLine

I’m storing lengths, not offsets, to make it easier to keep marks and spans together when text is inserted or deleted. If text is typed in the middle of a span, then all that needs to happen is to increment the length of the span within the mark’s record. If I need to find the mark that corresponds to a specific offset within the text, I just keep adding span lengths until the sum reaches the offset.

I’ll have more to say about this later. I’ll need to spell out exactly how this information is laid out in memory. I’ll also have to figure out how I want this information saved into temp files and then reloaded when the mark information is needed. But that’s for later. Right now I just wanted to get something posted.