String objects

Strings in REALbasic are immutable — that is, you can’t write code to just reach inside a String and rearrange its contents. If you write code to change the text in a String, your program will always create a new String object and copy all of the text from the old String into the new String. That’s just how REALbasic Strings work. The whole point of a text editor is to allow you to change the text in a file at any time, so this operation should be efficient, and REALbasic’s String class is not efficient for this.

This is how you insert a new piece of text into a String:

Dim offset as Integer
//...
buffer = Left( buffer, offset ) _
    + newText _
    + Right( buffer, Len( buffer ) - offset )

Here is what happens under the hood whenever the above code is run: REALbasic creates a new “buffer” object, copies all of the data from the old “buffer” and “newText” objects into the new “buffer” object, returns the new “buffer” object, and marks the old “buffer” object as empty space to be reclaimed by the operating system. This isn’t so bad if “buffer” is always a small String, but what if the String is the text from a 200KB file? Then every time the user types something, the editor has to borrow a 200KB block from the operating system to build the new object, then copy 200KB of data from the existing buffer into the new object, then let the REALbasic framework “collect” the old buffer so the OS can have its 200KB block back. This takes up both CPU time and memory. Surely there is a way to insert new text that takes up less?

There is, of course. Instead of keeping a file’s contents in memory as a single big long String, the editor could split the file’s contents up into many short Strings and store references to those Strings in an array. Inserting a String into an array actually inserts only the location of the String, not its contents, so insertion is quick. Now the code to insert new text into the buffer looks like this:

Dim index, offset as Integer
Dim leftPart, rightPart as String
//...
leftPart = Left( buffer( index ), offset )
rightPart = Right( buffer( index ), _
    Len(buffer( index )) - offset)
buffer.Insert index, leftPart
buffer.Insert index+1, newText
buffer.Insert index+2, rightPart

The code is longer, but it runs faster because there is less going on under the hood: New text is inserted by splitting only a single segment of the file and inserting the new text in between the parts. It isn’t necessary to append the parts together into a long String again, because the array keeps the parts in order.

This clearly alleviates the problem, but it does not solve it: To insert new data into the buffer, you still have to create a copy of data you already have.

In addition, there is another problem: Searching the buffer becomes very awkward when the buffer consists of lots of fragments in an array instead of one great big long string. So in this case the editor would probably append all of these items into a single string when a search is about to be performed. Again, the data in the many segments of the buffer would need to be copied into a separate String object just to perform a search. As before, this takes time that the editor could spend more productively doing something else, and it doubles the editor’s memory requirements because it has to make a second copy of all of the text data. There has to be a better way.

(Side note: Theodore H. Smith offers a REALbasic plugin called ElfData. The plugin offers special string classes (ElfData and FastString) that let you split and recombine strings much more efficiently than REALbasic’s String class, mostly by cutting down on the amount of data copying and memory allocation that has to be done at runtime. However, the problem (from my perspective) is the same: The ElfData and FastString classes still work like String objects. To insert text into a long string, you still have to break the long string apart and copy all of the text from the various pieces into a new object, and you need space in memory for two copies of your data to do that. A text editor would use memory more efficiently if it could avoid creating a second copy of the text altogether.)

MemoryBlock objects

Unlike String objects, MemoryBlocks in REALbasic can be updated in place: You can insert new text into a MemoryBlock without the REALbasic framework recreating the whole MemoryBlock each time. Further, as Charles Yeomans points out, if you use a BinaryStream object as the front end for a MemoryBlock, you end up with a buffer that acts like a file: you can “seek” to a given position anywhere within the buffer and read or write data at that point, and if you write data beyond the end of the buffer, then the buffer will be made large enough to hold the new data.

When you write ten bytes of data into the MemoryBlock, then ten bytes of space within the MemoryBlock, beginning with the current position, will be overwritten. If the current position is ten or more bytes from the end of the buffer, then no data in the MemoryBlock is copied or moved, so the write operation is extremely fast. However, if the current position is less than ten bytes from the end of the buffer, then a new MemoryBlock has to be allocated, and the data in the old MemoryBlock has to be copied into the new one. If you use a BinaryStream object as the front end for a MemoryBlock, as explained in Charles Yeomans’s article, the new MemoryBlock will be twice the size of the old one, so the need to move data to a new MemoryBlock should not arise often.

But if you want to insert text into a MemoryBlock, you don’t want to overwrite the text already in there. So it would seem that you’d still need to copy existing text to a new location to make room for the new text. However, if the text to be overwritten is actually considered garbage and not part of the text you want to keep, then you can “insert” your text extremely quickly. This is the basis for the gap buffer that is used in many text editors. A gap buffer is a text buffer with three regions: the text before the current position, the text after the current position, and the “gap” in between. The gap is of course just a stretch of nonsense text that isn’t considered part of the main text and that is present to be overwritten with whatever new text the user types or pastes in.

If you need to search the text being edited, the search might be easier if the gap is removed. This of course requires copying text within the buffer to remove the gap. However, no new objects are created, so the editor would not need to borrow more memory from the operating system to perform this operation.

MemoryBlocks are best

Clearly, if you want to write a text editor in REALbasic, it is better to use a MemoryBlock to store the text being edited. Inserting and deleting text will be much faster and will require less memory than if you were to try to store the text in String objects.

I like REALbasic. The language is easy to learn, easy to write, and easy to read. The REALbasic IDE provides lots of pre-built controls for building your program’s user interface, but you can also create your own controls from the ground up. If you have the Professional version of REALbasic, you can write your program once and use it to build executables for Windows, Mac OS X, and Linux all at once. And if some part of your program should prove to be unacceptably slow, you have the option of writing a REALbasic plugin in C++ to speed things up.

What’s good about REALbasic?

There is a lot to like about this language.

• The language is easy to pick up. It’s BASIC, after all.
• The language is object-oriented. You can create your own classes, or subclasses of existing classes. You can define interfaces as in Java and then create classes that implement those interfaces.
• Objects are reference-counted and are deallocated automatically when no other object points to them.
• Some of the features the language has include:
  • Basic data types: Integer, Single, Double, Boolean, Color, Date, Currency, String, Array, Variant, Dictionary (actually a hash table), MemoryBlock. REALbasic has many functions for dealing with variables of each type.
  • Low-level data types for working with binary files or C functions in a library or the operating system: Int8, Int16, Int32, Int64; UInt8 (Byte), UInt16, UInt32, UInt64; Ptr; CString, PString, WString.
  • Literal syntax not only for hexadecimal (&hFF), binary (&b11101101), and octal (&o137) numbers, but also for colors (&cFF6F00 for orange) and Unicode codepoints (&u2014 for an em dash).
  • Delegates are essentially typesafe function pointers. A Delegate can point at any function (or object method) with a specific list of argument types and a specific return type — for example, any function with a first argument “As Integer”, a second argument “As String”, and no other arguments, and that returns a Boolean. You can use these at runtime and change what your program does in response to a keystroke or a button press.
  • Support for exceptions. You can define your own exception classes, use the Raise statement to throw an exception, and Try…Catch…Finally to catch and handle exceptions.
  • Declare lets you call subroutines or functions in DLLs (on Windows) or shared libraries (on Mac OS X or Linux).
  • Assigns lets you set up a Sub to be called with “theSub(a, b) = c” instead of “theSub(a, b, c)”.
  • Extends lets you set up a Sub or Function to be called with “a.theSub(b, c)” instead of theSub(a, b, c)”.
  • #If…#Endif allows conditional compilation. This is most useful for allowing or “commenting out” code depending on whether (1) a constant is defined, or (2) the running within the IDE, or (3) the application is running on Windows or the Mac or Linux.
• The IDE lets you build your GUI by dragging controls onto a window and setting their properties, just as Visual Basic does.
• If the controls supplied with REALbasic don’t do what you want, you can subclass an existing control and write event handlers to change how it works, or you can even subclass the Canvas control and create your own custom control from the ground up.
• REALbasic comes with classes for all sorts of things — standard dialog boxes (for picking folders, files, or colors), regular expressions, XML, databases, shell commands, graphics (lines, shapes, text), networking (including sending data to and receiving data from websites), conversion between text encodings, cooperative (not pre-emptive) threads, sound, editing video (via QuickTime), etc.
• There is even a command to make the computer speak the text you pass to it. (This works on Windows and on the Mac, but not on Linux.)
• RBscript is a scaled-down version of REALbasic that you can compile into your application so that users can run scripts. Even better, these scripts are compiled, not interpreted, before being run.
• REALbasic (the Professional edition of it at any rate) is cross-platform — you can build applications for Windows, for Mac OS X, and for Linux from a single project file.
• REALbasic code is compiled into native machine code, not interpreted.
• You don’t have to pay any royalties for distributing your applications.
• REALbasic Personal Edition for Linux is free. If you have a Linux machine, you can write and test REALbasic code on it and not pay a cent to REAL Software until you’re ready to build executables for Windows or the Mac.
• If the code that REALbasic produces isn’t fast enough, you can write a plugin for REALbasic in C++ containing code to alter the contents of your REALbasic objects.
• Many developers have developed their own plugins and controls and offer them either for sale or free of charge.

What’s bad about REALbasic?

There are some drawbacks to REALbasic, though none of these are showstoppers for me personally.

• Every version of REALbasic other than the Personal Edition for Linux is commercial software. If you want to compile for any platform other than Linux, you have to pay up.
• REALbasic’s IDE can be constraining at times. You can only see one function or one class method in the IDE at a time. People accustomed to seeing two or more function definitions at once in a text editor may chafe at this.
• REALbasic’s controls reflect the lowest common denominator of the equivalent controls on the three platforms. For example, REALbasic offers a TextArea for letting the user edit styled text, but the TextArea does not have built-in undo; you have to supply that.
• REALbasic’s executables are large. A do-nothing GUI application is around 1.5MB because REALbasic compiles a large part of the runtime framework into the application. Even a do-nothing console application is about a megabyte in size.
• REALbasic does not support Cocoa (although a new release that does support Cocoa is supposed to be released by the end of the year), so any Mac OS X application built with REALbasic right now relies on Carbon. (This is one reason why developers who develop only for the Mac often prefer Objective-C and Xcode over REALbasic.)
• REALbasic currently produces only 32-bit code, not 64-bit code.
• Documentation for the plugin API is spotty, and actually writing the plugin appears to be something of a dark art. (I might try gathering this information and writing up a how-to on building a plugin with gcc in a later post.)
• REAL Software sometimes adds new features to REALbasic and releases them before they are ready. (The latest release, 2009R4, has new report-building functionality that is apparently not yet ready for prime time.)

Good enough

Every programming language and package involves tradeoffs. REALbasic, like every other programming language, has its good points and its bad points. I like its good points and I can live with its bad points, so I’m going with REALbasic.

Links

• Markus Winter gives his take on REALbasic.
• Thomas Tempelmann warns those new to REALbasic about some annoyances to watch for.
• Many developers who develop only for the Mac prefer Cocoa and XCode over REALbasic.