Personal computing history (part 6)

And, after a while, DOS was no longer king. Windows ruled.

And with Windows came Microsoft's Word for Windows. Version 6.

And then came macro viruses...

From the anti-virus vendors came much shouting "How do we parse these files? What is in these files?"

And from Microsoft came the grudging reply "We cannot tell you."

Not "Will not". "Cannot".

You see, back then Word files (and Excel files) were saved C++ object streams. To save a WORD file, you had C++ write the associated objects to disk. To read it back, you have C++ read the associated objects from disk.

So nobody had "designed" the file. It was just C++ objects.

If you use the right C++ compiler and objects, everything just works. Use something else...

So, the best that Microsoft (again grudgingly) could do was to provide a "reference" implementation of C++ objects to read/write an appropriate OLE file, and some documentation on the internal WORD objects (complete with bugs...)

So, what do you do with this? Well, the anti-virus vendors split into 2 camps:
      1) Use the reference implementation as the basis of the scanner
      2) Use the reference implementation to figure out the actual binary contents of a .DOC file.

The company I worked for went for option #2. I had much "fun" (for strange values of "fun") translating object streams into actual binary data.

But in the end, we knew what was in a Word file (or Excel file or...). Down to the bit. And could parse a Word file in ANY language...

Personal computing history (part 5)

So, I now have a pair of tools:
     VIMas a debugger
and
     CODEGEN as a disassembler.

What do I do with them?

Well, around this time, the company I was working for became involved with the first computer viruses.

And I had tools that would let me simulate running any software, and rip apart programs with ease.

Of course, some viruses tried to make it hard. They didn't want to be traced, or encrypted themselves, or even kept the parts of themselves that were not currently executing encrypted (i.e. decrypt the currently required subroutine before executing it, and re-encrypt it when done. This was the "Whale" virus). None of it mattered. One or two virses (like the "Whale" virus) made me create a "tweaked" version of my tools, but that was hardly even a speed bump.

And so I spent many years disassembling viruses and writing anti-virus software...

Personal computing history (part 4)

As I mentioned last time, the debugger I wrote has 2 instruction decoders. One of which creates both instruction string and executable information.

This turned out to be VERY useful.

Why?

Well, after a while, I also wanted to disassemble programs, not just debug them.

And I had an engine that allowed me to decode instructions and execute them. So I could know not only that a DOS call or BIOS was being made, I could determine what function was being asked for.

Add an algorithm to separate code and data, and you have a very powerful disassembler...

Personal computing history (part 3)

The one problem with interpreters (like VIM) is that they are slow.

In fact, VIM used about 400 instructions to interpret a single instruction. So your program ran 400 times slower than normal.

The early PCs and clones were running 8088 chips at 4.77Mhz. Since they weren't all that efficent (not like the single clock instructions of modern chips), you got less than 1 million instructions per second (MIPS). So, at it's fastest, your 1MIPS program was running at
          1000000 / 400 = 2500 instructions per second

Yah, it was slow.

This meant that all sorts of tricks were used to get any little bit of speed out of it that you could.

One of the biggest tasks of the interpreter was instruction decoding. Taking a byte stream like
          A3 34 12
and determinging that this was the instruction
          MOV [1234],AX

If you work with this for a while, you realize that you are much better off if you have two versions of the instruction decoder:
          1) produces readable strings and computer instructions
          2) just produces computer instructions

Most of the interpreters processing will be internal. You are not (usually) going to be watching every instruction. If you don't produce strings for the user (case 2), it is much faster.

If you are going to be producing strings for the user (for example, tracing a critical code section), you are not going to care (in general) how fast the routine is (the user can't keep up with the computer).

So, you trade code space for speed. A common trade-off.

You just have to be VERY SURE that both decoders are "in sync" so you can switch between them without anything breaking (or interpreting instructions incorrectly...)

Personal computing history (part 2)

So, why was this such a big deal?

Well, the best debugger is an interpreter. VIM stands for "VIrtual Machine", and is an interpreter for 8086 machine code.

With it's own memory space, so changing something important (like trashing the interrupt vector table) wouldn't crash the real machine.

Simulated periphirals (like interrupt controllers) and access to the actual machine hardware (keyboard, disk drive) so that it could do real-world simulations (like booting and running a different operating system the the one running VIM). And the ability to turn hardware access off, so you could protect your system from malicious software.

Breakpoint on data access, so you can see how a stray memory pointer is trashing your data.

Breakpoints were setup so that if executing an instruction would cause a breakpoint, the break occurred before that instruction executed (for example, if you wanted to see how memory location 1234 was being overwritten, you could set a breakpoint on 1234. Then, if executing the instruction

MOV BYTE PTR [SI],AL

would write to location 1234, VIM will stop before the instruction executed and trashed memory.)

VIM will also remember the last 2047 instructions executed, so you can figure out just how you got to this point.

Also, save/restore your debugging session to disk. So you can go home for the night and pick up where you left off in the morning. Or play "what if" on code execution, and go back to a known starting state any number of times.

So, I now had a tool that would let me do all sorts of neat stuff. And one that would become the basis for much of my work in the future (though I didn't know it at time time).

Personal computing history (part 1)

A long time ago, I wrote a lot of assembly language. I got very good at it.

Some years later, I was working for a company that did educational software. Back in the mid-80's a lot of PC software was copy protected. This was a major pain for us, as we had a $2000 program (back in the 80's this was even more of a painful cost than it would be now...) that broke when used on a faster (back then, 80286) machine.

So, my boss, knowing how good I was a low-level stuff, asked me to break the copy protection. No DMCA back then, I was free as a bird...

So, I started digging into the program with DEBUG. And taking notes...

After about 10 pages of scatterd notes with cryptic values written on them, I though to myself "Why am I doing this? I have a computer!" and so VIM was born. A month of effort got the basics working, and the copy protection was toast.

...to be continued...

Disk Rot...

Well, despite my warnings to others about software vanishing and keeping "old bits" around, I find that I've been bitten...

I'm doing some cleaning in the computer room (getting rid of old junk), and looked in a CD wallet that contains archives of my development work. The disks are succumbing to disk rot -- when I opened the sleeves in the wallet, they took the aluminum layer with them. Oops...

I think this was an interaction between that particular brand of CD and the plastic of the disk sleeve, since other CDs in the same wallet are fine.

Fortunately, I made multiple copies. On CD's from different companies. Stored in different locations and methods (CD wallets vs jewel cases).

I haven't lost anything, but that's just because I'm paranoid about such things. But not everyone is...