Getting published, and other bad ideas

Posted January 27, 2008 by Chris under

It’s been a weird couple of weeks. I sent in a short story to a publisher, for the first time ever, and the lead up to it was an interesting experience. Interesting in a purely scientific sense. I didn’t enjoy it.

Several years ago, on the advice of a friend, I sat down to write something that was going to be purely for me. A story that I could work on just for fun, that I would never show to anybody; a story where I could be free to try anything that came to mind. To my recollection, I sat down at around 1:00 in the afternoon, one Friday in the summer, to write just that—something just for fun. And when I got up around 6:00, that night, I had a first chapter written. 3000 words. It was the easiest thing I have ever written. And I had a ton of fun doing it.

And you know what? It was the best thing I’ve ever written, too.

Of course, since realizing that about it, I’ve been unable to write a single new word on it. I seem afraid that I will fuck it up. These days, when I sit down to work on it, I’m not trying to have fun—I’m trying to write something “great”, something that lives up to the writing in that first chapter.

It doesn’t seem to be a very effective strategy for me.

The short story I submitted last week was Dajoën, which some of you have read. I did the first version of it for a contest in my writing group. It was a short short story contest—stories were limited to 2000 words. And I had this little idea—to write a story about the parting of two friends.

Well, the very first version of it was . . . horrid. But, fortunately, I started again, and wrote the first official version, which won the contest. When I write from first person, I try to be the person I’m writing as, and for that character, he’s talking about a loss that was very important to him, and which he is still grieving. To be honest, I don’t know how much of that made it to the page. Probably not as much as I’d intended.

Anyway, the feedback on that version was very good—as far as it went. The general consensus seemed to be that the story was too slight—that it needed more to be really effective. And so, in the three years since, I have rewritten the story—from scratch—five times. Three of those versions were . . . horrid, and have never seen the light of day. The most recent version was written last weekend, with the intent that that would be the version I would send to the publisher.

And here’s the problem: of the people who’ve read all the “good” versions, the general consensus seems to be that the original is the best. It seems that every time I set out to improve the technique of the story, I do it at the cost of its heart.

So, after 3 years of rewrites, I sent the original version to the publisher. It was one of the hardest decisions I’ve made. Because, after so long trying to fix them, all I can see are its flaws.

I’m not even sure why I want my stuff published. It’s certainly not for the money: if Dajoën sells to this publisher, my advance will be $38. Of course, if I’m being totally honest, the same little part of me as usual wants recognition, for something I’ve made. And while that’s not a huge factor, it is a factor. But, thinking about it while having all this angst for the last week, I think the publishing matters because it is a route to getting readers. I write to share something—of myself, or of my experience—with other people, something that they might find useful. I want to say with fiction the things I can never seem to say with truth, when it counts. It’s the statement that “I know how you feel, and you’re not alone.” It’s to say that to somebody else, in a way that they’ll hear, that I write the stuff I do.

The thing is, I’m not sure I need a publisher for that. I wrote an article, a few days ago, related to a software project I’m working on, that I told 5 or so people about. To be honest, I didn’t expect most of them to read it all of the way through: the article was long, and I didn’t think the writing was very effective. So it came as quite a shock, two days later, when I looked at my web server logs and discovered I’d had over 80 readers of that article in one day. (I hadn’t realized that you could get stage fright, after the fact, but I did.) But it’s got me thinking: if I don’t write for the money, and the process of “perfecting” something for an imagined editor at a publisher causes me so much grief—and may actually cause the writing to get worse—then why the hell am I bothering? I have a website . . . .

The truth is, my writing seems to be at its best when I relax and just write. And every time I’ve ever rewritten a story, it’s lost that part that made it good, in the attempt to get something else. But, even though I know that, as long as I sit on a story—as long as I don’t consider it “done”—I can’t seem to resist the urge to fuck with it.

So I haven’t decided yet, but I’m seriously considering making Dajoën my first and only foray into professional publishing, at least for the foreseeable future. That, instead, I’ll write stuff and post it to my website as soon as it is finished. No rewriting, no perfecting every story to death. No angst about flaws. I’ll work on the projects that interest me, and stop worrying about which ones are “good enough” that they should be held for a real publisher.

I want writing to be fun again. I want to stop thinking about technique, and start thinking about story and characters instead. And you know . . . I have the strongest hunch, that if I do that? The technique will take care of itself.

Thanks for reading.

Why bother creating new programming languages?

Posted January 23, 2008 by Chris under

I know that at least a few of you are probably asking that question. And I guess it’s a fair question—programming languages take a certain level of skill to design, and a certain level of expertise to implement. Truth be told, there seems to be a common perception that language designers rate up there with brain surgeons and rocket scientists on the nerd scale. And, to make matters worse, programming languages have a bad reputation as being hard to learn.

So, if only rare developers can create them, and using custom languages eliminates the “commodity” status on programmers, why would you even consider creating your own?

Well, even if we accept those two points as truth, the answer is still simple: with the right programming language, you can get a lot more done.

A contrived example

Consider this program describing a common activity:


   activate neurons 9388908 through 9388940
   wait 0.0002s
   activate neurons 9700938 through 9800838
   activate neurons 3992893 through 3992897
   wait 0.0001847
   deactivate neurons 9388908 through 9388912
   deactivate neurons 9388908 through 9388912


   (40 or 50 *billion* instructions skipped for space reasons)


   wait 0.0827053
   deactivate neurons 9388908 through 9388912

Now, consider this same example in a higher-level language:


   hold the bat <here>
   bring it up to your shoulder like <this>
   watch for the ball
      as it approaches, swing the bat down and forward into line with the ball
         follow through

Okay, okay—there is a lot more to swinging a baseball bat effectively than either of these two examples show, but I think it demonstrates the fundamental differences between low-level programming languages and a high-level ones:

low-level languages high-level languages
are very precise are very terse
make few assumptions rely on assumptions
can be used for almost anything are very specific to a single problem space

To me, the value of new programming languages is not in offering different syntax for the same thing. That’s not what I’m talking about, here. If the primary reason you want to write a new programming language is because you want to use { and } to delimit lists, instead of [ and ], or because you think % is a better address-of operator than &, you and I haven’t got much left to talk about. ;-)

The value in building new programming languages is in offering better, richer, more expressive ways to describe a solution to a problem, ways that shift the focus away from the “trees” and onto the “forest”, instead. So we can all get more done.

I don’t know about you, but if I had to teach someone to swing a baseball bat, I know which language I’d choose.

A less contrived example

My present paying job involves supporting a large networked information system. The underlying software system—involving a half-dozen major components—is off-the-shelf, but has been heavily customized through data, configuration, and custom code. There are two complete environments—one for testing, one for production—and each environment contains about a dozen machines. All services within each environment have redundant copies on separate machines, to ensure maximum uptime.

When new customizations are to be deployed into the system, it generally involves copying files around the network, and running up to hundreds of commands on up to a dozen machines, all in a particular order determined by exactly what is being deployed. Some changes require service restarts, and some service restarts must cascade to other services.

Needless to say, this kind of thing involves a lot of error-prone work, if it’s done by hand. And getting it back out again—should the need arise—often proves even more difficult, as the undeployment process is generally less well tested (being complex, and not on the “critical path”; at least, not until it becomes critical).

So, to address this situation I built a deployment system that leverages several “little” languages to address different aspects of the problem, and brings them together to provide completely automated deployments and undeployments. And, unlike the manual undeployments of the past, the automated ones are reliable.

I had three goals in mind when designing the control languages for this system:

  1. minimizing the work necessary to build a deployment script (the most common activity)
  2. ensuring the effects of every deployment script could be completely reversed without requiring planning or additional input from the script writer (so it couldn’t be screwed up)
  3. isolating the deployment script writer from the details of the system topology (which varied from environment to environment, and could be changed at any time)

These design goals are important for more than just usability reasons: if the scripting language requires a lot of thinking, or a lot of work to use, errors will occur more often. And, in this instance, errors directly impact the reliability of production systems, and overall uptime.

So, the finished system provides one little language for writing deployment scripts, a second for describing the system topology and component relationships, and a third for controlling the deployment system itself.

The deployment scripting language provides primitives that do atomic units of deployment (things like: create a policy, delete a policy, update a filesystem directory, modify a particular configuration file in a particular way, etc.). You write the deployment script as if all of the underlying systems exist on one machine, as if nothing ever goes wrong, and as if nothing ever needs to be undeployed. You simply put the directives in the right order for their inter-dependencies, and provide them with whatever data they need.

When the time comes, the system reads in the deployment script, combines it with the topology and relationship information from the configuration file (the second language) and does all the work of getting files to the right place at the right time, taking backups of the data that’s about to change, generating the necessary shell commands, restarting services, and monitoring for errors. And, should the deployment fail, the undeployment command takes the exact same deployment script, and uses the configuration information and the backups to figure out how to reverse the order in a way that respects dependencies, and undoes everything that has been done.

As a result of this language-based approach, a reliable deployment script can be built in a couple of minutes, and freely deployed and undeployed in any configured environment in the minimum time. And by separating the concerns of system topology and primitive operation from the specific deployment script, a large pool of risk is removed from individual updates. Once the configuration of a system is correct, any correct deployment script will run correctly on that environment. And once a new primitive is known to work properly, it will work properly in any deployment script that uses it.

Could these benefits have been achieved without developing new programming languages? No. In the end, the solution described here is a language-based solution. The details could have changed—the deployment script and system configuration could have been written in XML instead of a custom language—but, from a conceptual standpoint, you’d still have had the same two languages. The only difference would have been in the amount of noise the programmer would have had to deal with when reading and writing those languages.

The truth about programming languages

The truth is that the points I started out with are both myths: designing a programming language does not require an uber-nerd; and learning a new programming language doesn’t require one, either.

The truth is, completely normal, approachable, friendly programmers make new programming languages every day. When somebody designs a new XML structure to communicate something from one application to another, that’s a programming language. When somebody creates a user interface that allows a user to control an application, that’s a programming language.

And the value of these high-level and domain specific programming languages is that they let you focus on a specific aspect of a problem, in terms that are natural to that domain. Describe to me in English how you would deploy this set of updates into the system. Don’t worry about how many instances of the service exist in this particular environment, or what machines they are on; don’t worry that any time you update a policy, you need to take a copy of the old one so you can undeploy later; don’t worry that in the test environment, restarts can take services completely down, but in production, service must be maintained. Give me the high level details, formalize it a tiny bit, and here’s your deployment script. Assume the system can figure out the rest. Please!

That’s the power of a language.

The cost of generality

Programming languages like Java and C—and all of the “commodity” general-purpose programming languages, in fact—are low-level languages. They are generally useful specifically because they demand you provide so much detail: by shifting the burden onto the programmer, they can be used to solve just about any problem. But by assuming so very little, such languages require programmers to write tons of (often repetitive) code, just to get anything done.

Low-level languages do, of course, generally offer a way to package runs of logic into re-usable chunks (Java’s class libraries, for instance). These facilities allow you to leverage the work of other programmers, and can certainly be very powerful. But their “shape” is ultimately constrained by the language they were designed for. It takes new languages to allow new shapes, and to make them feel natural enough to use. Ruby’s closures define a new shape, and they are ubiquitous in Ruby code. The inversion of control completely transforms how you think about and write Ruby code. Java’s anonymous classes, OTOH, try to offer the same services within the existing shape of the language, and inflict such pain in doing so, that they get only limited use.

New languages allow new ways to think, and that is often exactly what is needed to make leaps in productivity.

Conclusion

I could go on. But I won’t. ;-)

Here’s my point of view: computers are good at doing boring, predictable, repetitive stuff, and we aren’t. And for anything but the simplest of problems, languages are a necessary tool for shifting that burden from people to computers.

It all comes back to Norman’s Law of Conservation of Complexity: give people better tools, and they’ll do more with them.

It’s time we had better tools.