DBActive

About a week ago Marten send me email pointing to his article published on Jays Blog (Come on Marten, it is time for you to get your own blog). I should have replied much earlier but only found time to do that now. So here is my list

1. Be Pluggable

Unlike many OpenSource projects MySQL was single chunk of code and for years the only way you could officially extend it was using UDFs which was very limited. Compare this with other OpenSource projects such as PostgreSQL (plugable indexes etc), Apache, PHP or Linux Kernel. Yes in MySQL 5.1 the situation is changes - now there are plugable storage engines (something even PostgreSQL does not have) as well as Full Text Search parses but there is very long way to go before you could do any significant functionality ourside of storage engines as plugins.

Two aspects why I’m especially interested about it is Sphinx which is currently implemented as storage engine for MySQL but this role does not suite it well as well for our performance optimization works - we would like to get much more stats from MySQL and it would be great if we could write these as plugins instead of patches.

2. Be Scalable

I’d like to see MySQL Scaling to large amount of CPUs for all storage engines all operation systems and all workloads. Well this may be too much of the wish but at least it should happen for most common cases. Innodb scaling issues are the most common problem (and patches currently available only fix part of it) but far from being only problem. There are fair amount of issues with MyISAM and generally on MySQL level which needs to be taken care of. Many of these issues were known for years. The good thing is now with multi-cores coming even on laptops these issues can’t be under carpet any more and they are getting attention and gradually being fixed.

3 Be Distributed
One of the most common buzzwords used together with MySQL is Scale Out, but to say the truth MySQL does not provide much of the functionality for scale out itself, besides its fast and simple replication. You still would normally need to manually chop your data to the multiple databases and handle data merging on application together with high availability and all kinds of problems. The dream for many MySQLers is to be able to get bunch of boxes and use them same as single MySQL server with improved performance and high availability. Of course there is MySQL Cluster but it does not get where quite yet having very particular performance properties and operating requirements. In fact I do not think MySQL Cluster would ever be that thing as it provides too strong guarantees. It is same as with MySQL Replication - the reason it is fast and has low performance overhead on the master is at large extent because it is asynchronous. To offer great performance on distributed systems, especially if they are connected via not very fast network you need to come up with different consistence guaranty and semantics… and believe me most of Web applications could live with it. They anyway read stale data from the slave or memcached manually relaxing data consistence requirements.

But honestly most parts are OK - over time you develop practices and tool set to deal with multiple nodes nicely even if you do not have much support from MySQL to do so. What is really bad is Parallel query. Reporting with group by over 10.000.000 of rows in real time is one painful thing to do requiring complex application programming to do efficiently in distributed manner. Of course there are summary tables and other tricks we use when such need arises but we still can’t deny - MySQL does not handle complex queries well, being able to use only one CPU effectively on single node and no true federation support.

4. Be Solid
MySQL has very good stability track record, with exception of MySQL 5.0 it was typically safe to run it in production well before it was stable (I remember we had successful production on 3.23 alpha). However many of new releases score pretty poor on the scale of being Solid.

By by being Solid I mean Solid functionally complete features which work as uses intuitively expect them to work instead of having a lot of little known documented (sometimes) limitations. I’m saying about Prepared Statements in MySQL 4.1 which came without query cache support but even worse not all of statements could be prepared not to mention other issues. I’m saying about SubSelects which were added in MySQL 4.1 but you still could not run (and still can’t) most of your Oracle/DB2/MS SQL Queries on MySQL because subqueries optimizer weakness. I’m saying about XA which were released in MySQL 5.0 without consistent work with MySQL close Partner - Innobase to ensure Group Commit still works in 5.0 and so we do not get major regressions. I’m saying about Stored Procedures which were released without proper development and debugging support. These are just few examples.

Of course you could say “release early release often” is a good strategy and at large extent I would agree but MySQL is not released often any more. There is relatively long release cycle with great deal of planning and it looks like no features go in stable release just the day before it is released. With such release management I would expect MySQL to try to be a lot more consistent as such approach means if something is inconsistent it stays where for very long time. For example Query Cache support for Prepared Statements is only fixed in MySQL 5.1 and Most of other items are still open.

Well may be this is remainder of wild first VC-backed years of MySQL and not things are getting straightened up. I know on a lot of listed issues the work is going on and it just takes time to get them all fixed.

5. Do not forget about the roots.
MySQL has gained its popularity as database for Web but web users do not pay much because they are OK with GPL and often have smart guys inside who can figure things out and even patch MySQL if needed. So MySQL went to chase Enterprise market and develop enterprise features pushing back a lot of features which Web users would like. These features are often small, typically not SQL Standard compatible but help you if you’re web developer a lot. In my Early years in MySQL I have created more than hundred of such little projects suggested by MySQL customers and myself based on long MySQL usage for Web. Later I stopped adding these because I have not seen much traction for these tasks as well as of similar tasks created by other guys.
These would include support for query timeouts, parallel query execution, CONNECT BY support and various performance related features.

I’m not sure if MySQL got back to spend significant resources on taking care of Web guys needs but at least we have now Community tree and Web guys can take care of themselves if they want to. Jeremy Cole’s patches were accepted, our patches and I guess Google patches are in the queue which is very good sign. There is still fair amount of work to do to straighten up this process so good patches can quickly appear in community tree. Once this is done I think amount of contributions will increase a lot.

In a context completely unrelated to computers or my work, someone gave me a copy of one of those “word a day” calendar entries for the word “mortmain”. You can see the fuller definition here, but one definition is:

The oppressive influence of past events or decisions.

Only after I tacked it up in my office did people start pointing out to me the irony.

I”m reading two very good management books right now.

The first is a classic: Up the Organization, by Robert Townsend. Apparently this book first came out in 1970, was widely admired, and slowly fell off everyone”s radar, until Wiley republished it last month. Townsend ran Avis back in the day, and when you start to read a management book written in the 1960s, you expect to find secretaries, two-martini lunches, executive golf club memberships, etc. What you find instead is rather refreshing even by today”s standards. On Mergers:

If you have a good company don”t sell out to a conglomerate…. Conglomerates will promise anything for your people… but once in the fold your company goes through the homogenizer along with all their other acquisitions of the week, and all the zeal and most of the good people leave.

PS for Y Combinator kids: Don”t be smug because you think that conglomerates went the way of the dodo. “Conglomerate” is just an old word for what you call “Yahoo, Microsoft, and Google.” Oh and Condé-Nast.

Anyway. Townsend on management consultants:

[They] waste time, cost money, demoralize and distract your best people, and don”t solve problems. They are people who borrow your watch to tell you what time it is and then walk off with it.

It sounds like a cliché, right? Townsend probably invented that cliché, boychik. And it”s still true, and the jibe at Booz Allen later on in the book is still 100% on the money.

Even better is his disdain for marketing departments:

Marketing… is the name of the game. So it had better be handled by the boss and his line, not by staff hecklers. Once or twice a year for three or four days the boss takes ten, twenty or thirty of his key people… away to some secluded spot. On average they spend twelve hours a day asking unaskable questions, rethinking the business (What are we selling? To whom? At what prices? How do we get it to him? In what form?), four hours a day relaxing and exercising, and eight hours a day sleeping. It”s hard work. But more good marketing changes will come out of such meetings than out of any year-round staff department of “experts” with “marketing” signs on the door.

Boy, I sure wish I had learned that one a few months ago. Two years ago, Seth Godin wrote essentially the same thing.

Anyway, that”s just a few of the M”s. The whole book is full of great advice like that, albeit focused on larger corporations.

If you”re looking for something a little more, er, contemporary, Michael Lopp and his alter-ego Rands have just published Managing Humans: Biting and Humorous Tales of a Software Engineering Manager, which originated with some essays on his excellent blog Rands in Repose. (You can do that?)

Lopp has worked at Netscape, Borland, and Apple. He”s the quintessential Silicon Valley middle manager. I hope he doesn”t find that term insulting: he”s probably the best Silicon Valley middle manager there is. He”s brilliant, charismatic, and a poet-philosopher, and I could imagine no better boss.

You”ll find that an awful lot of his book is about managing managers, big company politics, and the human side of getting technical teams to work together. And he has a style quite his own. You can get a taste of it from his classic Incrementalists & Completionists:

What was intriguing about my email repartee with the co-worker was that we weren”t disagreeing about whether or not we should do something about the problem. We”re arguing about how much we should do. The disagreement reminded me there are two distinct personalities when it comes to devising solutions to problems: Incrementalists and Completionists.

Incrementalists are realists. They have a pretty good idea of what is achievable given a problem to solve, a product to ship. They”re intimately aware of how many resources are available, where the political landscape is at any given moment, and they know who knows what. They tend to know all the secrets and they like to be recognized for that fact.

Completionists are dreamers. They have a very good idea of how to solve a given problem and that answer is SOLVE IT RIGHT. Their mantra is, “If you”re going to spend the time to solve a problem, solve it in a manner that you aren”t going to be solving it AGAIN in three months.” I used to think that architects were the only real Completionists in an organization, but I was wrong. Architects are the only RECOGNIZED Completionists in the company, but the personality is hiding all over the place.

Finally. One more book.

The same publicist who sent me Up the Organization also included a copy of Ben Casnocha“s new book My Start-Up Life. Ben is a charismatic, energetic, brilliant 19 year-old who founded a successful software company, Comcate, at age 14. It”s all very adorable. He”s the Doogie Howser, MD of software startups, except for the fact that he probably has no idea who Doogie Howser is, given that the show went off the air when he was 4 years old, and, frankly, at age 4 he was probably too busy working on his second IPO to watch much television.

Ben is a seriously cool 19 year old. He”s very smart. He”s quite a good writer.

But.

But but but.

His book, unfortunately, tells you almost nothing about starting a company. It”s really, really thin on stories of what the actual company did and how things worked. Worse, the book is padded with really, really embarrassing sidebars in which Ben gives you jejune Great Thoughts about business management.

Great entrepreneurs show up, take small risks (and sometimes, large risks), raise their hand when they”re confused, and try to figure out what”s going on and how a situation could be made better.

When you show up and raise your hand, you”ve already outperformed 90 percent of the crowd.

And:

The person on the receiving end of the mentoring relationship should work hard to insure it”s not totally a one-way street.

Ben Ben Ben.

Yes, you”re smart and good looking. Yes, you know more about starting a software company than practically any other 19 year old. And sure, I”ll be happy to invest in your next startup, or hire you, or adopt you, whatever.

But. Mark my words. You”re going to reach the ripe old age of 23, and you”re going to look back on this book you wrote, and you”re going to say, “how on earth did anyone let me publish such self-important crap,” and you”re practically going to die of embarrassment. Trust me: I”m in my 40s, and I”m still morbidly embarrassed by the pompous, arrogant, self-important crap I write on this site here, up to and including this very sentence.

Feel free to skip this book.

Not loving your job? Visit the Joel on Software Job Board: Great software jobs, great people.

Ever been confused by cryptic remote procedure call errors? Take a look at some common errors, techniques you can use to identify errors, and solutions to specific problems.

How do you secure the desktop against malicious ActiveX controls without limiting application compatibility? We’ll take you on a tour of the ActiveX Installer Service (AxIS) in Windows Vista that addresses this issue with a new way to manage ActiveX controls.

MySQL AB today announced that McGraw-Hill Education, a leading global provider of educational materials, information and solutions, has chosen the MySQL database to power the network version of its widely-used “Breakthrough to Literacy” learning program.

MySQL’s high performance, flexibility and small footprint make it an ideal choice for educational software vendors and other ISVs looking to affordably integrate a world-class database into their product offerings. Today, leading educational software vendors using MySQL include American Education Corporation, eduphoria!, eSchool Solutions, LeapFrog Enterprises, Pitsco and Siboney Corporation.

As a medium of communication, blogs have their strengths and weaknesses. The informal nature of the communication makes it incredibly easy to communicate information on an ongoing basis. The down side, of course, is that the very informality of blogs can also trip you up if you don’t pay enough attention to follow up. I’ve certainly been guilty of that on this blog (there are still some loose ends I think haven’t really been tied up), and we’ve been guilty of that as a team.

One thing we haven’t always done a good job of is keeping people apprised of feature decisions that we’ve made as the VB 2008 product cycle has progressed. As a result, there has been some confusion as to what features are in and which features are out of the product. So let me start by giving a reasonably concise list of the feature set for VB 2008…

General features

• Friend assemblies. We now pay attention to the InternalsVisibleTo attribute.
• Relaxed delegates. As long as there is a way to map one delegate’s parameters to another, you can now convert freely between delegate types. (I’d wanted to see this extended into full delegate currying, but that will have to wait for a future release.)
• Runtime agility. The ability to compile without a VB runtime, or targeting another VB runtime.
• Calling Object members on interfaces. Minor feature, but useful in some situations.
• Ternary operator. A short-circuiting conditional expression like C’s ?: operator.
• Improved generic type inferencing. In VB 2005, we would not infer a type for a type parameter that had multiple inferred types (i.e. a parameter T matched two arguments typed as Integer and Long). In VB 2008, we will now pick the wider of the two types. (Actually, it’s more complex than this, but you get the idea.)

LINQ

• Local variable type inferencing.
• Anonymous types.
• Extension methods.
• Nullable types.
• Query expressions. This is the From…Where…Select… expression.
• Object initializers. For VB 2008, we will only support initializing read-write fields of non-collection objects (more on this further down).
• Expression trees.
• Lambda expressions. For VB 2008, we will only support expressions in lambdas, not statements.
• Partial methods.

XML

• XML literals.
• XML members.
• XML namespaces.

I think most everything on this list has so far been discussed, with the exception of the omissions from object initializers. Our original plans, going back to PDC05, included several more features for object initializers, such as being able to write to read-only properties, as well as collection and array initializers. In the end, the schedule for VS 2008 was not sufficient to implement these features with a high degree of confidence. Which unfortunately means that they will have to wait to a release beyond VS 2008.

You may need to use ORDER to force sequence numbers to be output in the order of request when you’re using them for timestamping.

Before using savepoints in your application, you need to be sure that your RDBMS supports this facility.

Many webmasters overlook an important part of search engine optimization: if you over optimize your web pages, chances are that your website rankings might drop because your site has been designed for search engines and not for websurfers. Read this article to learn how to avoid over-optimization.