ABSINTHE MAKES THE MIME GROW FONDLER

Blog consolidation

2010-09-23T08:35:00.000-07:00

I'm an irregular blogger to be sure but just in case anyone is subscribed to this blog, I've decided to start afresh in a new blog here. If you still care to subscribe, please change your links to the new blog as this one will go dead after this posting.

Getting started with home media streaming

2009-04-01T21:09:00.000-07:00

I've had a PlayStation 3 for about a year now. I use it for gaming and as a blu-ray player. It's great at both and it owes me nothing for all the entertainment I've gotten out of it. But I was feeling a bit greedy so I decided to see how hard it would be to use it as a streaming media player hooked up initially to my home server.

Now, don't get excited. My "server" is just a cheap Windows desktop sitting in my office at home. It's not like I have a data center in my basement or something. Over the last year or so, I've managed to rip most of my CDs onto an external terabyte drive. I would not have pictured myself walking out of Costco with a terabyte under my arms a few years ago but that's Kryder's Law for you. I've also experimented a bit with ripping DVDs but haven't quite found the right set of tools for that yet. Anyway, there is a fairly big wad of media (including all my digital photos) sitting on this drive.

My house is networked (wireless) and I've used the PS3 wireless connection connect to the internet to do software updates and putz around on it's browser (although without a keyboard and mouse, the experience is too clunky to be useful). I knew the PS3 was connected so, hey, why not stream that audio and video off the home server? This is where I started messing around with Windows & its Media Player.

I kind of like the interface to Windows Media Player but found a few annoying things that pushed me away from it. I now use Media Monkey which is faster, doesn't screwup my library and has a much smarter web-based tag lookup and automatic library organizer than WMP. But then I started reading about connecting the PS3 to the desktop/server and the most common advice was to download WMP 11 which allegedly supports DLNA, also supported on the PS3. I must have messed around off and on for a couple of weeks trying to make this work. I got WMP 11 to act as a server for other Windows machines on the network but couldn't get the PS3 and desktop to recognize each other at all.

So I gave up on WMP 11 and decided to install TVersity. Wow. This software has what is so often lacking in Microsoft software - the "it works" feature. I just installed it, fired it up, pointed it at my media directories and then walked downstairs to the PS3 and there was my media. I can now browse digital photos on my TV, stream any music I want and even stream video (although there are some glitches there). Getting still greedier, I wondered if I could stream the media out to my daughter's iPod touch (I'm too cheap to buy one for myself :-). The touch is supported through the web interface of TVersity so I wasn't sure what to expect but all I needed to do was type a URL into Safari on the touch and wham, all the media is there. Not only that, but the video is automatically transcoded on the server side before it gets streamed down to the iPod. Very slick (but I need a faster server :-).

What else can you do with TVersity? It also provides an interface to internet video sources like youtube or the various TV networks. Being in Canada, many of these are blocked (boo!) but I played with some that aren't and must say that this makes browsing internet video much more enjoyable (sitting on a couch with a beer and remote instead of perched over a keyboard). I haven't played too much with this except to verify that it works. The hard part will be finding decent content that our big brother government hasn't blocked.

TVersity will also apparently allow you to access your home server from the internet but, given how pitifully slow my uplink speed it, I haven't bothered trying that yet.

So, what about those glitches in video? Well, this may be showing some inexperience on my part in ripping the video in the first place but I've found two problems. First, the audio is just 2-channel stereo. I expect that is a problem with the way I've encoded the sound tracks in my ripped video but not sure yet. The second problem is that some videos will display just fine (as if I were playing a DVD directly, although without the menus) but others stutter showing that something isn't keeping up with the frame rate. I'm sure it's not the PS3. So that leaves the network, the "server" CPUs (assuming it's doing some transcoding) or the server hard drive. Since I'm connected using 802.11g (the PS3 doesn't have an 11n adapter yet), my first suspicion is the network.

I know I could test this theory by moving my PS3 and desktop next to each other and wiring them up with ethernet but I also know I'm going to need more network bandwidth eventually if I want to stream HD video so now I need a wired networking solution for home (even 802.11n is unlikely to cut it). I could pull a load of ethernet cable through the walls but I hate doing that, especially between floors, so I looked around for alternatives. I happened to be listening to the HDTV podcast and they reviewed some powerline networking gear from Corinex. The conclusion was that the setup was easy and the equipment delivered performance near to the claims of 200Mbps. So, that's my next step. I'll report back here when I get that running.

The universal translator

2009-03-18T07:36:00.000-07:00

I don't know anybody that didn't find the universal translator one of the coolest ideas on Star Trek. Being able to translate, in real-time, one language into another has got to have a transformational impact on society. Of course, the idea was not unique to Star Trek and some real-life versions of the device are actually starting to be used today.

However, in this blog post, I want to cover a different and easier universal translation problem that, to the collective shame of compiler engineers such as myself, has not yet been solved. I am referring here to the problem of universally translating software written in any of a number of different programming languages to be executed on any of a number of available computing platforms. Surely, given the well-defined nature of both programming languages and computer architectures, this should be much simpler than the translation between more ambiguous and more semantically rich human languages!

Sadly, we still live in a world where software is designed and built to run on some specific target machine or set of such machines. Yes, platforms like Java which promise "write once, run anywhere" have allowed us to make progress. However, even for Java, the practice of software development is closer to "write once, then test, debug and tune for each new platform".

The biggest negative impact of this practice is a forced trade-off between software availability and capability and/or quality. A given software developer or development organization can only afford to do so much porting, testing and debugging. Therefore, given that this is always required when moving to or certifying for new platforms, there is always a conscious trade-off between platform support and core software features or quality.

This where the universal translator comes in. In principle, compilers should be able to map well-written software into efficient code on any given platform. But do they? Well, gcc is certainly a great example of a compiler that does a good job of targeting lots of platforms and supporting a range of programming languages.

However, there are still two problems.

The first is that, while gcc is portable and widely used, it is not usually the best compiler to use on a given platform. Windows developers will more commonly use Microsoft's compilers. Professional developers on UNIX platforms will tend to use the platform vendor's compilers. Even on Linux, Intel's compilers are often preferred to gcc because of the greater performance they can extract from the hardware.

The second problem is that gcc doesn't support all the programming languages one would want to use and fails to provide good support for languages which are best compiled at run time such as Java or any of a number of dynamic languages such as Ruby or Python.

So .. if not gcc, then what?

Of course, both the Java Virtual Machine and the Microsoft CLI need to be considered as good candidates as universal translators. Unlike gcc, both of these are generally well supported and tunedb major computer vendors. They are also capable of and have demonstrated good quality runtime compilation and optimization.

The JVM, while not designed to support multiple languages, has been reasonably successful in supporting languages such as Ruby and Python. However, the JVM really is all about Java. There are technical challenges in trying to support a range of languages on the JVM, either because of their dynamic nature or because the language wants to manipulate low level constructs such as addresses and native memory that Java tries to abstract away. Even if these technical hurdles can be overcome (and projects like this one certainly show that it is feasible!), the fact remains that the providers of JVM implementations mostly worry about Java and other languages get short shrift.

What about the CLI? Well, unlike Java, it was designed to support a range of languages - at least the ones that Microsoft cares about. The CLI divides the world into "managed" and "unmanaged", referring mostly to the management of memory. This allows a certain amount of interoperability between these worlds (via, for example, Platform Invoke). However, support for languages which don't adhere to the constraints of managed code has been accomplished by forcing changes into the languages themselves. Managed C++, for example, is advertised as a set of extensions to C++ but more important than the extensions is a long list of limitations which really makes this a different language than C++.

So, if not Java or the CLI, then what?

I'm encouraged by the progress of the LLVM (Low-Level Virtual Machine) project at the University of Illinois. Like the JVM and CLI, LLVM is a virtual machine which can be targeted by compilers for a number of different languages and can be implemented efficiently on a number of different machines. Unlike the JVM and CLI, however, there are no constraints on the input language and no implied support for "management" functions like enforcement of memory safety. That doesn't mean that such management can't be implemented on top of LLVM of course. The VMKit project has shown that. While LLVM is lower level in terms of its instruction set and memory model, it is richer in other ways. It integrates data flow and control flow information with the code generated by the compiler, which preserves information that may be critical in generating good native code and accelerate later static or dynamic compilation steps. Of course, I would be remiss not to point out that there have been several successes and failures in introducing such interfaces in the past but I think LLVM is appearing at a point in time where compilation technology is ready to make it work and there is a broad appetite for this kind of solution.

Is LLVM the universal translator? Well, it's still under active research & development and not yet supported by major vendors but, in my opinion, is targeted at exactly the right place architecturally. A low level but portable virtual machine interface is a natural distribution format for software, written in any of a number of languages. It won't replace Java or CLI bytecodes but it could replace the native binaries used to distribute all kinds of other software. It also gives platform vendors the ability to optimize software for their operating systems and hardware with the portability and applicability of gcc but the dynamicity of Java. I'll be keeping my eye on it.

Are you a Pakled ?

2009-02-25T09:01:00.000-08:00

Many of you may be fans of Star Trek. I was for a long time. First the original series then the Next Generation, Deep Space Nine and Voyager series. My interest sort of waned sometime during the Voyager era. However, one episode I remember very clearly was from the Next Generation series and was entitled "Samaritan Snare".

In Samaritan Snare, the Enterprise encounters an alien ship (ok, this happens alot :-). The ship needs help but the aliens (called Pakleds) are inexplicably incoherent about what they need. It turns out that the Pakleds are really a race of technology scavengers, just picking up technology that seems about right for what they need and using it. They have no idea how the technology works. When things go wrong, they are clueless about how to fix the problem.

I remember for many years after the episode aired when I worked in the guts of our compilers how much entertainment we got from interacting with people who, to us, could easily have been Pakleds. These were otherwise intelligent people, many of them literate in the use or even programming of computers, who clearly had no idea how computers actually work. This didn't appear to matter until something broke. Then, they would desparately seek help but would be absolutely incoherent in describing the problem, often not much better than the Pakled's classic "It is broken, can you make it go?".

I've had alot of time to think about this phenomenon and it really isn't a joke. For years, decades really, the IT industry has been building an edifice of abstractions and interfaces (or, if you prefer frameworks) which have provided, with varying degrees of success, ways to interact with and program computers which are provide progressively richer experiences and semantics. We have also trained an entire generation of programmers to work exclusively at these higher levels of abstraction. This richness has come with costs, however.

The first cost is the one already alluded to - that computer users and programmers have become less and less aware of how the technology actually works. This isn't just true about the hardware. I would wager that most software developers know close to nothing about the operational details of anything other than the layer below them and the layer above (if they happen to be framework developers themselves). Does this matter?

Well, many people use the analogy of a car - that many people know how to drive one but few understand even the basics of how it works; yet, people get along quite well without this knowledge. This is true for the most part, although it becomes a bit more of an issue when stranded in sub-zero weather outside of cellular telephone range. Nevertheless, the relationship between cars and drivers generally works. Why isn't this true with computers?

I think this is simply a statement of technology maturity. The rate of change in the basic technology going into cars has been undergoing relatively slow change for a long time (recent innovations in alternate fuels and hybrids being a notable departure). Computer technology, on the other hand, changes like the weather. Software engineering is also a far less mature discipline than mechanical engineering, rarely employing basic notions of modularity and change control, let alone more advanced architectural processes. The result? Complex and often unforeseen dependencies among software components. This becomes a hornet's nest for a software engineer to enter when there is either some kind of failure or, equally, when the system isn't performing quite as well as expected.

Now onto the second cost. This is one that is not yet a big issue but is sneaking up on us. Performance, by and large, has not been a serious software issue for many years. That isn't because we've stopped trying to build new things with software nor is it because we've all become brilliant performance engineers. It is because exponential growth in hardware performance, and specifically advances in processor speed and memory capacity, have propelled us forward despite the additional software-driven overhead. As Bob Metcalfe said, "Grove giveth and Gates taketh away". If you are unconvinced about the magintude of software bloat, check out this sample call stack from a Java application running on the JBoss application server or this striking illustration of how much it costs in time and memory to do even the simplest operations with todays frameworks and protocols.

So what's the problem? Well, processor speed just isn't improving like it used to. In particular, Grove (or his successors) still giveth but it is not in the simple form of faster processors. Instead, we're just getting more that run about the same speed. And memory? Well, it is still getting denser, faster and cheaper but it is now one of the most expensive components of a computer, both in terms of dollars and power consumption. Meanwhile, Gates (or his successors) continue to taketh away, if anything at a faster pace.

This is not just an indictment of Intel or Microsoft, although both play prominent roles. The entire hardware industry is being driven by the same physical forces as Intel and virtually all software developers continue to create fatter, more wasteful (although, in fairness, probably quite innovative) software, oblivious of the collision course with hardware. Modern software developers are like Pakleds wandering through space not realizing that their ship is almost out of fuel and without a clue how to fix it if it does.

Are you a Pakled ?

Innovating without thrashing

2009-02-22T08:04:00.000-08:00

I enjoyed the recent article by Bill Buxton in BusinessWeek. It is entitled "How to Keep Innovating" but it is as much advice for the busy mind as it is a recipe for innovation. Bill starts by making the case that, while depth of skill is essential, especially in one's profession, there are diminishing returns to trying to perfect that skill. Instead, he advises that there should always be a new passion in the pipeline, something to broaden one's experience and inform ongoing innovation.

Along the way, Bill points out the temptations of the curious mind, which I definitely experience and sometimes call "thrashing":

Always being in the throes of a passionate beginning is one of my primary methods of sustenance. But as is all too frequently the case, too much of a good thing can cause its own set of problems. The energy of pursuing such passions can be addictive, and take over at the expense of other things that are equally or more important. It can become destructive. I found this out the hard way, and my wife was able to help me when I most needed it. It was she who reminded me that the limitlessness of life has to be shoe-horned through the limitations of the present.

Lacking the discipline to focus on a small number of passions at one time can lead one to become a jack of all trades but master of none (or very few at any rate). I have plenty of passions - mathematics, technology, literature, history, geopolitics, music and more. Unfortunately, there are times when I thrash between them deriving only a tiny amount of benefit from my investment in each (not to mention shortchanging other parts of my life like my family).

I'm getting better at this but am glad to see that I'm not the only one with this affliction!

Cringley on multicore

2009-02-12T12:45:00.000-08:00

I just picked up the latest copy of Technology Review and read Cringley's article on multicore.

First of all, I have to say that Cringley can write. I have struggled to explain this trend and the set of associated software challenges many times, to technical and non-technical people alike and never approached the concise and approachable description given in this article. I'm a techie at heart and it's at times like this that I realize I have a long way to go as a writer and communicator.

Anyway, self-incrimination aside, I think this article (assuming people aren't sick of reading about this subject by now) really does a service by putting this trend into a historical context that everyone can understand and by clearly explaining some of the subtleties here, like frequency not equating to performance and, for that matter core count not equating to performance. Those messages are suprisingly hard to deliver to people that insist on a simple characterization of how computers work or even more important to those that have no interest at all in how computers work. It also does a pretty good job of explaining why parallel programming might be hard to get right.

I do wish that Cringley had at least mentioned the distinction between multicore as a client/mobile phenomenon vs. multicore in the server space. He does mention map/reduce (which is completely a server-side programming model) but fails to observe, especially in his conclusion, that while client software may fail to drive for ever greater parallelism, there is a virtually insatiable appetite for it in the server world. However, like the video processing example he gave, while the parallelism is often easy to find in server-side programs, there are many reasons why it is difficult to extract.

Some of those reasons are bound up in the issues he does cover like the lack of good tools and languages but another big issue is the inertia of existing large software environments. This so-called "legacy" software (I've been told by many that hold this stuff dear that "legacy" is pejorative, kind of like used cars vs. "previously enjoyed " cars I guess) .. sorry side-tracked. This "previously enjoyed" software has a suprisingly long life - often measured in decades rather than years. The problem here is that, even if we had the tools and languages and skills and even the desire to make this software exploit parallelism, there aren't enough programmers in the world to even put a dent in it. We're talking about trillions of lines of code. Hard to fathom I know but it's true. And much of civilization depends upon it (that is not an exaggeration). This is a particularly hard nut to crack and we in the server/enterprise computing world have this problem in spades. On the client, people change their software alot more frequently - usually when the shiny new version is available or Microsoft breaks compatibility, whichever comes first.

Parochial interests aside, I highly recommend the article. It contains no new revelations but, if you work in this area, it just might be something your spouse or friends will understand when they ask what you do for a living.

Clouds without the nuts & bolts

2009-02-09T08:04:00.000-08:00

I recommend reading Jon Stokes interview (and part 2) of Russ Daniels of HP on Cloud Computing. I read quite a bit about Cloud and, as a techie, get completely frustrated by the lack of consensus about what it all means. On the other hand, it is clear that consumers and businesses both see alot of potential value in Cloud. Daniels does an excellent job in this interview of explaining that value, from both consumer and enterprise perspectives. He does it without getting into the nuts & bolts of what a Cloud is (which is good, because it's easy to get bogged down there) and, to his credit, without too much boosting of HP products & services.

Daniels' sees the cloud as the intersection of utility computing (resources on-demand) with SOA (loosely coupled application architecture) and Web 2.0 (pervasive accessibility and social experience). He pulls it together in a very compelling way and with some good examples that left me wondering whether I've fully understood the implications of Cloud.

GPUs in popular culture?

2009-02-06T15:08:00.000-08:00

I think now I've seen it all. NVidia hocking graphics processors on Charlie Rose. I may be one of maybe 5% of Charlie's audience who actually understood what the heck Huang was talking about, or even cared!

But it is interesting that a discussion of parallel computing actually gets air time like this. Rose is a bit of a technophile and this is PBS but, still, a program that features both multicore processors and LL Cool J is not something I expected to ever see :-)

Scalability matters

2009-02-06T04:58:00.000-08:00

The Gartner report on multicore architectures & server scaling has generated a bit of buzz recently. I agree with Nik Simpson that much of the coverage of this issue is sensationalized, partly due to the fear of the unknown and partly due to some none-too-subtle fear mongering by those who stand to gain or lose alot from the transition toward parallelism. Nik is also correct that software developers will focus on the platforms they know they need to support and, when new, more parallel machines are available, they will adapt.

However, an important point made in the Gartner report is that enterprise IT doesn't much like upgrading software and hardware at the same time. Well, in reality, they don't much like upgrading at all if they can avoid it but trying to change too much at once is often a recipe for disaster. The point made in the Gartner report is that, to the extent that new software is needed to get value out of new hardware, this will inject a new source of risk into adopting new hardware. Historically, this has not been such a big problem because old software could generally ride the wave of frequency scaling without modification but, as we all now know, that gravy train is no longer in service. Logically, then it follows that software that is prepared to scale with new systems over a fairly long period of time (many years) without substantial modification will offer compelling value over software that needs a brain transplant every time a new, more parallel system arrives at the loading dock.

So what does that mean for software developers? It means that we need to think as hard about scalability as we do about reliability and security. People rarely question "over engineering" in those areas (at least in the enterprise applications or middleware space) because they are so critically important. I assert that now scalability is in the same category. A business that can't exploit the latest systems will fail to exploit cost sand energy savings and possibly fail to capture growth opportunities because their applications can't keep up to demand.

I'll write more about this scalability challenge in the future. For now, the important thing is not to panic. Multicore is a geometric growth trend like frequency was before it but we do have time to get the software right and there is plenty of help and tools available to get the job done.

Obama continues to be a study in leadership

2009-02-05T17:56:00.000-08:00

"I made a mistake" was the headline of this Anderson Cooper interview of President Obama. It's worth watching just to see how masterfully he admits an error and moves on. The result is that the interview is not about Daschle at all but about substantial policy issues. It made me think of times when I've hesitated to admit a mistake and how that has resulted in damaged trust. Heck, if the President of the United States can admit his errors and move, then anyone can!

debategraph, a very cool collaborative tool

2009-01-30T20:40:00.000-08:00

You've got to check out debategraph. This is one of the coolest collaboration tools I've seen in a while. It is sort of a cross between a mind map and a wiki.

The team that has put this together has focused on creating a forum for public debate that allows multiple arguments to be expressed without one occluding the other. It also allows the argument to converge on a consensus based on voting. If you check out the site, you'll see active debates on the Israel-Gaza situation, Climate Change, the Global Financial Crisis and many more.

I wonder, though, how effective an environment like this may be for collaboration vs. debate. In some ways, it reminds me of the collaborative Jams that IBM has done over the past few years. In both cases, technology is used to enable mass collaboration but it seems that debategraph may have several advantages, including very slick visualization and navigation of a discussion and real-time updates of the views of the majority of participants through the voting mechanism. One important behind-the-scenes part of the Jam technology, however, is the ability to run analytics on the jam threads to find relationships and active lines of debate/discussion. Those analytics are really useful for mining real insights from a discussion potentially involving thousands of participants like World Jam.

I'm looking forward to the day when tools like this become as natural a form of communication as email is today.

Do we need broadband investment?

2009-01-29T05:36:00.000-08:00

Sam Palmisano and groups of other CEOs met yesterday with President Obama. It is clear now that the President and Palmisano are on the same page when talking about updating infrastructure for energyeucation, healthcare and transportation, as I had inferred from the inauguration speech. The claim is that this set of investments can create 1,000,000 jobs. If true, and if those jobs are sustainable, that is a remarkable contribution to climbing out the economic recession.

There are dissenters of course, even on an issue like broadband that seems on the surface to be so obviously valuable. Steve Wildstrom at Business Week makes the point that there will be diminishing returns to either driving toward 100% penetration of broadband or pushing for higher speeds on existing broadband connections. I'm sure that is true at some point but penetration sits at just 25% today so I doubt that diminishing returns will kick in until we get far north of 80%. Also, we really need to look at the definition of "broadband" as used in OECD reporting. Any connection that delivers more than 256Kbps is considered "broadband". Steve makes the valid point that 100Mbps is overkill but 4-8Mbps isn't. In fact, it's getting darn close to necessary to participate in the emerging internet of rich digital media. Will it be necessary to push to higher rates? Maybe. But let's get most people up to the 4-8Mpbs range and then worry about it. In terms of global competitiveness, the US sits at 15th in broadband penetration. Not too bad I guess (Canada is faring a bit better at 10th). However, the penetration is Korea and in the Nordic countries is far higher. Furthermore, the fibre penetration in Korea and Japan is far greater than any other country. In the US, it is almost zero.

So I respectfully disagree with Mr. Wildstrom (even though I am a loyal reader and listener). The US and most of the major Western economies had better start catching up with internet infrastructure or they will find themselves pushed out of the internet economy.

A Canadian watches Washington

2009-01-20T19:32:00.000-08:00

A historic day for America and for the world! I am Canadian and proud of it but no country has a closer relationship with America so this day is special and significant for us as well.

The inaugural was full of hope and confidence as it should be and is just words as any speech is but words can move people and nations and I hope today's words will. The focus on traditional American, ultimately puritan values is exactly the right way to unify divisive factions in the country and maybe even inspiring enough for those in conflict or oppressing populations across the world to take notice that America is changing. Of course, actions are what matter so I can't wait to see what Obama will do with this mandate.

Technology figured in the inaugural as well. Obama said:

The state of the economy calls for action, bold and swift, and we will act - not only to create new jobs, but to lay a new foundation for growth. We will build the roads and bridges, the electric grids and digital lines that feed our commerce and bind us together. We will restore science to its rightful place, and wield technology's wonders to raise health care's quality and lower its cost. We will harness the sun and the winds and the soil to fuel our cars and run our factories. And we will transform our schools and colleges and universities to meet the demands of a new age. All this we can do. And all this we will do.

I think it is remarkable how similar this commitment is to the advice offered by IBM CEO Sam Palmisano:

$30 billion in government investments in expanding broadband access, computerizing health-care records and improving the electrical grid could create more than 900,000 U.S. jobs.

Smart grids. World-class broadband. Digitized healthcare records.

These are all elements of the Smarter Planet vision. They are all also priorities for Canada. I remember the excitement within IBM when Palmisano made the speech to the Council on Foreign Relations. Why were IBMers excited and why are so many still charged up about this? Because it is a promise that we will use this opportunity of an economic recession to invest in growth in the future. Obama is making the same promise about the country that Palmisano made about IBM:

The state of the economy calls for action, bold and swift, and we will act - not only to create new jobs, but to lay a new foundation for growth.

Who has multicore fatigue?

2009-01-13T15:45:00.000-08:00

I follow processor and system architecture pretty closely so may be more vulnerable to fatigue but I'm guessing a lot of you are sick and tired of being lectured about the dark clouds of multicore gathering over the IT industry. This most recent article in Dr. Dobbs, while reasonably accurate, is like so many before it - almost bereft of hope!

The fact that our processors and computer systems are becoming more parallel is fundamentally a very good thing. We are getting much more improvement in each generation in performance per dollar and per watt than ever before. Yes, it will be a pain for some software to adapt. Yes, we will need new tools. Yes, programmers will need to learn a few new tricks. But please stop the gloomy lectures and prognostications!

Computers are evolving, as ever, into more and more capable and efficient thinking machines. We will not only adapt our software to parallelism, we will find ways to harness this new capability in remarkable new ways. More and more of the repetitive details of our everyday and business lives will be automated. New treatments and cures will be found for human diseases. We will learn much more about how to repair our ailing planet. Those are the things that matter, not the fact that we'll need to add a course or two to the typical computer programming degree.

Multicore: Get over it and start innovating!

Is a smarter planet a less secure planet?

2009-01-12T08:52:00.000-08:00

I was taken a bit by surprise when I heard Michael McConnell (soon to be ex-Director of National Intelligence) list cyber-attack as a weapon of mass destruction (alongside nuclear and biological) in a recent interview with Charlie Rose. Of course, this theme of pervasive computerized control being vulnerable to attack has been done and redone in popular media, from the latest Die Hard flick to the more recent Eagle Eye and, it appears, the most recent installment of 24. However, it really is a different matter when the DNI stays awake at night thinking about it.

So how real is the threat? Well, there is no shortage of study in this area, although it seems that most of the work has been focused on the potential for damage, theft and espionage as a result of unauthorized access to critical information on goverment or commercial information systems. That threat is real and, while perhaps more likely to be perpetrated by criminals than hostile governments or terrorists, it is an important element of national defense. However, an even larger threat emerges as the idea of a Smarter Planet becomes reality. The idea of the smarter planet is that, as the physical world becomes subject to ever more pervasive digital monitoring and control and those sensors and actuators become networked over the internet, much greater efficiency can be achieved and the data that is produced can result in new value. Indeed, IBM has been advising the Obama transition team on the merits of IT infrastructure investments in the upcoming stimulus package, notably including $10B for "smart-grid infrastructure" (referring here to the electrical grid).

So, is a smarter planet a less secure planet? Clearly, this is not necessarily the case and there are many diligent security specialists that will be part of this ongoing rollout of digital control. However, like any area that is new to computerization, there are fundamental misunderstandings about what it means to be secure. For example, this report on threats to SCADA systems concludes that industrial users of SCADA fail to properly implement authentication, have no established practice of updating systems to defend against newly discovered vulnerabilities and are in a state of denial about the connectivity of their systems to the internet. Furthermore, most threat assessments have been done assuming the current level of monitoring and automation of physical infrastructure which, while substantial, can be expected to dramatically increase (if one believes in the Smarter Planet ideas) in capability and connectedness in coming years.