Build a Resilient Modern Home Storage & Backup Solution

I’ll admit it, but my home network tends to push the edges of what consumer technology wants to support.  Two months ago, I had one of those terrible technology events that forces you to rethink your entire network: my Netgear ReadyNAS NV+ failed in a disasterous way, causing me to lose my entire iTunes Library.

As a result, I embarked on a process to rethink my offsite data backup and storage solutions for my household, which in this modern age of iPhones, iPads, AppleTVs, and countless media devices has become fairly complex.  Since the solution that I settled on required quite a bit of research, experimenting and simplification, I’m hoping some readers will find it interesting.

Call it: “Adam’s Home Storage Solution, Fall 2011 Edition”.

Overview: Network Design Diagram

You can see above the relevant elements of my home network topology.  It’s anchored to the internet via AT&T UVerse, which provides a 24Mbps down, 5Mbps up service over VDSL.  The router for my home network is plugged into an 8-port Gigabit switch, which is effectively the backbone for the entire house.  As part of the process of revisiting my network, I discovered that historically I had used a mish-mash of old Ethernet cables, some Cat 5, some Cat 5e, and it was affecting some connections.  A quick trip to Fry’s ensured that, for just a few dollars, I had Cat 6 cables for all Gigabit devices.  (This turned out to be important, particularly for connections to my iMac, wireless base station, and NAS box).

Basic Storage Topology

While my network supports a wide variety of clients, the backbone of my solutions is very Apple-centric.  As a result, my solution is optimized for the following decisions:

  • My media store is based on iTunes
  • My primary server is an iMac running Mac OS X 10.7 (Lion)
  • My on premise backup solution is Time Machine

I was able to simplify my storage needs for the network as follows:

  • The iMac uses the built-in 256 GB Solid State drive for the system & applications
  • The iMac uses the built-in 2 TB standard drive for local storage of most media (downloads, documents, pictures)
  • The iMac uses a 4 TB Seagate GoFlex External USB 3.0 drive for the iTunes library
  • The iMac and all other Macs in the house use Time Machine to backup to the Synology DS1511+, which has 8.3 TB usable space.

The Synology DS1511+ has dual Gigabit Ethernet ports, which allows for particularly good performance when multiple machines are trying to read / write to it at the same time.  Configuring the box to support Time Machine for multiple clients is not obvious, but I’ll write up a separate blog post on that issue.

Overall, the performance of this solution is excellent.  iTunes performance from the Seagate is excellent, both for the primary machine as well as for remote devices utilizing Home Sharing to access media (like the AppleTVs).  We are able to run video off this solution to all 3 AppleTV devices simultaneously with no issues.  Copying a 250MB file to the Synology box takes approximately 2 seconds, and it offers no measurable delay in terms of Time Machine incremental backups, viewing, and restoration.  The entire initial backup of 4.05 TB via Time Machine to the Synology box took approximately 26 hours.

Backup Solutions

Let’s not forget that the impetus for this entire redesign was the tragic and unnecessary demise of the Netgear ReadyNAS NV+, causing massive data loss.  Without belaboring the point, I hope that no one who reads this will ever make the mistake of buying a Netgear ReadyNAS.

That being said, it did lead me to significantly reconsider a multi-tier solution for document protection.

I would have loved to go purely with a cloud-based solution, but the performance is just not there yet for multi-terabyte systems.  Not only does it take an inordinate amount of time to upload terabytes to the cloud, but in the case of data loss, recovering the data would be equally slow.  Uploading 400+ GB to the cloud took me approximately 40 days… 4 TB would have taken over a year!

As a result, I factored my content into what I absolutely could not live without.  I settled on my 450 GB of photos and home movies that would be devastating if lost.  For $90, I subscribed to Crashplan Pro, which offers unlimited storage and came highly recommended by everyone.

As a result, for this crucial data, I have 3 levels of protection:

  • Primary storage
  • Secondary backup via Time Machine to Synology RAID can tolerate up to 2 disk failures simultaneously
  • Tertiary off site backup to CrashPlan

For the rest of my data, I have a fairly robust solution, but I’m considering storing 4 TB drive offsite somewhere periodically to add that “tertiary” level of security / safety.

Final Thoughts

The above solution may seem like overkill to some.  OK, probably to most.  However, you can simplify the solution above based on your needs.  For example, if you have only 200 GB of data to protect, maybe CrashPlan is the right “set and forget” solution for your network.  Maybe the 4 TB Seagate drive is sufficient for your Time Machine needs.

For those of you interested in the Synology box, I plan to write up a follow-on post on how to configure the Synology DS1511+ for Time Machine on Mac OS X 10.7 Lion.

Final Solution: Quicken 2007 & Mac OS X Lion

In July I wrote a blog post about a proposed solution for running Quicken 2007 with Mac OS X Lion (10.7).

Unfortunately, that solution didn’t actually work for me.  A few weeks ago, I made the leap to Lion, and experimented with a number of different solutions on how to successfully run Quicken 2007.  I finally come up with one that works incredibly well for me, so I thought I’d share it here for the small number of people out there who can’t imagine life without Quicken for Mac.  (BTW If you read the comments on that first blog post, you’ll see I’m not alone.)

Failure: Snow Leopard on VMware Fusion 4.0

There are quite a few blog posts and discussion boards on the web that explain how to hack VMware Fusion to run Mac OS X 10.6 Snow Leopard.  Unfortunately, I found that none of them were stable over time.

While you can hack some of the configuration files within the virtual image package to “trick” the machine into loading Mac OS X 10.6, it ends up resetting almost every time you quit the virtual machine.  I was hoping that VMware Fusion 4.0 would remove this limitation, since Apple now allows virtualization of Mac OS X 10.7, but apparently they are still enforcing the ban on virtualizing Snow Leopard.  (Personally, I believe VMware should have made this check easy to disable, so that expert users could “take the licensing risk” while not offending Apple.  But I digress.)

You can virtualize Snow Leopard Server, but if you try to buy a used copy on eBay, it’s still almost $200.00.  Added to the $75.00 for VMware Fusion, and all of a sudden you have a very expensive solution.  Worse, VM performance is surprisingly bad for a Mac running on top of a Mac.  In the end, I gave up on this path.

Enter the Headless Mac Mini

For the longest time, you couldn’t actually run a Mac as a headless server.  By headless, I mean without a display.  It used to be that if you tried to boot a Mac without a display plugged in, it would stop in the middle of the boot process.

I’m happy to report that you can, in fact, now run a Mac Mini headless.

Here is what I did:

  • I commandeered a 2007-era Mac Mini from my grandmother. (It’s not a bad as it sounds – I upgraded her to a new iMac in the process.)
  • I did a clean install of Mac OS Snow Leopard 10.6, and then applied all updates to get to a clean 10.6.8
  • I installed Quicken 2007, and applied the R2 & R3 updates
  • I configured the machine to support file sharing and screen sharing, turned off the 802.11 network, turned off bluetooth, and to wake from sleep from Ethernet.  I also configured it to auto-reboot if there is a power outage or crash.
  • I then plugged it in to just power & gigabit ethernet, hiding it cleverly under my Apple Airport Extreme Base Station.  It’s exactly the same size, so it now just looks like I have a fatter base station.

I call the machine “Quicken Mac”, and it lives on my network.  Anytime I want to run Quicken 2007, I just use screen sharing from Lion to connect to “Quicken-Mac.local”, and I’m up and running.   Once connected on screen sharing, I configured the display preferences of the mac to 1650×1080, giving me a large window to run Quicken.

I keep my actual Quicken file on my Mac OS X Lion machine, so it’s backed up with Time Machine, etc.  Quicken Mac just mounts my document folder directly so it can access the file.

Quicken: End Game

This solution may seem like quite a bit of effort, but the truth is after the initial setup, everything has worked without a hitch.  I’m hoping that once Intuit upgrades Quicken Essentials for the Mac to handle investments properly, I’ll be able to sell the Mac Mini on eBay, making it effectively a low cost solution.

For the time being, this solution works.  Mac OS X 10.7 Lion & Quicken 2007.  It can be done.

 

Steve Jobs, BMW & eBay

There have been so many articles posted on Steve Jobs in the past week, I really thought I wasn’t going to add one here on my blog.

However, yesterday, John Lilly wrote a great piece on Steve Jobs yesterday, and I realized I might have a story worth telling after all.  I find myself fortunate, in retrospect, to have joined Apple in 1996 as an intern, and then full time in 1997 just weeks before Steve Jobs took the helm as interim CEO.

A Tale of Two Meetings

As an outgoing intern of the Advanced Technology Group, I actually did attend the meeting that John describes in his blog post.  However, as a full time engineer on WebObjects, I also had the opportunity to attend a different all hands that Steve Jobs called for the entire Rhapsody team (the codename of the project that became Mac OS X).

If you haven’t read John’s post, it’s definitely worth reading in tandem with this one.  He does a great job capturing the insights from the ATG meeting.  Instead, let me add to the story with my recollection of the Rhapsody meeting that happened the same week.

(Note: It has been over fourteen years since the meeting, so don’t take this as a literal play-by-play.  I have no notes, so all quotes are from memory.  But this is how I remember it.)

The “Michael Dell” Meeting

The mood of the Rhapsody team meeting was energetic, but mixed.  More than any other group at Apple, the Rhapsody team required a combination of talent from both long time Apple engineers and newly merged NeXT engineers.  There was a palpable sense of excitement in the room, as particularly the NeXT team had a huge amount of respect for the “incoming administration”.  At the same time, there was an element of discontent around suddenly finding themselves part of a large company, and even some skepticism that Apple was salvageable.

Steve got on stage at the front of the room in Infinite Loop 4, and put a huge, larger than life picture of Michael Dell on the wall.  He repeated the news fodder that Michael Dell had been asked recently what he would do if he was running Apple Computer.  (At the time, Dell was the ultimate success story in the PC industry.)  Dell said that he would liquidate the company and return the cash to shareholders.

A few gasps, a few jeers and some general murmuring in the audience.  But I don’t think they expected what he said next.

And you know what? He’s right.

The world doesn’t need another Dell or HP.  It doesn’t need another manufacturer of plain, beige, boring PCs.  If that’s all we’re going to do, then we should really pack up now.

But we’re lucky, because Apple has a purpose.  Unlike anyone in the industry, people want us to make products that they love.  In fact, more than love.  Our job is to make products that people lust for.  That’s what Apple is meant to be.

What’s BMW’s market share of the auto market?  Does anyone know?  Well, it’s less than 2%, but no one cares.  Why?  Because either you drive a BMW or you stare at the new one driving by.  If we do our job, we’ll make products that people lust after, and no one will care about our market share.

Apple is a start-up.  Granted, it’s a startup with $6B in revenue, but that can and will go in an instant.  If you are here for a cushy 9-to-5 job, then that’s OK, but you should go.  We’re going to make sure everyone has stock options, and that they are oriented towards the long term.  If you need a big salary and bonus, then that’s OK, but you should go.  This isn’t going to be that place.  There are plenty of companies like that in the Valley.  This is going to be hard work, possibly the hardest you’ve ever done.  But if we do it right, it’s going to be worth it.

He then clicked through to a giant bullseye overlayed on Michael Dell’s face.

I don’t care what Michael Dell thinks.  If we do our job, he’ll be wrong.  Let’s prove him wrong.

All I can remember is thinking: “Wow. Now that’s how you regroup, refocus and set a company in motion.”  I had seen speeches by Gil Amelio in 1996, and there was nothing comparable.  Please remember, at this point in time it wasn’t at all obvious that Steve or Apple would actually succeed. But I felt like I’d witnessed a little piece of history.

Fast Forward: eBay 2006

That meeting left a huge impression on me that extended well beyond Apple.  Steve’s actions and words at Apple in 1997 represented the absolute best in leadership for a turnaround situation.

It wasn’t until 2006, however, that I found myself at another large technology company looking to rediscover itself.  In the summer of 2006, I was one of a relatively small number of product leaders to tour a draft of a new initiative at eBay called “eBay 3.0”.  Led by the marketing team, a small, strong team had done a lot of research on what made eBay different, and what people wanted from the eBay brand.  The answer was that eBay was fun, full of serendipity, emotion, thrill.  The competition of auctions, the surprise at discovering something you didn’t know existed.  This reduced into a strong pitch for eBay as “colorful commerce”.

I was excited about the research and the work, because it echoed some of the things I remembered about Steve & Apple, and the simple vision he had for a company that made products that people lusted for.  But I also remember voicing a strong concern to several members of the team.  I told them about Steve’s speech to the Rhapsody team, and asked: “Does eBay want BMW market share, or Toyota market share?”  At the time, eBay was more than 20% of all e-commerce, and all plans oriented towards growing that market share.

Unfortunately, eBay tried to do both with the same product.

It’s not typical for a large, successful public company to basically say market share doesn’t matter, and to drive a company purely around a simple focus and vision.  When things are the toughest, unfortunately, that’s when leadership and vision matter the most.

Final Thoughts

Who would have imagined that Apple would have the largest market capitalization in the world?  Who would have thought that in the year 2011 that Apple – not Microsoft, not Dell, not Sony – would be defining the market for so many digital devices and services?

Most importantly, who would have thought that a leadership mandate that eschewed market share would achieve such dramatic gains?

Apple so easily could have gone the way of SGI, the way of Sun.  Instead, it literally shapes the future of the industry.  All because in 1997 Steve was able to offer a simple and compelling reason for Apple to exist.  A purpose.  And it’s a purpose that managed to aggregate some of the most talented people in the world to do some of their best work.  Again and again.

So I will add here a simple thank you to Steve Jobs for that meeting, and for changing the way that I think about every company’s purpose – their reason to exist.  Rest in Peace, Steve.

Bug in iPhoto 11 with iCal Import for Calendars

This is one of those simple blog posts where I write about a frustrating problem, and how I worked around it.

The Culprit

iPhoto 11 and it’s Calendar feature.

The Issue

When you try to import iCal dates into a Calendar, it frustratingly deletes events if they “collide” on the same date.

Example

Let’s say you have two iCal calendars, one for your family birthdays and events, and one for your friends birthdays and events.  Let’s also say that your brother is born on April 11th, and your friend is born on April 11th.

When you import both iCal calendars into iPhoto, only one of the birthday events will show up.  This does not happen if both birthdays are in the same calendar – only if they are in two different calendars.

What’s worse is that this also affects the native support for holidays.  So any friends or family born on July 4th are definitely out of luck.

Solution / Workaround

It’s not perfect, but here is my solution:

  1. Uncheck the holidays checkbox on the calendar import.  This gets you one “clean” calendar import that won’t hit the bug.
  2. Go to iCal and export each of the calendars that you want to add to your iPhoto calendar.
  3. In iCal, create a new calendar called “2012 iPhoto Calendar” or something like that.
  4. In iCal, import each of the calendars you exported, in the order you want them to appear.  Add them to the new “2012 iPhoto Calendar” calendar.
  5. Once you are done, quit iPhoto.  It only detects iCal changes at launch.
  6. Launch iPhoto
  7. Import the new iCal calendar “2012 iPhoto Calendar”.  All your dates will appear, in the order you combined them.

Hope this helps someone out there.  For my rather elaborate family calendar efforts (which involve five separate family calendars of birthdays, anniversaries, and key dates), this was an essential fix.

Proposed Solution: Quicken 2007 & Mac OS X Lion

Right away, you should know something about me.  I am a die-hard Quicken user.  I’ve been using Quicken on the Mac since 1994, which happens to be the point in time where I decided that controlling my personal finances was fundamentally important.  In fact, one of my most popular blog posts is about how to hack in and fix a rather arcane (but common) issue with Quicken 2007.

So it pains me to write this blog post, because the situation with Quicken for the Mac has become extremely dire.  Intuit has really backed themselves into a corner, and not surprisingly, Apple has no interest in bailing them out.  However, since I love the Mac, and I love Quicken, I’m desperately looking for a way out of this problem.

Problem: Mac OS X Lion (10.7) is imminent

Yesterday, I got this email from Intuit:

It links to this blog post on the Intuit site.  The options are not pretty:

  1. You can switch to Quicken Essentials for Mac.  It’s a great new application written from the ground up.  In their words, “this option is ideal if you do not track investment transactions and history, use online bill pay or rely on specific reports that might not be present in Quicken Essentials for Mac.” Um, sorry, who in their right mind doesn’t want to track “investment transactions”?  Turns out, at tax time, knowing the details of what you bought, at what price, and when are kind of important.  At least, the IRS thinks so.  And they can put you in jail and take everything you own.  So I’m going with them on this one.  No dice.
  2. You can switch to Mint.  I love Mint, and I’ve been using it for years.  But once again, “This option is ideal if maintaining your transaction history is not important to you.”  Yeesh.  For me, Mint is something I use in addition to Quicken.  Unfortunately, Mint is basically blind to anything it can’t integrate with online.  Which includes my 401k, for example.
  3. You can switch to Quicken for Windows.  Seriously? 1999 called and they want their advice back.  Switch to Windows?  Intuit would get a better response here if they just sent Mac users a picture of a huge middle finger.  By the way, to add insult to injury:  “You can easily convert your Quicken Mac data with the exception of Investment transaction history. You will need to either re-download your investment transactions or manually enter them.”

This is an epic disaster.  I’m not sure how many people are actually affected.  But the Trojan War involved tens of thousands of troops, so I’m going with Homer’s definition of “Epic”.

What’s the Problem?

There are really three issues at play here:

  1. Strike 1. Around 2000, Intuit made the mistake of abandoning the Mac.  Hey, they thought it was the prudent thing to do then.  After all, Apple was dying.  (The bar talk between Adobe & Intuit on this mistake must be really fun a few drinks into the evening.)  Whoops.  This led Intuit to massively under-invest in their Mac codebase, yielding a monstrosity that apparently no one in their right mind wants to touch.  From everything I hear, Quicken 2007 for the Mac might as well be written in Fortran and require punch cards to compile.  Untouchable.  Untouchable, unfortunately, means unfixable.
  2. Strike 2. Sometime in the past few years, someone decided that Quicken Essentials for the Mac didn’t need to track investment transactions properly.  I’ve spent more than a decade in software product management, so I have compassion for how hard that decision must have been.  But in the end, it was a very expensive decision, and even if it was necessary, it should have mandated a fast follow with that capability.  It’s a bizarre miss given that tracking investment transactions is a basic tax requirement.  (See note on the IRS above)
  3. Strike 3Apple announces the move from PowerPC chips to Intel chips in June 2005.  Yes, that’s *six* years ago.  Fast forward to June 2011, and Apple announces that their latest operating system, Mac OS X Lion, will not support the backwards compatibility software to allow PowerPC applications to run on Intel Macs.

Uh oh.

This is Intuit’s Fault.

With all due respect to my good friends at Intuit, this problem is really Intuit’s fault.  Intuit had six years to make this migration, and to be honest, Apple is rarely the type of company to support long transitions like this.  You are talking about the company that killed floppy drives almost immediately in favor of USB in 2000, with no warning.  They dropped support for Mac OS Classic in just a few years.  It’s not like Apple was going back to PowerPC.

If you examine the three strikes, you see that Intuit made a couple of tactical & strategic mistakes here.  But in the end, they called several plays wrong, and now they are vulnerable.

Intuit would argue that Apple could still ship Rosetta on Mac OS X Lion.  Or maybe they could license Rosetta to Intuit to bundle with Quicken 2007.

Apple’s not going to do it.  They want to simplify the operating system (brutally).  They want to push software developers to new code, new user experience, and best-in-class applications.  They do not want to create zombie applications that necessitate bug-for-bug fixes over the long term.  Microsoft did too much of this with Windows over the past two decades, and it definitely held them back at an operating system level.

A Proposed Solution: VMware to the rescue

I believe there is a possible solution.  Apple has announced that Mac OS X Lion will include a change to the terms of service to allow for virtualization.  If this is true, this reflects a fundamental shift in Apple’s attitude toward this technology.

The answer:

  • Custom “headless” install of Mac OS X 10.6.8, stripped to just support the launch of Quicken 2007.
  • Quicken 2007 R4 installed / configured to run at launch
  • Distribution as VMware image

OK, this solution isn’t perfect, but it is plausible.  Many system utilities are distributed with stripped, headless versions of Mac OS X.  In fact, Apple’s install disks for Mac OS X have been built this way.  A VMware image allows Intuit to configure & test a standard release package, and ensure it works.  They can distribute new images as necessary.

The cost of VMware Fusion for the Mac is non trivial, but actually roughly the same price as a new version of Quicken.  I’m guessing that Intuit & VMware might be able to work out a deal here, especially since Intuit would be promoting VMware to a large number of Mac users, and even subsidizing it’s adoption.

Will Apple Allow It?

This is always the $64,000 question, but theoretically, this feels like really not much of a give on Apple’s part.  They are changing the virtualization terms for Mac OS X Lion, so why not change them for Snow Leopard to0.

Can We Fix It? 

I’m a daily VMware Fusion user, which is how I use both Windows & Mac operating systems on my MacBook Pro.  If Intuit can’t work this out, I just might try to hack this solution myself.

In the end, I’m a loyal Intuit customer.  I buy TurboTax every year, and I use Quicken every week.  So I’m hoping we can all find a path here.

Feel free to comment if you have ideas.

 

 

Home Network Wireless Topology: Fixed

I think I’ve finally found a wireless network topology that works at my house.   It took a bit more equipment than I think should have been necessary, but in the end, it was a small price to pay for having my increasing array of network-dependent devices running smoothly.

Since my guess is that there are a few other suckers like me out there trying to get this to work, I’ll share my final solution.

Problem

Until recently, my home network was plagued by the following issues:

  • AppleTV in the living room would fail to stream, seamingly due to lost connections
  • Tivo HD in living room would periodically complain of being unable to connect to network
  • Nintendo Wii was shockingly slow connecting to network
  • Tivo HD in bedroom would be unable to play video from other room
  • AppleTV in bedroom would periodically fail to stream

Now, it’s not like the above happened all the time.  I never had a problem with an iPhone / iPad / Windows laptop / MacBook connecting to the network.  It was largely restricted to my video devices.  Unfortunately, it was infrequent enough that I could believe everything was configured correctly, but often enough that deep down, I felt like there were Gremlins in the building.

The Solution

The culprit turned out to be a circa-2008 Airport Extreme that I was using to drive my 802.11N network from the office.  It turns out, the older Airport Extreme can handle either 2.4Ghz or 5Ghz frequencies, but not both simultaneously.  Since the iPhone / iPod uses 2.4Ghz, for compatibility you are effectively stuck at 2.4Ghz.   In addition, my office is literally at one corner of my house from the bedroom.  Not ideal, spatially, for the hub of my network.  The living room is more centrally located.

I began to suspect that the number of wireless devices that I owned had crossed some threshold, and the amount of interference and cross-talk was leading to unpredictable behavior.

As a solution, I purchased a newer Airport Extreme base station, with dual-band support.   However, instead of replacing the old base station, I added it to the living room as a network extension of the existing wireless network.  In order to do this, you need to do the following:

  • Open up the “Airport Utility” in the  “Utilities” folder in “Applications” (on Mac OS X 10.6 Snow Leopard)
  • Click the “Manual Setup” button in the bottom left, to configure the base station
  • Select “Extend a wireless network” under the “Wireless” tab

It’s a little tricky, but there is no option to extend a network under the default set-up flow.

This provided three benefits:

  • All the devices in the living room are now connected via Ethernet to the Airport Extreme base station.  Significantly less chatter on the network.
  • The newer devices in the house are now seamlessly connecting via 5Ghz when they can to the Living Room base station
  • The bedroom devices are selecting the living room base station instead of the office due to signal strength.

Basically, there is a fairly constant 2.4GHz wireless “pipe” between the Living Room base station and the Office base station, and devices through the house are auto-selecting to the best connection.  The living room is aggregating the traffic over it’s ethernet switch and wireless endpoints, and then piping to my office network when necessary.

In the office, my iMac (which is my iTunes server) is connected via a Gigabit Switch to the Airport base station, the Infrant ReadyNAS NV+, and the AT&T Uverse Router.

I’m assuming that the bridging implementation between the two Airport Extreme base stations is extremely efficient – more efficient than having a large number of device independently competing for access to the base station in the Office.

I’ve already noticed now that the new AppleTV 2 devices are extremely happy with this setup, and I get 720P HD streaming in both the living room and the bedroom from the iMac with only a few seconds of buffering.  Hopefully, this should prove a durable and performant topology for 2011.

Steve Jobs is The Mule. Is There a Second Foundation?

This blog post could have been titled “We don’t live in the universe of maximum probability“, but that didn’t sound quite as exciting.

This weekend, I was having a friendly debate with a close friend about the state of the open web, when the now typical issue rose up: Apple, it’s support of native applications, and the resulting impact on the web.  I immediately thought about the fact that, in the 1990s, we would have never have dreamed of the technology landscape of 2010 — a landscape where Apple was the dominant force in mobile computing.  A world where we would see a massive resurgence and interest in client applications (yes, that’s what those pretty iPhone and Android apps are).  A world where Apple was the most valuable technology company in the world.

Then it hit me.  The parallel to one of the best science fiction stories of all time.  In fact, it’s the story that led to the name of this blog.

Isaac Asimov’s Foundation Trilogy.

Asimov’s Foundation is based on the future history of the Galaxy, when a lone scientist, Hari Seldon, invents a new science called “Psychohistory“, that allows him to predict the future.  This science allows him to project that the Galactic Empire will crumble and bring about 30,000 years of dark ages.  Instead, he develops a plan to create a “Foundation” to preserve knowledge, and reduce the period of regression to a mere 1000 years.  Unfortunately, his plan is disrupted by an unpredicted complication.

Check out this synopsis from wikipedia, and see if it sounds familiar:

The Mule is a fictional character from Isaac Asimov‘s Foundation series.[1] One of the greatest conquerors the galaxy has ever seen, he is a mentalic who has the ability to reach into the minds of others and “adjust” their emotions, individually or en masse, using this capability to forcibly enlist them to his cause. Individuals who have their emotions adjusted behave otherwise normally, with their logic, memories and personality intact; even if they are aware of the manipulation, they are unable to desire to resist it. This gives the Mule the capacity to disrupt Seldon’s plan by invalidating Seldon’s assumption that no single individual could have a measurable effect on galactic socio-historical trends on their own, due to the plan relying on the predictability of action of very large numbers of people.

Tell me that doesn’t sound like Steve Jobs.  You can read the full article here.

Just replace:

  • “Steve Jobs” for “The Mule”
  • “Apple” for “The Union of Worlds”
  • “The Open Web” for “Seldon’s Plan”

And I think you have a fair approximation of what’s happened in the last five years.

One of the hottest debates in mobile right now is whether to focus on the mobile web or native applications.  Ironically, Apple is the one who started this debate, since they were the first company to launch a phone with a truly modern web browser (Mobile Safari), and then proceeded to launch a simple, accessible native application platform on top of it.

In all seriousness, the reason that the native applications on the iPhone (and iPod / iPad) are such a viable threat is due to the fact that they are working.  When I say working, I mean that any company who takes their mobile web property, and then deploys a native iPhone application, tends to see a significant boost in their engagement metrics.  Apple has solved a distribution and engagement problem for mobile applications at an unprecedented scale, and it shows in the numbers.  Metrics usually speak louder than philosophy when making tactical decisions, which is why you see the incredible investment and interest in native applications for iOS devices.

In the story, the Mule is defeated by the Second Foundation, and rendered harmless and without ambition.  He dies without a successor, hence the name “The Mule”.

I think the question we should all be asking at this point is, “Is there a Second Foundation?

2010 Pinewood 8th Grade Graduation Speech

Today, at 6pm, I was invited to Pinewood in Los Altos Hills to give the commencement speech at their 8th Grade graduation.  I graduated from Pinewood junior high school in 1987, so it was somewhat of an honor for me to be asked to come back 23 years later to speak to the graduating students.

I wrote the speech last night (on an iPad) at the local Starbucks.  After a number of twitter questions, youtube searches, and other research, I decided to adopt the high level framework from Steve Jobs 2005 Commencement speech at Stanford, replacing his stories with my own, and adding my own form of 8th grade humor.  I did stick with his “dots” lesson, but you can see I changed the lesson from it quite dramatically.

Overall, quite a few people seemed to enjoy the speech, as a number of the students, parents and faculty came up to me afterward.  It seems like the students liked the jokes at the beginning, while the parents liked the third story on painting behind the refrigerator.

While I ad-libbed a few jokes, the notes below are exactly what I brought up onto the podium with me.  Let me know what you think.

Ice Breaker:

  • Last time I gave the 8th grade graduation speech here it was 1987
  • Weighed 85 pounds
  • I was 12 years old
  • Had to stand on a milk crate to reach the microphone to give my speech

Who am I now?

  • I have a wife, 3 beautiful boys, and two really fat dogs.
  • I am an executive at one of the cooler technology companies in the Valley right now.
  • It is part of my job to buy and play with every single new tech toy that comes onto the market.  Yes, it’s true. It’s my job to get the iPad the day it comes out.  Yes, I get paid for it.

(By the way, I appreciate you laughing at all my jokes.  If you don’t think they are funny, don’t be afraid to just laugh at me.  I’ll take it.)

Humorous Anecdote:

Wasn’t sure what to speak about.  Fortunately, they have this thing called the Internet now, and it’s pretty good.  I have over a thousand followers on Twitter, so i asked the for ideas.  I searched YouTube.  Poked around Facebook.  Even asked my younger cousins, who are in junior high now.

Not surprisingly, the ideas were spectacularly bad.

  • Some people said I should include a lot of quotes from Family Guy. I did a search and found over 768 funny quotes from Family Guy.  I’m 99% sure that literally none of them are OK for me to say out loud here.
  • Other people said I should ask the girls whether they are on Team Edward or Team Jacob.  I don’t really even want to know what that means.
  • I got a suggestion to talk about video games.  Apparently, Splinter Cell: Conviction is just awesome.  While that’s probably true, I’m not sure what to tell you about games except that you should treasure these years – once you have kids, you pretty much have until the age of 7 and then they start beating you.
  • Apparently, a lot of people think it would be funny if I gave a lot of advice to the boys in the class about girls.  Unfortunately, I still don’t understand high school girls, so not much help there.  Girls, in case you are curious about high school boys, all you need to know is that they really don’t mature much from here.  Don’t overthink it.

Anyway, since none of those ideas panned out, I decided I would cover three stories today and keep it relatively short.

I am going to tell you some things tonight that you are not going to believe.  But they are true.  Just three stories about:

  1. Coins
  2. Volleyball
  3. Painting

First, Coins.

  • There are a million little things that make you, you.  Don’t ignore them.  When I was little, i loved numbers.  I used to punch 2x2x2 into the calculator until it got too big for it to display.  Yes, I know that I am not normal.  I’ve always been a geek.   But who knew that knowing all the powers of 2 would be a uniquely valuable skill when it came to computers?
  • Hobbies are good.  You’ll be surprised where they’ll take you.  I collected baseball cards and coins.  Yes, I’m a dork.  At the time, I had no idea that I’d end up at business school, and that I’d have a natural sense for markets and trading.  I also had no idea that 20 years later there would be a company named eBay, or that it would do $60 Billion in sales.  I also had no idea that I’d end up working for that company.
  • Steve Jobs said a few years ago that a lot of life is about connecting the dots.
  • The wonderful thing about high school is that you are still busy adding dots to your picture.
  • You’ll spend your life connecting a lot of these dots, but it may not be for years or decades.
  • Don’t let anyone discourage you right now from learning and investigating.  If you find something interesting, don’t let anyone tell you that it isn’t worthwhile or cool. Pursue your hobbies, and do them deeply.  You’ll be constantly surprised later at how your life connects the dots.

Lesson 1: Draw lots of dots.

Second,  Volleyball.

  • In my senior year of high school here at Pinewood, I was a starter for the Varsity Volleyball team.  This was a big deal for me, largely because I wasn’t actually always good at Volleyball.
  • In fact, when I first tried out for the team my sophomore year, I didn’t make it.  (The fact that I was 5’3″ at the time may have been a factor).  I made the team my junior year, but mostly as a substitute.  But I practiced.  2 hours a day.  Extra trips to the gym, practicing against the wall, etc.  I didn’t make starter until senior year.
  • There are two types of skills in this world: ones where you’ll have natural talent and ability, and ones where you won’t.   Everyone is different, and I was pretty fortunate to be naturally talented in a bunch of areas.  But there are far more things out there that you won’t be naturally gifted at.
  • Don’t limit yourself to the things you’re good at.  Everyone is afraid of looking foolish, and that keeps a lot of us from pursuing things that we’re interested in, but that we’re not immediately good at.   Don’t fall into that trap in high school.  If you are interested in something, don’t just try it.  Do it, and do it well.
  • Pushing forward and mastering something that you’re not naturally great at gets you way more than just a skill.  It teaches you persistence and diligence.  More importantly, it gives you the confidence to learn and do anything.
  • It also teaches you to not take your talents for granted, and how special it is when you *do* have a unique gift in area.

Lesson 2:  Don’t limit yourself.

Lastly, I promised to tell you about painting.

  • I’ve always liked to work with my hands, and now that I have a house, I’m always doing something to it.  When you paint a room, like the kitchen, you always reach a difficult point – do you paint behind the refrigerator?
  • After all
    • no one else will see it
    • you can fix it later
  • But in the end, there are good reasons to paint behind the refrigerator.
    • first, you know it’s there
    • take pride in your work
    • act as if people are watching
  • Character is what you do when no one is watching
  • Important in high school, tremendously important in college & adult life
  • Some of the worst things that important people have done in the past decades have been because they thought they could get away with cutting either legal or ethical corners when no one was watching.  Many of you will turn out to be important people someday, and like they say, practice makes perfect.

So if I leave you with anything

Lesson 3: Be the type of person who paints behind the refrigerator.

Congratulations to you all.  Thanks for having me here today.  Take care.

iPad Tips & Tricks: My First 36 Hours

Around 11am on Saturday morning, I purchased one of the first iPads from the Apple Store in Palo Alto.  Yes, that one.  (I missed Steve Jobs by about 20 minutes).  I thought it might be useful to readers if I captured my initial thoughts, tips, tricks and gotchas from my first weekend with the device.

What I bought & why

I decided to purchase the 64GB Wi-Fi only model.  Rationale:

  • Wi-Fi is everywhere I’m going to use this device.  I don’t travel very much, so this device will be used mostly at home, relatives houses, work, and the occasional Starbucks.  When I travel, I tend to fly Virgin America and stay at hotels with Wi-Fi.  Paying the $130 premium for the 3G model, plus the $30/mo. data plan seemed unnecessary.  Besides, if I need it in a year, will be cheaper to sell this one and buy a new one than to pay all that money just for the “option” of using 3G.
  • I realized that storing a dozen movies and few TV show seasons is pretty likely.  Movies are 1.5GB each typically, so not hard to see using 30-40GB for video on the device.  So yes, I’m one of the suckers that paid $200 for 48GB of flash.  Rip-off, yes.  Pay-to-play, I guess.

Initial thoughts with the physical device

  • It is much thinner than I expected. I realize this is an illusion based on the increased screen size, since it’s not thinner than an iPhone.  But it’s surprisingly thin to look at and hold.  Still feels sturdy, though.
  • Orientation. Despite the natural orientation vertically, it really wants to be held horizontally.
  • Surprisingly, it feels safe to “let go”.  When I’m sitting down, I find myself letting go of the device in my lap, and using two hands.  Not to touch type, mind you, but to hit different areas of the screen with different hands.  For the first few minutes, I used it like an iPhone, but once I let go, I got much faster with the device
  • The touch is addictive. The combination of the speed of the device and the touch interface makes it incredibly engaging.  You just want to interact with it.  Everyone who has played with it has said the same thing – there is something about this form factor that demands engagement beyond the iPhone or iPod Touch.
  • iBooks is gorgeous. If Apple worked a deal where you got the physical book and the iBook for one price, I’d be on this train immediately.
  • The new Video Player is great. I was confused at first, because I expected the video in the “iPod” application.  But once discovered, the new Video app is excellent.

Takeaways

  • iPad native applications are an order of magnitude more interesting than iPhone apps.  While a few iPhone apps are enjoyable on the device, the full native applications are phenomenally better.  I didn’t realize this until I had the device.  The native interface controls, larger real estate, and the design of the native iPad apps (both Apple & 3rd Party) are phenomenally better.  So much so that I think the iPhone apps are just a numbers game for marketing purposes.
  • The pricing for iPad apps is higher.  Almost all the games cost money, versus the plethora of free games on the iPhone.

Gotchas / Minor Problems

  • It was not obvious to me at all why I couldn’t rotate the display.  Turns out, the hardware switch on the device is not a vibrate/mute switch like it is on the iPhone.  It’s a “rotation lock”.  I spent at least a few minutes looking through every setting on the device to figure out why it wouldn’t rotate.  This is a place where a simple contextual help bubble when you click the lock icon would have helped.  This is a UI blunder.  Let me put it this way, if you wanted to confuse the maximum number of users, you would design a device that looks exactly like an iPhone, use a control exactly like the iPhone, and then change it’s function.  I think they should have left the mute button the way it was, and added a rotation lock separately.
  • Some MP4s won’t play. Still trying to figure this out.  Some of my older Handbrake generated MP4s won’t play on the device, even though they play on the AppleTV.  This seems foolish – all these devices are powerful enough now to deal with the files that play on Mac Minis.  They should make them more tolerant of all MP4s.
  • Storage is stingy for video. As predicted, I quickly filled up the device with photos & video.  Let’s hope they take these to 128GB and 256GB soon.
  • Volume & Rotation Lock on the right, not the left. Why did they change this from the iPhone?
  • Adding Apps from the App Store is a pain on the device. They should make this easy.  Instead, every time you click install, it boots you out of the App Store, and puts the icon on a new page.  Why?  Did they really want to torture iPad users?
  • Get Ready to Spend Some Time on Configuration. My goodness, Apple.  Did you really need to make it so that by default, all my iPhone apps move to the iPad as well?  What a mess.  I had to uncheck all of them, and manually check the ones I wanted.  I could have really used some intelligent defaults, like initially only installing apps that were suitable for the iPad.  This was a miss for the iTunes team.

Final Assessment: Is it worth getting the iPad 1.0?

Absolutely.  Believe the hype.  This is a transformative device.  The engagement of the native iPad applications tells me two things:

  1. Native > Web.  Look, I love HTML 5.  I love web development and web apps.  But the seamless touch interface, animation, and graphical richness of the iPad native applications has raised the bar again.  HTML 5 may help web apps catch up to the dated, stale interfaces of Windows and Mac applications.  But the iPad is something different.  The story on the iPad will be native apps for the next 2-3 years, minimum.
  2. Touch is for real. Watching my 5 year old with the device, I realize that touch is the 80 in the 80/20 of future device interactions.  I’m realizing how many things I don’t really need a physical keyboard for.   And the mouse?  Except for detailed graphic design, I don’t need it at all.  I’m not saying keyboard/mouse is going away anytime in the next decade.  But I am saying that an increasing number of use cases may not need it.  I can already see how photo viewing via touch will give way quickly to photo editing via touch.

LinkedIn for iPhone 3.0 is LIVE!

Just a quick note to say that the new version of LinkedIn for iPhone is now live in the iTunes App Store.

Download LinkedIn for iPhone

I wrote a fairly lengthy piece on the official LinkedIn blog, so no need to replicate the full walk-through here.  In any case, check out this new home screen:

This application represents a huge achievement for the team.  It’s really a complete redesign and re-architecture of the entire stack supporting the application, based on an end-to-end design that was driven by user feedback and business metrics.

Building iPhone apps is a wonderful throwback in some ways to the days of client software, except with the advantage of over a decade and a half of web-based architectures.  There is a richness to client applications that the web still doesn’t replicate, and a complexity and depth to their design that is often under-appreciated.

Of course, the team had fun too.  The “Themes” feature, for example, was never part of the original plan.  It was originally a last minute easter egg that we included for fun in internal testing.  It proved so popular, however, we felt like we had to include it for everyone.

There are hundreds of things I love about this new application.  Even the way it presents a user’s profile is thoughtful, as LinkedIn is designed to allow you to put your best foot forward as a professional:

Of course, I wouldn’t be a product manager if I didn’t also have hundreds of things I’d like to see improved in the application.  It has been fun to watch the Twitter stream all day, as the feedback has been mostly positive.  Still, while this application represents a big leap forward for LinkedIn on the iPhone, it’s really just a beginning.  What’s most exciting about the architecture of this application is that it will let us rapidly innovate and improve the mobile experience through 2010 and beyond.

So here’s a quick shout out to the team – thank you for the hard work and effort in 2009 to produce an iPhone app we can be proud of.   I couldn’t be more excited for 2010, as we change the way people think of mobile business applications.

Mac Pro Crash Recovery: A Tale of 36 Hours

Yeah, it was that kind of weekend.

I went to check email Saturday morning, and was greeted with quite a shock.  My Mac Pro was locked in some sort of grey screen.  No icon, no progress, nothing.

A quick press of the power button confirmed it – simple power down.  No real OS boot.

On Friday night, before bed, I had shutdown the machine.  Some apps had been misbehaving, and I thought a full shut down & reboot was in order.  Apparently that reboot had failed.

I don’t know how “normal people” deal with problems like this.  When I say “normal”, I mean people who haven’t actually developed software on the Mac, who haven’t worked repairing Macs, and who haven’t spent countless hours futzing with their own machines.

Just in case its useful, here’s what I did.  The good news is that it proves out the benefit of using backup software, like Time Machine.  The bad news is that it also proves that this stuff is still way too hard:

1) Tried to reboot. Yes, I know, not rocket science.  But there is always that hope that just rebooting will magically “fix” the problem.  In this case, rebooting went into an endless loop.  Grey screen, Apple logo, spin icon… then grey screen and reboot.  Kept repeating.  Bad news.

The lack of either the blinking folder or the regular boot sequence told me I was on dangerous ground.  It was either a hardware issue, or the system was corrupted.  In either case, the machine was not getting to the normal boot sequence.

2) Tried to boot of DVD. For those “Dodgeball” movie fans, “If you can dodge a wrench, you can dodge a ball.”  For Macs, “If you can boot of a DVD, then your hardware can boot anything.”  It’s not totally true, but true enough.  In my case, it proved harder than you might think.   The machine wasn’t getting far enough in the boot sequence to load Bluetooth, so my wireless keyboard was useless.  Fortunately, I keep a USB keyboard around.  Plugged in, holding down the “C” key (nostalgia: the “C” is for CD, and they never migrated to “D” for DVD.)  In any case, if there is no DVD in the drive with a bootable OS, it opens the tray.  Got the tray opened, popped in the Mac OS X 10.5 Leopard DVD, and began to boot.  Total time spent: 20 minutes.

3) Diagnose the boot drive. On the Mac OS DVD, a little known trick is that while the installer is running, you can go to the Menu Bar, and select “Disk Utility” to run diagnostics on your disk.  I did so, and discovered some bad news.  My main system drive, a 300GB Western Digital, had problems.  Worse, Disk Utility basically told me that I was crazy if I thought it could fix them.  Drat.  Time spent: 10 minutes.

4) Get Mac OS X installed on another hard drive. Running the system off DVD is slow, and you are limited in options without a full Finder.  Fortunately, my iTunes HD had a few hundred GB free.  Installed OS X on that drive and rebooted.  While that happened, I went to have breakfast and actually get productive chores done.  Time spent: 30+ minutes.  Who knows, I didn’t come back to the machine for several hours.

5) Assess whether the system drive is lost cause.  I was ready to run down to Fry’s to get a new HD (or better yet, a new SSD.  Why not turn tragedy into opportunity?)  Unfortunately, the disk mounted.  Interesting.  I did get a strange system warning that I’d never seen before, telling me the disk had problems and that I should reformat it.  Never waste access to a dying disk.  I immediately tried to use Disk Utility to create a disk image of the disk, but it failed.  (You do this by dragging the hard drive icon from the desktop over the Disk Utility application).  Some cryptic error.  Fortunately, a Finder copy of my user directory worked, providing an extra backup of files, just in case.  Time spent: 20 minutes.

6) Reformat system drive. Well, Mac OS X told me to, right?  I was surprised, but I tried it.  Disk Utility was able to reformat the drive – I noticed the old formatting was Mac OS X, without journaling enabled.  Wow.  Was the drive that old?  In any case, I reformatted with the appropriate GUID setting for booting Intel macs, and with journaling enabled.  Afterward, a quick Disk Verify confirmed a shocking outcome… the drive was fine.  Time spent: 10 minutes.

7) Reinstalled Mac OS X on System Drive. Tempt fate?  Sure, why not.  This was the first time I had a hard drive crash after using Time Machine, and I was eager to try it out.  When you install Mac OS X 10.5 now, it asks you if you are migrating from another machine.  You can specify a Time Machine backup.  I was pleased to see the last one was from 10:59pm on Friday… less than 1/2 hour before the “great crash of 2009”.  Unfortunately, this process seems to take hours.  160GB of material for some reason took over 3 hours.  No way I’m sitting around for this!  Time spent: 3 hours+

8) Get everything up to date. I came back to the machine that evening.  It booted, seemed fine.  Even had my old accounts on the login screen.  I signed in, and everything looked normal.  All files/folders in the right places… except iPhoto failed to launch, and iTunes complained that it didn’t understand my library.  Whoops.  The DVD installed Mac OS 10.5… but we’re on 10.5.6 these days, and my apps and files had been upgraded.

Brief rant: I’m really wondering why Apple hasn’t tied the update logs from it’s automatic updates to the restore from Time Machine.  It’s pretty obvious that the Time Machine backup has a system on it that has a series of updates installed – would not be hard to boot the OS with instructions to download and install those updates.

In any case, Apple Mail tried to “import” my Mail folder.  I cancelled that and quit.  iTunes offered to create a new library, and I declined.  Phew.  Hope I’m safe.   Ran the System Update system preference, and discovered about 10 updates waiting for me.  Downloading them and installing would take… 2 hours!   Let it run over night.  Time spent: 2 hours + one nights sleep.

9) Get everything up to date… again. In the morning, after breakfast, checked on the machine.  Was booted, looked fine… except now Apple Mail had lost all of my old mail, and iPhoto still wouldn’t boot.  iTunes was fine, though.   Ran System Update again… and there were another 8 updates, clearly waiting for the last 10 to run.  Great.  Fine, let’s update some more.  Time spent: 1 hour + me leaving for the morning.

10) Restore Mail. Thank goodness I’m paranoid.  I copied the “Mail” folder in my “User > adamnash > Library” folder from the “extra” backup I had made to my System drive.  3GB to copy, but hard drive to hard drive over internal SATA 2 bus is wicked fast.  Time spent: 15 minutes.

11) Everyone lived happily ever after. It was about 11:30am on Sunday, literally about 36 hours since the crash happened.  And everything was back to normal.  Seriously, I doubt you could have easily figured out anything had happened.  Even little details like my browser history were there.  Firefox re-opened with the same 20 tabs I had open on Friday.  It was if the last 36 hours had been a test, and since I had kept calm and walked through the steps, I had passed.

So, what did I learn from this?  A few things:

  • Keep a Mac OS X boot DVD handy. Most people lose track of this, because it came with their Mac when they bought it.  Don’t lose it.  I prefer the retail disc myself – it’s worth the cost to have one.
  • Disk Utility is your friend. There was a time when Apple utility software sucked, and you had to go third party.  There are still superior third party tools out there (and for serious hard drive crashes, you need them.)  But these days, starting with the standard Apple software is a good bet.
  • Migration Assistant has come into its own. I’ve used it now for work and home.  It’s very good.  Not perfect, but better than hand-crafting system restores.  Very impressed with the Time Machine integration.  If it was smart enough to handle Apple Update history, I’d be truly happy with it.
  • Don’t underestimate the value of an extra hard drive. The reason my restore was relatively painless is that I had another hard drive that I could boot the system off of.  Without that, you have to depend on the DVD.  Ouch.  If you have a tower, and extra hard drive is cheap insurance (and extra storage).   If not, consider a cheap firewire external drive.
  • Time Machine is good. Look, if you care about your files, you should backup.  Period.  Time Machine makes it painless.  I’m really impressed – backup systems are only really tested when you need them, and I needed Time Machine today.  It came through.
  • Beward of hard reboots.  The reason my system had problems is likely due to a software conflict I had been ignoring – XTorrent and my .Mac screensaver.   I would come home to a locked up machine, and would be forced to hard reboot the system.  Hard reboots = increased risk of file system damage.  I played Russian roulette one too many times, and paid the price.  36 hours of it.

Mostly, however, I discovered that after 18 years of fixing/restoring Macs, it’s still stressful dealing with a crash like this.  I just can’t imagine why any normal human being would know or care about all the steps above, or how they would be expected to keep multiple backups, hard drives, and techniques handy to manage this type of issue.  It’s 2009 for goodness sake.  By now the computers should be taking care of themselves.

In any case, I hope the above proves useful to a reader or two.  If not, maybe the story will prove either entertaining or depressing, depending on your perspective.

iPhone 3.0 Event Next Week: March 17th

top

Got the graphic from CNET.  They have some details about the event:

Apple distributed invitations Thursday for a March 17 special event in Cupertino, Calif., to discuss the iPhone 3.0 software and a new software development kit.

Next Tuesday’s event will come a little more than a year after Apple unveiled the original SDK at the iPhone 2.0 software event, setting the stage for over 25,000 iPhone applications to make their way onto the App Store. Speculation about a new iPhone had mostly centered on new hardware features, rather than software upgrades, but it seems Apple has something up its sleeve.

Hoping to see OS-level support for some missing basics:

  • Clipboard (cut & paste)
  • Background processing (some form of mult-tasking where apps can receive updates even when they aren’t front-most)

I’m wondering if we’ll see any significant hardware enhancements or new models announced.  The iPhone currently drives developers to really focus on a single screen size… would be nice to see more robust handling for multiple sizes/shapes to give more flexibility to hardware in the future.  It’s not that you can’t make resolution-independent applications today – you can.   It’s just not encouraged or optimal.

MegaPhone?

iTunes Pass Could Be the Key to Digital TV

Read this news today with great interest:

Apple, EMI unveil iTunes Pass

Apple has just launched a new service called Pass for its popular iTunes music store.  It’s like a season pass for a favorite artist, in this case the electro band Depeche Mode…  Before thinking this is Apple’s entree into the music subscription business, which is something Steve Jobs has pooh-poohed in the past, note that iTunes Pass is quite different.

Under an all-you-can eat music subscription plan at a place such as Rhapsody, you have access to the material only as long as you keep paying a fee. With iTunes Pass, you own the content that has been downloaded, even after the pass expires.

I’ve written a bit about this in the past, but it’s shocking to see Apple so close, and yet so far, from what might be a truly disruptive innovation in digital television.

The iTunes Pass is the right idea, but the wrong market.  We don’t need this for music, we need this for digital television & movies.

When Apple launched Apple TV, it highlighted the ability to subscribe to a season of television.  The idea was you pay a fee up front, and then as episodes of that show come out, they would automatically download to your iTunes (and then synch with your AppleTV), allowing you to watch a show with both digital convenience and without waiting until the end of the season to buy a DVD.  This was a great concept for a few reasons:

  1. Ability to easily “catch up” with a season/show that was already underway.  This is a very common problem, particularly with serial shows where you miss the first few episodes before friends/news reaches you with a recommendation.  (This is the reason I didn’t get to watch 24 in real time until Season 3…)
  2. Ability to “own” the shows permanently – not a transient state like a traditional DVR.
  3. Ability to watch on TV.  Let’s face it, that’s where you want to watch the show, not on your PC.
  4. Automatic download, in the background.  Shows would be waiting for you as they appeared.

The problems, however, with the execution were equally significant:

  1. Lack of seasons.  For many shows, Apple provided the current season, but not previous seasons.  Thus, if you wanted to watch “Lost”, you couldn’t “catch up” with Seasons 1 & 2.
  2. Timeliness.  Next day would have been OK, and in the beginning that was the plan.  In reality, some shows would show up days or weeks later.  Ideally, the download would literally begin at the official showtime EST in the US.
  3. Pricing.  They used DVD pricing, which frankly is just ridiculous.  The idea that you’d reasonably pay $30-$50 for a season of TV that can be had through traditional distribution channels for free makes sense in the historical model of buying specific shows/seasons in low volume, but isn’t a mass market play for general television consumption.  If in a given season (Winter 2009) I’m watching 6-8 shows, there is no way you are going to get $200+ out of me for the privilege unless you 100% substitute for my cable bill.

The pricing issue in particular led me to a hypothetical model that could potentially benefit both Apple & the networks by disintermediating the traditional cable/sattelite duopoly.  Basically, every network could attempt to become “HBO”.  HBO pioneered the idea of a premium channel – an extra monthly charge you’d pay for unlimited access to content.  Thanks to the VCR, that also included the ability to “time shift” that content for your own use.

On iTunes, NBC could be as valuable as HBO.   Imagine an iTunes Pass where, for a monthly fee, you could subscribe to The Office.  You would automatically get new episodes as they come out, as well as download old episodes (like a podcast).  These would all be watchable on AppleTV, as well as your iPhone.  That’s something worth paying for.

Now imagine that for a monthly subscription fee, you could actually do that with any NBC show.  The Office.  My Name is Earl.  ER.  30 Rock.   Whatever you want.  You would become an “NBC subscriber”.  NBC would have a premium revenue stream, and would then focus on providing high quality content to lure in new subscribers, and to keep existing subscribers.  They would also have their own “distribution channel” within iTunes – they could now launch new shows and pilots for a fraction of the cost & risk by delivering them to people automatically, and making the marginal cost of subscribing to a new show effectively zero for the user.  Sunk cost.

Subscribing to a network could be an upsell from an individual hit show.  Subscribe to The Office for $5/month.  Or subscribe to all of NBC for $10/month.  Network families could offer the same bundle of channels to individuals that they currently offer to the cable & sattelite companies.  Get the entire family of NBC channels for $15/month.

There is likely some price in that ballpark where NBC would be agnostic between someone watching them on cable vs. subscribing on iTunes.

Per-show pricing doesn’t get you scale.  But owning a customer relationship as a network is incredibly valuable.  These subscriptions could also be platform agnostic long term – no reason you can’t have versions that support Amazon, Apple, Microsoft, Netflix, etc.

People already are beginning to question their premium movie-channel cable subscriptions in favor of Netflix/Blockbuster service.  Network subscriptions could substitute for the primary cable bill.

Right now, I watch shows on Fox, NBC, ABC, HBO, SCIFI & FX.   There is a price where I’d gladly shift over to a digital subscription to get the benefits of the content combined with the benefits of professional, digital files that I could watch anywhere, anytime (time shifting & location shifting).

The thing that I love about this model is that, from a games theory perspective, there is significant value to the first “defector” – the network that moves to this pricing model first.  For example, if HBO offered this service through iTunes, I’d subscribe immediately.  Obviously, the cable & sattelite companies would fight this tooth & nail – but I’m not sure they have a leg to stand on in preventing this.  After all, Comcast can’t prevent NBC from offering channels to DirecTV, etc.

iPhoto ’09: Fix for JPEG Files Displaying as Pure Black on Edit

I’m sharing this fix with the world, so that others need not live my pain.

Last night, I returned from Lake Tahoe with 451 beautiful shots of our family snow trip, all taken with my Canon 40D SLR.  Each shot was captured in both large format JPG and RAW format.

Unfortunately, after loading all my images into iPhoto ’09, I ran into a real problem:

When I double-clicked any of the JPG files to edit/view them, they displayed a purely black screen.  It was strange because the thumbnails were fine, the RAW files were fine, and when I opened the JPG files in Photoshop CS3, they were fine.

There was no way around it.  Relaunching iPhoto did not help.  Rebuilding the library did not help.  Rebuilding thumbnails did not help.  Reloading the images from the compact flash card did not help.

I shuddered to think about the wisdom of upgrading to iPhoto ’09.  After all, at least iPhoto ’08 could display JPG files.  My only hope: the Canon 40D is a popular camera, and has been out for a while.  This must be a solved issue.

My searches on Google turned up a few articles and discussions, but nothing convincing.  Some threads on the Apple Discussion forums.  A post or two on other Mac sites.

Fortunately, I found the answer.  But let me first tell you what it wasn’t:

  • It wasn’t the PowerPC (I have an Intel-based Mac Pro)
  • It wasn’t file size
  • It wasn’t iPhoto ’09
  • It wasn’t the Canon 40D

Unfortunately, several sites fingered these things as culprits.  All wild goose chases.

Here is what it was:

  • A corrupted install of Mac OS X 10.5.6

Hard to believe, but the auto-update I had done just before leaving for vacation was the culprit.  Thanks to one tip, I downloaded the full combo installer for the Mac OS X 10.5.6 Upgrade from Apple.

A full re-install of the update, a reboot, and all was well.

I hope this tip finds someone out there in good stead.  Seeing your precious photos reduced to a black screen is frightening to the core, even if you know the photo files themselves are not corrupted.

Read Lag Problems on Seagate Freeagent 1.5TB External Hard Drive on Mac OS X

This is one of those quick posts that I write hoping to save others a lot of time.

For the past two weeks, I’ve been trying to diagnose and fix problems I have been having with my external 1.5TB Seagate Freeagent Hard Drive.  I use the hard drive as my iTunes library, which I drive FrontRow using Mac OS X 10.5 on a Mac Mini on a 1080P LCD TV in the living room.

Basically, it’s our video server.

The problem was as follows:

  • Loading Front Row, we’d easily get to the list of movies (I have about 500+ movies on the system)
  • Flipping between titles, however, there would be a 3-5 second lag before the cover art and title of the new movie would load
  • The lag would also occur periodically while scrolling
  • The light on the front of the hard drive would pulse, instead of glow straight, during video operations

Needless to say, with a video library driven by remote, this type of lag is intolerable, and makes navigating a 500+ movie library incredibly frustrating.

After doing a number of Google searches, I found reports of read/cache problems on Mac OS X and Linux with the 7200.11 drives.

Here is an official note from Seagate on the topic, and here is a community thread on the issue.  Thinking I needed new firmware, and since there was no download on the website, I resigned myself to calling technical support.

I called Seagate Technical Support, and after waiting 20 minutes, I got on the phone with a man who had never heard of the issue, had no idea how to get the firmware for an external drive, and no idea what a Mac is.

Fortunately, while he put me on hold for 15 minutes on 3 separate occassions, I had time to think.

The problem cited on line was a read issue with video, but it seemed to focus on RAID setups.  That didn’t sound like my situation.  The light was pulsing, likely because the drive was trying to lower power use.  Why would it do that?

Answer: Maybe it was overheating because I had it in the cabinet with the sattelite receiver, which is warm, and there is poor ventilation.

So, I took the drive out the cabinet, and reattached it in the open air, on top of the cabinet.

Problem solved. Video navigation is snappy, and the light no longer pulses.  Clearly, because of the heat, it was going into low power mode in some way to try and cool itself.  That would lead to lags when it spun up for a read request from FrontRow.

So, if you have a Seagate Freeagent Extreme drive, and it exhibits this behavior, make sure it’s not overheating in an enclosed space.

As a funny aside, not only did I solve this entire problem while listening to hold music with Seagate Technical Support, I also wrote this entire blog post.  The guy is *still* off somewhere, trying to find the firmware revision for my model of drive.  I’m worried that he booked a flight to Malaysia to investigate the manufacturing of this drive personally.

Oh well.  The end story:

Unventilated enclosure for hard drive, BAD.  Seagate Technical Support, TERRIBLE.  1.5 TB iTunes Hard Drive, GOOD.

Update (1/20/2009): Unbelievable.  It looks like Seagate released the firmware patch for the 7200.11 hard drives this week, and it actually bricked the drives.  Not for me.  I’m watching this thread, and I’m not installing anything from Seagate until it’s safe.