Last week I spoke¹ to a group of midshipmen at the US Naval Academy about innovation and change. I had not been back in 20 years and it was a wonderful experience to return. In the active Q&A afterwards, several midshipmen asked for the slides, but since the deck is crafted for speaking rather than reading, it doesn’t function effectively on its own. Here are some of the key points I covered.

Change

Young military leaders today are going to face more transformation than any previous group. This is not an assertion to make lightly. 1930’s graduates saw the battleship give way to airpower and submarines. My classmates and I trained to execute the Lehman doctrine when the Big Red Machine² poured through the Fulda Gap in Germany. Instead, we watched the Berlin Wall fall, the USSR collapse, 9/11 and tried to rebuild nations.

Changes over the next decade will dwarf prior periods because of two related trends: worldwide smartphone adoption and Moore’s Law.

As smart phones become ubiquitous worldwide, three billion people are moving from virtually no technology to the entire Internet³. Instead of adopting each communication technology sequentially, smartphones with data connections enable huge numbers of people to skip landline telephones and terrestrial broadband in the developing world. The smartphone will be the first computer for billions of people, more than doubling the number of people connected to the Internet.

Moore’s Law⁴ emerged from Gordon Moore’s observation that the number of transistors on a chip doubled every 18 months. Effectively, this meant that every 18 months, a computer’s computational power would double while size and power requirements would reduce by half. Moore’s Law has held up for 40 years, transforming computing capacity and efficiency.

This 18-month doubling grows exponentially. As shown on the graph below, exponentials start slowly and often under-perform a linear prediction. As a result, change is easy to ignore at the beginning. But after a few years of doubling, the growth accelerates quickly and rapidly passes the linear prediction. Exponential change results in disruption and dramatic upheavals.

Moore’s Law is constrained by physical infrastructure improvements. Building new chips means building hugely expensive new plants. Software growth is not hampered by these limitations, so change today can happen even faster than Moore’s Law predicts. New products and ideas can appear as quickly as a programmer can produce new code⁵. As a result, anywhere it becomes possible to put a computer, we move the rate of change to at least Moore rates and possibly much faster.

This should be concerning to anyone in the military, because militaries are built to fight the previous war. The US has been fortunate over the last several hundred years to overcome this problem by learning quickly. But how can we learn and adapt faster?

Learning

The fastest way to learn is leveraging existing information. Nobody did that better than Matthew Fontaine Maury⁶, who joined the Navy in 1825. The Navy in the 1800’s was absolutely critical to society, communication, and science. Until transoceanic cables revolutionized communication in 1858⁷, sailing ships were the fastest, most reliable way to transmit information. During his 14 years at sea, Maury studied and recorded weather and wind data while mastering the skills of the age, but he was frustrated because contemporary navigation techniques did not focus on finding the fastest or safest paths to destinations. He began writing under a pseudonym, publishing his weather observations, agitating for more modern navigation techniques, and arguing for a service academy to better train Naval officers.

While on leave in 1839 Maury broke his leg in a carriage accident, leaving him unfit for sea duty. Soon after, his writings were discussed in Congressional hearings and his identity exposed. His last request for reinstatement failed and Maury was assigned as the director of the Navy Depot of Charts and Instruments⁸ in Washington, tasked merely with calibrating marine chronometers via astronomical observations. Maury moved beyond his normal duties when he discovered the Depot’s cache of logbooks, delivered after every voyage. The logs contained regular recordings of location and wind, so Maury experimented with using this data to expand his previous observations of weather and wind. He began aggregating the data and looking for patterns.

Clear patterns emerged.

(This image is huge, click to embiggen.)

This is every voyage before 1838. There is no need to draw the continents since the data clearly defines coastlines⁹. (Thanks to NOAA, the Maury data is available for download¹⁰). Maury aggregated wind data in addition to location, producing a different view and a great reminder to avoid the North Atlantic in winter.

Maury’s next insight was that the best way to improve the quality of his charts was to collect more data. Although the initial maps were a strategic asset for the US Navy, he published them broadly. “Wind and Current” charts¹¹ in 1847, followed by “Sailing Directions” in 1848. In his 1855 book, “The Physical Geography of the Sea”¹², Maury assured the world that anyone who shared data with him would have access to the results, saying he wanted to publish data

“in such a manner that each may have before him, at a glance, the experience of all.”

Navigators used his data immediately. In 1849, the California gold rush was underway, so speculators flocked to the dangerous, 200-day voyage from New York to San Francisco around Cape Horn. In 1851, the clipper Flying Cloud left New York, trying to set a new record. Her navigator, Eleanor Creesy, built her course around Maury’s work. Just 89 days later, Flying Cloud arrived in San Francisco, setting a new record the stood until 1989¹³.

Within a decade after Maury first published, worldwide navigation had changed.

(This image is huge, again click to embiggen.)

Travel by sea shifted to a set of common routes, using the data Maury continued to publish. Common routes are clear. The animation below compares the decade before Maury first published to the decade after “Physical Geography” was published.

In 1853, he keynoted The Maritime Convention in Brussels¹⁴, where he proposed a standardized reporting format for meteorological data – if it was 2005, he would have proposed a public API to let computers talk to each other – to enable more efficient and accurate data collection. Within several years, most major navies were sharing their data with Navy Depot, now rechristened as the Naval Observatory.

Change

Any type of change is hard, but institutional change – with the need to overcome tradition and bureaucracy – is the most difficult. With the rate of change continuing to accelerate, the key skill set for any leader is the ability to recognize disruption, gather the data to act on it, and then effectively push for change. In the mid-1800’s, over the course of 15 years, a disabled Lieutenant changed the US Navy and the world. He did it by finding space to maneuver (as a trouble maker exiled to the Navy Depot), demonstrating value with his early publications, and creating a massive network effect by establishing the Naval Observatory as the clearinghouse for Navigational data. 150 years before Web 2.0, he built a valuable service around common APIs and aggregated data by distributing it freely to the people who needed it.

Maury’s story inspires me every time I revisit it. It challenges me, because if Maury could accomplish what he did in the age of sail, how could I aspire to less today?

notes

1. Thank you to Professor Mark Hagerott and everyone who helped pull this together!
2. No, not the Cincinnati Reds. The Big Red Machine referred to the ground forces of the Soviet Union, the Red Army. There was a scary amount of armor waiting to roar into West Germany back in the day. You can read about the Lehman Doctrine on wikipedia.
3. For lots of good data on how this will happen and the possible impacts, visit Internet.org.
4. Moore’s Law image CC BY and lots more on Moore’s Law via wikipedia.
5. Marc Andreesen talks about this as “software eating the world”.
6. Read lots more about Maury on wikipedia, The Navy Department Library, and his letters archived at VMI.
7. No easy task.
8. Precursor to the US Naval Observatory.
9. Ben Schmidt has done a much more thorough analysis on this data. I didn’t bother with machine learning, using very simple heuristic to reconstruct voyages from the data.
10. For details on how I pulled the data and the code used to generate images, checkout the repository on github.
11. You can see a copy of one of those charts here.
12. NOAA has it in their digital collection.
13. Lots of good links about the Flying Cloud and Creesy on wikipedia.
14. You can read the contemporary coverage of the conference in the New York Times archive.

Addendum

A few more renders of from the dataset that might only be interesting to me. First, compare this set from the 1860’s to the previous render from post-1853.

Second, the World War 2 dataset, where the Pacific island hopping strategy jumps out from the tracks.

And, finally, a gif animation of the voyage data from 1660-1850, a decade at a time.