Myths of Innovation – Why problem definition is more important than finding solutions

Please subscribe for more articles.
We promise not to spam you.

I recently finished reading Myths of Innovation by Scott Berkun – and its a great book that I will definitely recommend for everyone Read. The reason I bring him up is his theory of generating idea and defining them is similar to what I face as a product manager while defining the problems. The process of Innovation is challenging and somehow with working knowledge, we all know what is required to innovate. However, there are still some aspects and stories that can always inspire us. Myths of innovation solidified my knowledge about following the path of idea generation. Key, as Scott also states, is to have a lot of ideas tested and crossed out. You need to have a bank where you can always go and pick an idea from which to test and develop a hypothesis. The goal during idea generation is to think without constraints and filters. Once you have a substantial problem, follow the path of learning as much as possible before committing to a solution.

My principles of product management align here – As a PM, I am committed to exploring the problem space to make sure that there really is a problem in the first place. Understanding the root of the problem will enable you to think of solutions that will attach at different levels, and I promise you will find a simpler solution.

In a corner somewhere inside me, I do feel there is a factor of luck that plays a vital role in the success of an idea. Considering your idea goes through 8 phases or processes of work and each phase could be a success or a failure, the probability will yield a success rate of 0.39%. Newton, Edison, Archimedes, Steve Jobs, Bill Gates, etc. were hardworking and great individuals who also got lucky. One thing I always tell myself is honoring luck doesn’t diminish my accomplishment.

Leaving luck aside, there is always a path and process towards innovation or idea fulfillment. Here are some steps that will help you at least get started in this direction.

  • Idea generation – The first part of this process is to generate ideas in bulk and think without constraints. Scott identifies these rules for exploring and creating the idea as:
    • Produce as many ideas as possible
    • Produce ideas as wild as possible
    • Build upon each idea
  • Self-knowledge – There must be self-knowledge that makes you aware of the environment or challenges that inspire the best results for your personality helps you make smart path choices.
  • Pivot or Persevere – You need to be intense in your approach at the same time be ready to step back and analyze the correctness of your path. It’s the combination of intensity and willingness to reconsider assumptions, minimizing the chances of following dead leads and maximizing the potential for finding a better path.
  • Exploring the problem space, more – The most critical path towards a successful innovation is how much have you explored the problem space. Problem-finding and excessive obsession with researching the problem space will provide the path towards success. And this is not new, especially to product managers, we have known this, the bigger question is do we follow this approach. Jumping to conclusions, inaccurate surveys compilations, wrong interpretation of customer expectations, and creating a problem that does not even exist plagues us to show success and work to just deliver results in short term. Good innovators or PMs are always working on hypothesis development and validation. Product sense is very important, and it includes much more than just finding and solving the issue.
    • Intuit is one of the great pioneers and their product management stories have personally inspired me to the core, especially their “take me home” initiative to learn how customers use their product. Scott Cook, CEO and Founder of Intuit, shows us how problem definition can lead to a success story. While working on the turbo tax, he figured the competition is not the software tax industry and it’s not about creating the best tax software in the market. According to Cook, the biggest competitor was pen and paper. Pens were resilient and solid substitutes and yet the entire industry overlooked the pen and paper. He creatively shifted the focus of his team to find a solution to replace pen and paper. Now the challenge was not to create the best software but to enable people to feel comfortable switching to computers to file taxes. This is all about creating mind shift change to define problems that really required the solution. This is what other inventors have done as well and this is the reason all great inventors were more than lucky. They picked real problems and worked on framing and defining problems to create a successful vision.
    • Another prime example is Edison, while he is acknowledged for the development of the light bulb; he was very late to the party. So why has he been considered the great genius when he did not even start working on the problem first. His success also came from designing the problem differently. He thought of light bulbs as a system, asking questions like, “How do you get power to home to power the lightbulbs? And where does that power come from?”. Creating the lightbulb alone was useless and Edison knew why. Edison was unwilling to create great lightbulbs that no one would buy. The real task to him wasn’t about making a working lightbulb as we all had known him for. Instead, Edison’s framework of this problem was to make an electricity system in cities that can use to adopt his lightbulb
    • There is a similar story for PDA, “Personal Digital Assistants“. For decades, people talked about handheld devices that could manage your calendar, contacts, and personal information. In the 80’s and early 90’s HP, Siemens, Sharp, and Apple invested millions in new product development which all failed. It seemed like a personal assistant would not be developed until 1996 when Palm Pilot successfully overcame the challenge. PDA became a billion-dollar industry, influencing the design of phones and computers forever. The key success to Palm’s success was that they defined their challenge differently than other competitors. Instead of focusing on engineering constraints or loft dreams of revolutionizing the industry they focused on what customers wanted. Jeff Hawkins the CEO of the palm went to the people/users and started documenting their needs. He soon had this list
      • Fits in a shirt pocket,
      • Syncs seamlessly with PC,
      • Fast and easy to use
      • No more than $299

These goals in 1994 when palm started exploring the space were unrealistic given the technology available. Handwriting recognition, color displays, fancy keyboard were all nice (could have) ideas but not necessary (must-haves) for the customers. The great thing is how he wrote his goals. Like the first one, the goal was not to be small or handy, but that it fits in the pocket of the shirt. Andrea Butter and David Pouge who wrote “piloting palm” wrote:

“Hawkins, who presided over these meetings, was unyielding when it came to keeping what he saw as a non-essential feature out of the product. If the new machine were to fail it wouldn’t be because it had been junked up with unnecessary functions, like its predecessors…. Soon the team became experts at killing features”

Andrea Butter and David Pouge – Authors of Piloting Palm

Please subscribe for more articles.
We promise not to spam you.

All these greats worked really hard and knew that odds of success are against them. Any wrong step would lead to millions of dollars in debt and can put you and thousands of others working for you, out of business. However, the path they took to define problems led them to success. There were still chances that their invention were not acceptable for that time or by society due to multiple reasons. My understanding from this book and in general says that change management is the biggest constraint in customer adoption of your product. Convincing the customers to try your product is one of the biggest challenges we face as product managers. And change management could be because of a multitude of factors including:

  • Culture – Innovations do change societies, but they must first gain acceptance by aligning with existing values. Japanese invented firearms before Europeans but their culture saw swords as the symbol of value, craftsmanship, honor, and respect. The goodness of innovation plays a critical role in the cultural adoption of products. Innovation can be good for you but how do you judge its goodness for others, industry, economy, society, and world. Ukraine war is a good example, of the use of drones to bomb cities on both sides of the border. How do we judge the goodness of drones in this case? Drones initially were thought to be for entertainment purposes, then we thought they will be good for logistical purposes and last-mile delivery. Now bombing cities with drones in the recent war between Ukraine and Russia, what is the score on the goodness scale here? People who support Ukraine in the war may say Ukraine is using drones to save themselves and hence are right, whereas Russia is wrong. This ethical dilemma on cultural adoption has played and will always play a critical role in the success of product developments.   
  • Dominant design – This goes back to my point on change management. People once used to the idea of using one technology or using technology in a certain way, it’s very difficult to change the user behavior. Either your product should offer an improvement on the existing design or provide a significant upside to the user to switch.  QWERTY keyboard design created by Christopher Sholes was developed so that the mechanical keys on the typewrite won’t Jam. He did not envision millions of people and future designers sticking to QWERTY. Now changing to another format is very tough, there has to be a valid reason for me to switch because I will have to learn the whole new typing style.
  • Inheritance and Tradition – When you confuse your comfort for a belief as it is good. US measurement system is a typical example of a tradition and inheritance, why learn a new system even though 190 out of 193 countries are using a metric system. The strongest push in the U.S. came at the start of the 20th century, Alexander Graham Bell, and other notables testified at congressional hearings on metric conversion. The head of the new Bureau of Standards put forth the metric system as a vital national interest. But charges of elitism and wasting money came from a public that increasingly believed the U.S. should be the leader in global affairs and not just another follower. An interesting fact is NASA lost a $300 million Mars orbiter because one equation failed to convert units from the US to English units. Due to this $300 million orbiter was sent on a certain path of destruction.
  • Political and economic – Will you hurt someone with your invention. Once an inventor has resolved the issue does not mean that society will accept the solution. For society, innovation is always associated with feeling good or bad. The effect of these feelings can either empower or reject your authority. For example, when Galileo claimed the sun was the center of the solar system, he faced persecution from the Church and the western world. It was not the idea that probably caused this outrage. The church never cared about what was the center of the solar system. They were not angry because Galileo suggested Earth rotated around the Sun, they were angry because their authority was challenged. So innovative ideas are not rejected based on their merit, they are rejected based on how they make people feel.

The economics associated with financing the invention to make it available to the customer is tricky. No one knows how the market will behave to the new invention. No one knows (especially the investors) if there is a market in the first place. The fact of the matter is ideas are rejected, not on their merit, but on their value to the priorities of the moment.

As managers, we have a responsibility to safeguard our ideas when necessary. That is why working for great companies that support idea generation is a tremendous opportunity because they provide an environment where open discussions about the life of ideas happen. Managers build an environment to ask questions and brainstorm on hypotheses as a group.

Innovation flourishes in the greenhouse. What do I mean by a greenhouse? A place where the elements are just right to foster the growth of good ideas, where there’s heat, light, moisture, and plenty of nurturing. The greenhouse we’re talking about, of course, in the workplace, the way spaces take shape in offices and teamwork together.

Tom Kelly, GM of IDEO and the author of “The Art of Innovation”

I can quote Peter Drucker here who wrote:

“Management tends to believe that anything that has lasted for a fair amount of time must be normal and go on forever. Anything that contradicts what we have come to consider a law of nature is then rejected as unsound”


Having said that, inventions are never performed alone. This brings me to another point were these gentlemen alone in their inventions, if not why only Edison or Newton or Archimedes became famous, what happened to their team? The most common convenience is the order of exposure. Most of the world learned of the idea of electric lights and bulbs from Edison. In people’s minds, Edison is the deliverer of the idea. Even if later, it was discovered and proved that others had the idea first or made a working lightbulb before him. So can we say, whoever is most visible in bringing something new, will forever be associated with that thing? One popular example for the readers of our generation is Apple Inc., well recognized as the innovative company behind user-friendly Mac, iPhone or iPad, etc. However, history shows that the first graphical user interface and the desktop computer were made by Xerox PARC and SRI system in the 70s, nearly a decade before Apple Mac which was launched in ‘84. The first iPod sold by apple was in 2001 which was way after Sony Walkman first sold in 1979 or digital music players from SaeHan, Diamond Multimedia, and Creative Labs in the late 90s. Apple much like Edison, earned the well-deserved credit for vastly improving existing ideas, refining them into excellent products, and developing them into businesses. But Apple did not invent the GUI or the computer mouse or digital music. Similarly, Google did not invent the search engine.

E=mc2, the most famous five characters of the world, credited to Einstein were based on concepts that came from many people. Faraday, Lavoisier, Newton, and Galileo were essential building blocks that made that formula possible. Each contribution, E for energy, M for mass, and C for speed of light – was a concept developed by others: Einstein’s breakthrough and genius was his approach in bringing them all together.  Innovations rarely involve someone working alone and never has in past inventions happened without reusing the ideas from the past. Wise innovators initiate partnerships, collaborations, and humble studies of the past, raising their odds against the timeless challenges of innovation.

At the crux of it and after reading Myths of innovation, I can certainly say innovations are not epiphanies but rather a hard work of years, the process of finding ideas, evolving those ideas, and definitely, Public Relations has helped our innovators overtime to prove their worth. Their timing of understanding societal needs have helped with success. If an innovation occurs at the wrong time your invention may not see the daylight while you live. We all know technology adoption happens in phases and is unlucky for you if you are developing something ahead of your time. Maybe people will remember you posthumously for your contributions.

As mentioned earlier as well, innovators go through three phases: “Early”, insights, and after. In the early period, you frame the problem and basically become one with the problem. You explore the problem space and associated areas, gain knowledge, and understand the domain. The “insights” period is the period of incubation of knowledge, you digest this knowledge by experimentation and rough attempts at a solution. During incubation, you struggle to find the solution and there are long pauses when progress stalls. Berkun says, big insights if they happen, occur during the incubation period. During incubation you start observing the world from a different lens, you start to relate from what you have read, and in your subconscious, you are always thinking about the problem and its solution. “After” is the period when you have your epiphany (which is a myth by the way) to come up with a solution.

Please subscribe for more articles.
We promise not to spam you.

Nikhil Varshney

Nikhil Varshney is a product manager by profession and technologist by nature. Through this blog he wants to showcase disruption in the technology world. The idea is to break the concept into simple layman words to help everyone understand the basics