Why are release notes/change logs important?

I got involved in a Twitter/X conversation about whether release notes/change logs are important. It started with this post:

I can understand this sentiment from a consumer perspective. But after having spent 14 years delivering enterprise sales software, I believe release notes are undervalued at most software companies.

(Note: I’m using “release notes” and “change logs” interchangeably here.)

Release notes are important in enterprise software to the admin in charge of the software that you’ve sold to them. Because eventually, someone important above them (think: a VP or higher, often not directly in their reporting chain) will ask, “WTF! This software was working and now it isn’t. Admin, did the vendor change something?”

Admin then has to go figure it out. This is outside of their normal workday and set of responsibilities. Best case for the admin, they look at the change log, see that something did indeed change (and ideally for the better), and can copy/paste that back to the Important Person. Then they can go back to looking at Tiktok videos in between checking email.

The less good case for the admin is that they don’t find anything in the release notes. They then have to reach out to support/CSM and try to figure out what is going on. This takes time and adds to their work. They don’t like that.

If you can provide thorough, accurate release notes, this can increase the chance that the Admin hits the “best case” described above.

Some additional thoughts:

  • The more critical the software is in a revenue-generating workflow, the more watchful the admins will be to the vendor’s change logs/release notes
  • Many companies don’t bother with detailed App Store change logs, because they publish these directly in their product and away from competitors eyes. You wouldn’t believe how much critical information we gleaned from competitors about their product capabilities and core focus simply by watching what they published to their public change logs
  • Around 20-30% of our client admins overall watched our change logs at Mediafly. In some industries, that number was 100%

A practiced method to solve hard problems

Here at Mediafly, we are faced with hard engineering, product, sales and marketing problems every day. Each of us takes a different approach to solving these problems. Some of us like to create pros/cons lists. Others dig deep into data and use that help answer every question. No one approach is the “right” approach for everyone.

I recently had a conversation with our Engineering Manager[1], and he described his approach to solving hard problems.

  • Take an attempt to solve the problem, but don’t stress about it if you can’t figure out the solution yet.
  • Review the key aspects of the problem right before you go to sleep. This involves working on the problem from multiple angles. Meaning, if you tried one solution and it doesn’t work, try another. If it did work but it’s ugly, just note the key parts of why it works and why it’s ugly. You need to get intimate with the problem and be really familiar with it from all angles.
  • Now, let the problem go. Sleep on it, take a shower, go for a run. Do something to take your mind off of it entirely.
  • When you least expect it, an insight will find you. When the solution does find you, immediately explain it to as many people as you can. Don’t worry about whether they are an expert in the subject domain. Just start explaining. The mere process of explaining acts as a forcing mechanism to refine the solution further. It also serves as a filter; if what you thought was initially a great idea turns out to not be, attempting to explain may allow you to filter out the seemingly-good idea much more quickly, and get back to solving the problem another way.

I’ve watched him apply this method of problem solving over the years, and it truly is a thing of beauty. He will often take 2, 3, 4 attempts at particularly thorny engineering problems. He will sometimes throw away the code he wrote for an attempt and go back to the drawing board. He will restart this process from scratch as necessary. But, regardless, he almost always comes up with a solution that solves the problem elegantly. And watching his success has led me to begin adopting this approach for problems of all sort that I face as well.

[1] Special thanks to @laimis for being the inspiration for this method and this post! And he credits A Technique for Producing Ideas as inspiration for this process.


Navigating the Product Roadmap

Navigating the Product Roadmap

As is the case at many other startup/early growth phase companies, we at Mediafly haven’t hired a product manager. “Product management may be the one job that the organization would get along fine without”, and we’ve lived by that model for the past few years. Because of this, we operate lean and create our own best practices for the meaty parts of product management, most notably the product roadmap.

The product roadmap is necessary for any enterprise-focused SaaS company that is selling multi-year subscriptions to Global 2000 companies like we are. It’s used as a communication tool for:

  • Clients, both the ones signing the checks as well as the heaviest users/admins of your product. They want to understand (and influence) what is coming down the pipeline for your product
  • Product development team members, including designers, developers, QA, and infrastructure. They want to understand what they should be thinking about from a UI/UX/architecture perspective
  • Sales and marketing. They want to know what they can sell, how it should be marketed, and provide input from a client, competitive and market-driven standpoint

Defining the Roadmap

An Example of Mediafly’s Product Roadmap

It can be easy to let the roadmap balloon into a complex, time-sucking monster. The roadmap itself is housed in Google Sheets, and broken up by product category (for us, those are SalesKit, ProReview, Airship CMS, Interactives, Reporting, andInfrastructure/Security). We organize specific entries in the roadmap into five columns, with the following definitions:

  • Discovery: Ideas/features that are “baking.” We usually hear of these ideas from clients, prospects, users, or the market, and are looking to gather more information for these ideas before we move them to the next phase.
  • Evaluation: Once an idea has gathered sufficient input, it moves into the Evaluation phase. During this phase, we’re putting together benefits statements, wireframes, high-order engineering estimates, and public-facing selling documentation.
  • Plan: After we’ve committed to building an idea as a feature, we attach a delivery date (usually a quarter, sometimes a month), and move it to Plan. Here, we are working on detailed requirements, wireframes, and phasing. We are also conducting usability tests and starting to talk about the upcoming feature with our clients and prospects.
  • Build: Once the feature reaches our design and engineering teams, it is in Build.*
  • Release: Recently released features are moved here. At this point, we’re training our clients, updating documentation, and analyzing usage in preparation for improving our newly released feature.*

It’s All in the Details

Sample Mediafly Details Document

Some entries in the product roadmap will be hyperlinked to a Details Document, which is hosted in Google Docs alongside the roadmap itself. These documents will vary in structure, but often contain:

  1. Estimates
  2. Requirements
  3. Current and proposed approaches
  4. Discussion of tradeoffs and limitations
  5. Specific customer requests

Every 3-4 weeks, I will review the roadmap with our business leaders (currently, our CEO and EVP of Sales and Marketing). We focus on what’s changed since our last meeting and discuss new data that we’re seeing from clients, prospects, and the market. Then every 2 weeks, I will review the roadmap with the Product, Engineering and Customer Success teams. We focus on what’s coming up in the near future, and often will dive into the specific points to discuss better ways to solve specific problems.

Ad-hoc, I will be asked to present the roadmap to our customers. This is usually accompanied by a short presentation that explains what the roadmap is and how it works. Over the course of any week, I’ll spend about 1-2 hours reviewing and adjusting entries within the roadmap based on conversations, research, and incoming data.

We’ve managed to keep the product roadmap useful and relevant without too much overhead over the past few years by following this model. As we grow the entire Mediafly team through 2016 and beyond, this process will certainly evolve. I’m looking forward to that evolution, and will keep you up to date as it progresses.

If you’re involved in product management at your organization, please comment below with your best practices. I’d love to hear from you!

(Cross-posted on the Mfly Blog)

A frustrating experience with returning to Google Drive from a Doc/Sheet/Slide

About a year ago, it became clear that Google Drive was attempting to focus on Docs, Sheets, and Slides as separate products. They launched separate apps for Docs, Sheets, and Slides on iOS and Android. It makes switching between a Doc and a Sheet more annoying. Maybe they are following the new trend in apps for a service to create a bunch of mini-apps (for example Foursquare’s recent split for the Swarm app).

But you know what’s really frustrating? Working with Google Docs/Sheets/Slides on the web. The service assumes that, because I am working on a spreadsheet now, the next document I work on MUST ALSO be a spreadsheet.

Google Drive Sheets upper left button

Here, I am working on a Sheet. But the next document I want to work on is a Doc. The only path out of this Sheet that I can see on the page is the giant green box next to the title of the Sheet. If I tap on that, I get sent to a list of other Sheets.

Sheets

 

Why, Google, why? Why don’t you send me to your beautiful, robust Google Drive home? There, I can get a view of everything I’m working on.

Instead, I find myself:

  • Opening a new tab
  • Typing drive.google.com
  • Then having to switch accounts (because, of course, Drive defaults me to my personal account and not my work account)

If someone over in the Googleplex can hear me, please consider changing this default behavior to point back to Drive.

Praise and criticism of “The Other IPO Roadshow”

Deepak Jeevan Kumar of General Catalyst Partners wrote a pretty good overview piece on TechCrunch describing his point of view of how to launch a F500-focused enterprise software company. Many parts of it ring true based on our experiences at Mediafly and my previous experiences:

CIO offices in Fortune 500 companies are trained to play it safe. Many companies stick to Oracle, VMWare, EMC and Cisco not because their products are the best in the world, but because no one got fired for selecting one of them.

That’s so very true. Often this is driven by their own experiences, but just as often it’s driven by internal politics. We heard recently from a customer (who was discussing another part of their business, thankfully) “if we don’t go with Microsoft for [terrible product A], our prices will go up on [pretty good product B used somewhere else], and we can’t have that.”

Do not use unpaid PoCs even if you have to wait one to two months to get a paid PoC.

This is very important. Customers that don’t put money behind a proof of concept simply don’t have enough skin in the game. We’ve been on the wrong end of this story before:

  • Prospect says “well, if you have feature X, I’ll be willing to buy then.”
  • You spend days/weeks/months building feature X, in hopes of securing their business
  • You show feature X to prospect
  • Prospect says “Great! Now, if you have feature Y, I’ll be willing to buy then.”
  • Repeat

Customers who are not willing to engage in a paid proof of concept are not worth it, no matter how large.

 

Some parts of Deepak’s perspectives don’t seem right:

Second, power and influence in the early days can come from public silence for enterprise startups (unlike consumer startups). Your competitors, the media and your customers like a game of treasure hunt to find out what you are doing.

In reality, as a “stealth startup”, no one really cares what you are doing. And the last thing customers want to do is to spend time trying to dig to learn your secrets. They are usually too busy running their business.

As you are selling to a select group of Fortune 500 customers, there is no point in announcing to your powerful enterprise tech competitors (e.g Cisco, Oracle, IBM, HP) what you are up to. Convince your design customers first before you open up the kimono.

Again, even if your largest competitors figure out what you are up to, it’s highly unlikely that they will try to usurp your untested, unproven ideas. They are busy running their businesses and executing on their product and company vision, to bother trying to knock you out. Of course, all this changes when you start to generate real traction and, more importantly, win deals from under their nose. But you are not at this stage yet, pre-customer startup.

That being said, I do agree with the general premise that you shouldn’t spend time talking about the product too publicly. But for these reasons. I believe that’s the case because you should be 100% focused on executing on delivering your product to these large companies that are taking a chance on you. Spending time engaging in PR is a waste of your time, as you haven’t proven anything yet.

 

Overall, this article is definitely worth a read for anyone starting down this road.