“The Bitter Lesson” and open source LLMs

Today I learned of a 2019 paper called “The Bitter Lesson

This paper postulates that, over the past 70 years, the biggest drivers of AI advancement were not due to special human-introduced nuances into what makes the model smart, but rather, dramatic advancement in computational resources driven by Moore’s Law (exponentially falling computational cost).

What does this mean?

Major AI leaps are driven not by companies rolling in a ton of “special sauce” into their AI models, but rather because it becomes dramatically cheaper to throw more hardware at the problem.

This gives me hope that the future of LLMs won’t be beholden to companies like OpenAI, Anthropic, and the like. But rather, we’ll see open source models catch up to and possibly surpass OpenAI’s GPT for raw text-to-text generation.

I am seeing that some open models claim to be on par with GPT 3.5, such as Mistral-7B and orca-2, but the empirical evidence is mixed. (p.s. hat tip to Anton Bacaj, who is a wealth of cutting-edge information around open models. https://twitter.com/abacaj )

Of course, there are other competitive areas where the private companies’ inherent advantages will allow them to dominate over open source models (marketing, enterprise features, APIs, wrapper support, stores, integrations, etc.). But at least the core offering won’t be wrapped up in a tight little, expensive, box.

Getting a Dell OptiPlex back up and running with Microsoft Windows 7

This post chronicles the month-long adventure in getting one of our Dell OptiPlex 3020M (you know, the tiny one with no CD/DVD drive) desktops back in good working order.

First, the problem:

  • Boot times start taking extremely long to get to the Windows loading screen, and then it would spin endlessly
  • Safe mode boot didn’t help this process along

As we were still under warranty with Dell, I reach out to them. Here is what happened:

  • I emailed Dell Support. They replied very quickly, and after a couple of days of back/forth with their tools, we isolated a bad hard drive.
  • I went through Dell Diagnostics with them (which is roughly baked into the BIOS). Turns out the hard drive was bad.
  • They sent me a replacement hard drive via FedEx Overnight.
  • I swapped it out and started the desktop. After booting to Windows and getting through a few prompts, I get stuck in a never-ending Dell configuration loop that is (what I call) the gray-on-black modal dialog of hell. Loop runs for 24 hours with no progress or hard drive activity. I can’t do anything (mouse and keyboard are disabled).

Dell OptiPlex 3020M stuck in gray on black loop of hell

Dell OptiPlex 3020M stuck in gray on black loop of hell

  • Dell Support directed me to download an .iso file that contains a preconfigured Windows 7 for my system. I didn’t know such a thing exists! Now I want to go get one for every one of the machines we maintain…
  • I download it, burn the .iso to USB with Rufus on the Surface Pro we have here (remember, no DVD ROM drive on the desktop I’m trying to recover), and try to get it to run.
  • I am able to boot to it and get through a few prompts. But very soon, I encounter a new error: “A required CD/DVD drive device driver is missing.”

"A required CD DVD driver is missing" dialog

“A required CD DVD driver is missing” dialog

  • I tried a number of things suggested by others on the Internet, including: switching the USB to another port midway through the installation, downloading Dell’s drivers and having Windows attempt to find the correct driver, buying an external CD/DVD writer, trying to burn the .iso to DVD (doesn’t work), and trying to just install Windows 10 from a later Dell machine onto this machine (also doesn’t work). Many hours were wasted.
  • I reach back out to Dell and complain about how much time I’ve wasted. After complaining, Dell Support sent someone onsite to replace the hard drive (again). After this was done, I was back at the neverending gray-on-black loop of hell.
  • I opened a new support ticket with Dell, this time around OS issues. I was sent new instructions for creating the Windows 7 Recovery USB key, which largely maps to this article. The article was very interesting, because:
    • It recommended using the diskpart Windows command line tool, vs. Rufus which I used previously, and
    • It didn’t quite work as is, as there was one significant discrepancy I discovered from those directions. I have a 64GB USB 3.0 key. I had to create a 16GB FAT32 partition (which is known to boot well; I can’t trust NTFS or ExFAT as boot partitions) vs. simply creating a full-disk 64GB partition. It took me a few tries to figure that out.
  • Success! This got me through the two key issues I had in the past (gray-on-black and missing device driver)
  • But, I encountered another problem: an error that said “Windows cannot be installed on this disk. The selected disk has an MBR partition table. On EFI systems, Windows can only installed on GPT disks.” Thankfully, this link indicated that I can safely delete all of the existing HDD partitions and create a new, big one. I did so, and the installation continues.

I wish the process wasn’t as difficult as it was. But at least I’m thankful that it works, and that my interactions with Dell Support helped me down the correct path, eventually. They were patient during the process, and had no problem sending their onsite tech to try to assist, which is the best I can hope for with business-level support.

Dear Enterprise Software Product Managers: Consider Scale

5425752305_7023df6048_o-1

Discussions about design and UX for software these days so often focus on onboarding. Scott Belsky, founder of Behance, even suggests A good discipline to help you stay simple is to focus at least 50% of your effort on onboarding and the first-time-user-experience. Providing a great onboarding flow is the quickest way for your users to find value in your new feature. After all, the sooner a new user is able to find value, the more sticky it’ll be for them, and the less churn you’ll experience, right?

Makes sense, and it gives a great starting point for how to think about a new feature. For example, from Mediafly’s point of view:

  • A newly signed-up user will start with 0 content, 0 salespeople, 0 users. Envisioning that scenario is very straightforward.
  • A small business might be using SalesKit by Mediafly to manage 50 pieces of sales collateral among 20 salespeople, distributed to 500 prospects. There is some complexity with this level of information, but for most features you might design, it’s probably pretty straightforward.

Often, however, this is where the design of a feature stops.

When you’re working with enterprise organizations with large user counts, diverse business processes, very large data sets, or whatever key metrics you track, however, you need to consider the user experience when there is high volume of use in these key metrics.

Example 1.

From the beginning, we designed Mediafly’s content management system (CMS), Airship, to start as simply as possible. From day 1, users could drag in content from their laptop, with reasonable defaults, and immediately get value. As our customers adopted our CMS and scaled out their use across diverse business processes and groups, we continued to discover issues that we could never have foreseen at launch.

Recently, a large customer (a major CPG enterprise)began uploading merchandising layout diagrams, hierarchically organized by region, for each of their tens of thousands of their customers’ stores to our system. This dramatically increased two key metrics: their volume of content (tens of thousands of new documents) and frequency of updates (thousands of changes every week).

Automating the upload and management process on their end is a no-go, as there is no common backend system where these documents reside. And, asking people to update these layout diagrams with our Airship CMS would require 20-40 hours a week of navigating, clicking, and dragging/dropping.

To address this, we conceived of a new upload model in which an adminstrator of a one of their region’s merchandising layout diagrams could organize the new content hierarchically on their laptop, zip up the file, and upload it into our system. We would then interpret the results and update the content automatically in the correct location. This solution at once solves both the problem of content volume and update frequency. And it can be reused for other customers who encounter similar challenges.

We spend as much time solving user experience challenges of scale as we do thinking about how to build compelling new features whose adoption will begin at very low volumes.

Example 2.

We recently released the ability for our content administrators to create special links to view content, which has been a hit with our Media/Entertainment customers. The link can have a password, be tied to a user account, or be public. Creating a link is straightforward, and initial reception and usage of this feature started off as very positive.

But, after a few months, we began to hear feedback from content admins about some challenges they were feeling as their use cases for links expanded. The volume increased dramatically in some of these use cases. We now see that some admins have to create as many as 200 individual links for individual users in a single day, usually around television pilots or key screening seasons.

After diving deeper into some of these workflows, we created a process diagram to show what the typical process is to create a link. The content admin:

  • Switches to their email client and composes a new email
  • Pastes in a template that they use for their emails
  • Switches to Airship
  • Finds content in the hierarchy
  • Navigates to the Links tab
  • Taps Create Link
  • Configures the link
  • Saves the link
  • Clicks the option to copy the link to the clipboard
  • Switches back to their email client
  • Pastes the link in
  • Sends the email

Whoa, that’s a lot of steps. Imagine having to go through this process 200 times in one day! For some admins, it requires the entire day.

We have since simplified the process to create a large number of links, and continue to improve upon the feature to solve the problem of high volume even further.

How these experiences have changed us

As we design new features, we now include an extra question to answer: What will this look like at high volume? At the design phase, we strive to have a hypothesis on how we would address problems of scale, and to see what we can do to simplify the initial UX even further should scale arrive faster than we can roll out a redesign.

However, just like most things we do from a product and engineering perspective, we operate iteratively. We certainly won’t prematurely optimize for scale. But by simply adding this question to our checklist of considerations, we’ve opened up the ability to solve the seemingly inevitable high volume issues that will arise.

(This post was cross-posted from the Mediafly Blog.)

Announcing SalesKit Meeting Tracker

(Cross-post from Mediafly and LinkedIn)

Here at Mediafly, we work hard to make the lives of salespeople better. We talk to a lot of salespeople, and what we hear from reps over and over again is “I hate my CRM.” This feeling comes from many reasons:

  • While CRM is very valuable to sales management, salespeople find these systems add little to no value to their primary job as salespeople
  • After a long day or week of meetings, salespeople have to drag themselves into their CRM to record notes of what happened in each sales meeting they had—instead, they want to be working towards their next deal or spending time with their families
  • CRM systems in general are clunky, complex, and require a lot of clicks/taps to accomplish the most basic of tasks

What does SalesKit Meeting Tracker do?
Once SalesKit Meeting Tracker has been turned on for your company’s environment, tap on the “Meetings” button and start tracking your meeting.

Start Meeting

When you’re done, stop tracking, record the meeting’s details, and send them to Salesforce.com.

 

Complete Meeting

Your meeting’s details will appear within the Activity History for the account, contact, and opportunity you’ve chosen.

What’s coming next?
We have a rich roadmap for SalesKit Meeting Tracker, including integration to our other app platforms (web, Windows, Mac, Android) and to other CRM platforms (Microsoft Dynamics, SAP Cloud for Customer).

Strategy is Culture

Mediafly in the Inc 5000

(Cross-posted from the Mediafly Blog)

When I used to envision how small companies create a great strategy, I would often imagine a single or multiple leaders sitting down and writing out something to the effect of “here are the three things that we’ll do to win.” The picture in my mind is complete with a boardroom or conference room, men and women wearing business suits, and hours of PowerPoint presentations at the ready. My experiences at a management consulting firm reinforced this: oftentimes, decisions started as hypotheses by the most senior leaders, and were then justified back to them by consultants who gathered and made sense of data.

After growing Mediafly by 657% over the past three years (and that’s not even including 2014 or 2015 numbers), I’ve come to shift my mental picture of how strategy is built at small companies. The big conference room is replaced by a small one, often with people on the other end of GoToMeeting; suits are replaced by jeans; PowerPoint is replaced by conversation with a few bits of data thrown in; and the confidence of “here are the three things that we’ll do to win” is replace by “here are what we think might be good to try, based on a little bit of data, a lot of intuition, and a willingness to adjust as we learn.”

But the biggest surprise in my mental picture comes from how we develop strategy. Our strategy is highly influenced and often derived by our incredibly empowering culture and system of ad-hoc conversations.

What makes our culture empowering?

At Mediafly, engineering, customer success, marketing, sales, and even interns have open, candid conversations with the executive team every day, passing along ideas and feedback in both directions. Customers can pick up the phone or write an email to anyone on the team, and expect to get help to solve issues. If team members find something they don’t like about any system, they are empowered to proactively fix it. Many team members work across divisional lines (engineering and marketing; sales and product; marketing and customer success), which empowers those team members to fix and grow even more.

What are ad-hoc conversations at Mediafly?

As a result of the culture, our team members find themselves in ad-hoc conversations about all parts of the business every day. Conversations will form at each others’ desks, over lunch or drinks, on our messaging systems (GChat, HipChat), and of course by email. These all serve to align how each of us thinks about our company’s place in the world. All of this coalesces and informs our company’s strategy.

A couple of great examples of what I mean:

Example 1:

Our strategy focuses on “content mobility”. Previously, this meant mobile (and primarily iOS) apps. However, several inputs came together to help us broaden and redefine this this past year:

  • Sales: We had several sales meetings in which it became clear that many companies simply won’t ever buy iPads for their sales staff
  • Product: After analyzing our usage data, we realized that the primary platform for using Mediafly is through our Web Viewer on a desktop browser (not a mobile device and not one of our apps)
  • Customer Success: A few conversations with existing customers crystalized that a desktop app would go a really long way to embracing Mediafly
  • Market: Windows Surface and touchscreen Windows 8 hybrid tablets are starting to gain traction

These inputs came in through our empowered team members thinking about their areas, via conversations at each others’ desks and at meals.

Assimilating these inputs was easy, when there were so many of them pointing in the same direction. As a result, we’ve begun to work on a desktop application and plan to release it within the next several months (stay tuned!).

Example 2:

We originally developed our Interactives Platform back in Q2 2012. At the time, we assumed Mediafly would build a consulting arm that caters to creating Interactives via consulting services to our enterprise customers, while our core team focuses on enterprise software licensing. We operated under this approach for almost a year, netting a reasonable amount of consulting work.

However, again, several inputs from our empowered team members, gathered via many conversations, helped reshape how we think about this:

  • Engineering and Product: Every time Engineering was pulled into an Interactive engagement during this time period, we had to delay work on the product side for a custom piece of work. However, the product side is where the bulk of our “enterprise value” would eventually come from.
  • Team: Getting sucked into consulting engagements of this form was extremely hard, and team members were working way longer than normal. This was largely because we weren’t used to the multiple back-and-forth design rounds required for consulting to overlap with product work, and time was tight.
  • Industry: we started to meet some great agencies and software development firms, and believed they could be incredible partners.

We made the decision in 2013 to terminate all new Interactives development for our customers (with a few exceptions). The result has been amazing: better products, better focus, happier customers, and partners who are doing amazing work.

As we continue to serve more customers with our phenomenal products and services, we will strive to:

  • Ensure team members continue to feel empowered to bring up ideas and issues,
  • Keep lowering friction for ad-hoc conversations to take place, and
  • Drive company strategy based on the output of our culture, as much as it is based on other external factors

A frustrating experience with returning to Google Drive from a Doc/Sheet/Slide

About a year ago, it became clear that Google Drive was attempting to focus on Docs, Sheets, and Slides as separate products. They launched separate apps for Docs, Sheets, and Slides on iOS and Android. It makes switching between a Doc and a Sheet more annoying. Maybe they are following the new trend in apps for a service to create a bunch of mini-apps (for example Foursquare’s recent split for the Swarm app).

But you know what’s really frustrating? Working with Google Docs/Sheets/Slides on the web. The service assumes that, because I am working on a spreadsheet now, the next document I work on MUST ALSO be a spreadsheet.

Google Drive Sheets upper left button

Here, I am working on a Sheet. But the next document I want to work on is a Doc. The only path out of this Sheet that I can see on the page is the giant green box next to the title of the Sheet. If I tap on that, I get sent to a list of other Sheets.

Sheets

 

Why, Google, why? Why don’t you send me to your beautiful, robust Google Drive home? There, I can get a view of everything I’m working on.

Instead, I find myself:

  • Opening a new tab
  • Typing drive.google.com
  • Then having to switch accounts (because, of course, Drive defaults me to my personal account and not my work account)

If someone over in the Googleplex can hear me, please consider changing this default behavior to point back to Drive.

IAM policy to allow full user access to an S3 bucket

Since I wrestle with this about once per month, here is the IAM policy to allow full user access to an S3 bucket.

{
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:ListAllMyBuckets"
         ],
         "Resource":"arn:aws:s3:::*"
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:ListBucket",
            "s3:GetBucketLocation"
         ],
         "Resource":"arn:aws:s3:::bucket-name"
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject",
            "s3:GetObject",
            "s3:DeleteObject"
         ],
         "Resource":"arn:aws:s3:::bucket-name/*"
      }
   ]
}

Amazon Web Services and “cost stabilization”

When deleting a large chunk of data from Amazon S3, AWS struggles a bit with figuring out how much you actually owe.

We at Mediafly deleted ~30% of the tens of terabytes of data from one of our AWS accounts yesterday. One would expect the appropriate cost graph to drop ~30% as well. However, this (obfuscated) chart, output from our installation of Netflix Ice, the excellent open source tool contributed by Netflix, appears as if we outright deleted all content from our S3 bucket.

Amazon AWS S3 costs after deletion

 

Seeing this graph raised some alarms (“oh crap! Did we just delete everything??”), but thankfully the reality does not match what was depicted here. Time will tell when and how that information is updated, however.

Not all DRM is bad (and other thoughts)

HTML5 VideoI’d like to follow on my earlier post that HTML5 needs DRM (specifically, Encrypted Media Extensions, or EME) with some other thoughts.

Not all DRM is bad. While technologists and consumer groups disagree with DRM in principle (users should own content they purchase outright) and implementation (DRM is easily circumvented, so why bother?), there is at least one legitimate use cases for DRM: internal content. Oftentimes internal content is meant for a specific person, group or organization, and is not meant for or ready for public consumption.

EME compartmentalizes DRM and brings developers and users back to the web. Today, the only options for DRM’ed video or document playback consist of heavy, closed plugins such as Flash or Silverlight, or closed apps that are entirely off the web (iOS, Android, Windows 8, desktop). Both of these options compete with the web as a platform, and steal developer and consumer interest away. But fencing DRM capabilities into an HTML5-compatible plugin pushes DRM into a background and can help bring development back to the web.

“No DRM” principles can’t change economic reality, but contained DRM can. The Free Software Foundation and Electronic Frontier Foundation push for “no DRM”. One of their many arguments is that DRM doesn’t work. A determined hacker can bypass any encryption or management scheme that exists, with the benefit of time. So why bother polluting standards with DRM? Yes, it is true that DRM can be circumvented with time, but it doesn’t change the economic and legal realities of today:

  • Companies that create content want some levels of assurance that their content won’t be stolen when delivered to third parties, and write this into contracts. No system better than DRM (specifically, encryption and revocation) exists today.
  • Simply providing content that is DRM-free but watermarked is insufficient (not to mention extremely expensive and technically challenging). I will likely write more on this later.

Baby steps to a DRM-free world. As Peter Bright @ ArsTechnica suggests, one of the benefits of EME is to allow content creators to tiptoe into DRM-free distribution, which may lead to an eventual transition to “No DRM”. Expecting them to dump DRM from all existing contracts and approaches without a gradual path to achieve this simply will not work because of the realities of today. And EME can provide that gradual path.

Further reading. Some good, thoughtful* articles on the pros and cons of this decision:

 

* Thoughtful compared to the “I hate DRM! Boycott Netflix!” one-sided nonsense that pervades vocal tech commentators today.