It’s an experience as old as the personal computer itself. You’re working away on an Extremely Important Project, really making progress, when—ARGH! The app or operating system suddenly freezes up… and you hadn’t saved your work yet. You lose everything. DISASTER! 
This nightmare scenario is something that Microsoft Program Manager Juan Karmy and the rest of his team set out to fix, once and for all, several years ago in Office. As an Office “Shared PM,” he’s in an ideal position to help develop a cross-app solution for this common calamity.
“The job title basically means I work on features that are launched across the Word, PowerPoint, and Excel desktop client for Windows and Mac platforms,” Juan explained. It’s a position he moved into after about half a decade of working in a variety of tech roles both in his native Chile and at Microsoft, where he started as a developer on the Office for Mac team.
How Juan and others on his team developed a solution—dubbed smart save reminders—to handle the frustrating experience described above is a tale that has many twists and turns, encompassing different user scenarios, in-app pop-up messages, machine learning, the Office Insider program, and more. It’s all covered in our latest Insider “Behind the Scenes” interview!
How did you decide to tackle the “crash before saving” scenario in Office apps?
Juan: We started by trying to better understand what we internally call the “crash recovery” space. We looked at feedback from users, what our competitors were doing in that area, and where the potential gaps were in our apps.
We decided to tackle the issue from a couple of angles: reactive and preventative. On the reactive side, I spent an entire year digging into the data to figure out how many people crashed, how many saw our Document Recovery experience, and how many were indeed able to recover their data from that experience.
On the preventative side, we came across dozens, if not hundreds, of feedback comments from customers who lost critical documents, such as research papers. They were downright angry. So my focus went to, how can we help these users avoid data loss? Because I could feel their pain.
We wanted our products to have an intelligent way to understand when users might be working on something that’s worth saving, and then prompt them to do so. That’s how my team and I eventually came up with smart save reminders.
Why was solving for this scenario so challenging? Why couldn’t it just be, say, a 15-minute timer that then generates a reminder?
You mean, why would we over-engineer something that could be just done with a timer? That’s a good question.
It goes back to what problem are you trying to solve? The reason that we took a more nuanced approach is that our apps are used for a variety of purposes. For example, people use Word to write a thesis paper or a big report for work. But some also use Word to take notes or collect their thoughts, and then port that content to a different application without saving.
What were some of your other considerations?
We needed to figure out the best mechanism to prompt customers to save. We wanted to use a callout that pops up in the corner of the app, even though we know that users generally find those distracting. And these callouts are used for multiple purposes in the app, which could contribute to potential dissatisfaction. So that was a tricky one to balance.
The timing of the prompts was another issue we considered. We didn’t want to nudge users just for the sake of nudging them. We wanted to be able to hint to them that they could save their file at an opportune time, when we wouldn’t be interrupting their workflow.
How did you resolve that conundrum, and figure out how to program the app to nudge users at the right time?
We decided to rely on technology, more specifically machine learning. We wanted the feature to consider factors like the amount of content added to the document and the edits made, as well as the amount of time spent on it.
All of that information was used as signals to feed into our machine learning model. This model then performs iterations to understand how to weigh the available signals. It also considers what we refer to as “key productivity moments,” times when we should be very careful to not show the reminder. For example, if a user is in the middle of typing furiously in their document, we don’t want to interrupt their train of thought. Instead, we wait for them to be in an idle condition before showing the callout.
We feel strongly that this use of machine learning contributed to the very successful rollout of smart saver reminders in Word in 2020. We got a ton of super-positive feedback, including one user who said, “I love this. There are so many times that I leave a document open for many hours, and I don’t save it. And then I live to really regret it.” That’s exactly what we wanted to mitigate, and we felt very proud that we were able to address this customer need.
How does the Office Insider program help you in your work?
Insiders, and more specifically their feedback, are key to helping us reach out to a broader audience of “real customers.” They’ve helped us better understand why people use some of the features that we’ve developed over the years, and what they’re looking for in the products moving forward. This feedback then informs our product design and development process. [For more details about how Office PMs use your feedback, see our Behind the Scenes interview with Excel’s Steve Kraynak.]
The program is also a great way to get the word out about new features. In the case of smart save reminders, we rolled the feature out to Insiders and published a post on the Office Insiders blog. More than 15 news outlets picked up the story and wrote pieces about the feature, which in turn helped us get more feature awareness and usage, as well as better telemetry.
Smart save reminders is categorized as a cross-app feature, but thus far it’s only been released in Word. Why?
Well, it’s complicated.
Sometimes shared teams can go in and add a feature to the three applications [Word, Excel, and PowerPoint] without much help. In this case, a number of signals and other systems needed to be present in order for the feature to work as expected. Word was the perfect candidate, because it already had all the different parts that we needed in place. We partnered with them, but we didn’t actually need any engineering work from them, which was great.
We’re currently working with the Excel and PowerPoint teams to make sure all the pieces are in place, and we’re looking to bring smart save reminders to those apps in the future.
What superhero character do you connect with most in real life?
The best way for me to answer that question would be: I would really enjoy going to a party and having a drink with Tony Stark. That’s all I’m going to say.
Be sure to sign up for the Office Insider newsletter and get the latest information about Insider features in your inbox once a month!