To launch the best developer products, your product team needs to work closely with your DevRel/DevEx team and your platform users, says Shopify’s developer experience lead Shayne Parmelee. In his talk, from DevXCon San Francisco 2018, Shayne explores the ways in which links between technical design, internal testing and user feedback make a successful go-to-market experience.
So today I’m gonna be talking about API go-to-market. My name is Shayne Parmelee. I’m developer experience lead at Shopify. So I’ll start with giving a little bit of back story on Shopify and what our company is because many times when I talk to crowds, people have no idea what Shopify is. So we’ll start with that and then we’ll go into talking about some API go-to-market after that. Cool, so the first part is what is Shopify? So Shopify is an e-commerce platform that allows merchants to be able to sell anything to anyone in the world and ship anywhere.
So we’re based out of Ottawa, Canada, and we have all different kinds of merchants on the platform. So this is one type of merchant who’s on Shopify, Northbound Notebooks. They make beautiful full-grain buffalo hide leather notebooks, all brass hardware. They have really thick like vellum-like paper on the inside. They’re a small merchant, but they make really great products. My biggest problem with these notebooks is that like I never feel like I have anything important enough to write down in them because it’s so nice. So I end up using like a 50 cent Hilroy notebook.
Another shop that we have on the platform is Fogo Island Shop. So Fogo Island, I don’t know if anyone’s heard of that, but it’s a small rocky island on the east coast of Newfoundland. And this island they’ve been doing…their primary industry has been fishing for the last 400 years. But with the fishing crisis on the east coast, this is an industry that has more or less dried up for many people. So they pivoted. And a lot of these people using the boat making and net weaving skills have been passed down for generations decided instead to start making furniture. So this now is furniture that they’re shipping all over the world and it’s really revitalized the community because they’ve been able to sell products like this, they’re beautiful.
We also have merchants like Kylie Cosmetics. She’s really famous and sells lots of makeup. Also she is the sister-in-law of Kanye West who’s also on Shopify. So he sells super cool shoes and also $240 sweatshorts. I don’t know if anyone’s wearing sweatshorts right now. They look pretty comfortable. I’m thinking of maybe getting into a more entry-level sweatshort to start. I don’t think I would start with those ones, but there is that. There’s all kinds of different merchants on Shopify.
So, looking a little bit at the scale of what we’re dealing with here, so we have more than 600,000 businesses on the platform right now. We’ve done more than 55 billion in GMV. This past Black Friday in 2017, we did a billion in sales of merchants to one another. These merchants aren’t just selling to people who are in their own backyards. They’re selling to people all over the world. So, Shopify merchants ship packages 12.6 billion miles on Black Friday and Cyber Monday. So, for scale, that is way further than Pluto at the edge of the solar system, almost all the way to Voyager 1. On here it says 13.1 billion miles, but it’s probably way further than that. Now, this is a pretty old slide.
Yeah, so our goal really is that Shopify makes commerce better for everyone. Now, there are some pieces that are really hard to this, especially when you’re a product company. So, number one is that commerce can be really complicated. When you’re starting a business and you’ve never done it before, not only do you have to worry about all the things that our regular business owners have to worry about, but you also have to worry about online marketing and actually creating your online store. Maybe you’ve never made a website before. So we’re trying to make it as easy as possible for as many people as we can.
However, merchants’ needs are also really highly diverse. Selling a product in Kansas City is way different than selling a product in Mumbai. People expect different ways to pay. People expect different ways to ship or pick up their products and different ways that they’re gonna be supported after they buy this thing. So our answer to this like many other people in this room ends up being platform. So if we think about it this way. If a merchant is running a marathon, 25 miles of that is gonna end up being built by Shopify. But there’s always gonna be at least one highly divergent mile in this race that people are gonna not get exactly what it is that they need from the core product and so enter platform.
So if we look at the Shopify API. Right now we have about 80 endpoints that are accessible via the Shopify API. We serve about 45 million API calls an hour. So there’s a lot of volume right now that’s coming into platform, and that’s allowing people to fulfill these diverse use cases. But we’re also a very nimble company. So this year at Unite, which is our annual developer conference that we do for our partners that are in our community, we launched more than a dozen new APIs and features and also a graph QL API in tandem with our existing Rest API. So we’re releasing a lot of things. So there are some challenges to providing a platform for this type of product.
So number one is that we currently use continuous versioning. So we do not have a versioned API. Everything that we launch, we’re launching it with new APIs. And when we’re changing APIs, we, of course, offer deprecation periods, but we don’t have a lot of legacy clients that are not what’s currently in the admin. We have a small team. So the team that actually brings these APIs to market is actually only three people that is managing the launch cycle and deprecations of these APIs. And we have a constantly evolving product. Shopify does up to 50 deploys a day of new code to production, and each of these deploys could have a number of pull requests for new features that are actually being added to the product.
So I’m gonna talk a little bit about how this year we’ve managed to have essentially no negative merchant impact with any of the new launches that we’ve done and the APIs that we’ve changed. These are things that have worked for us. If anyone sees any of these slides and it’s like, “Well, that’s really stupid,” like please come and talk to me after because I’ve never been to DevXCon before and I can’t believe the amount of knowledge that there is like in each of these rooms about these topics. So I would love to hear everyone’s opinions about the things that I’m about to share.
So, number one, whenever possible, developer functionality shouldn’t be replaced by product without being augmented by more API features. So a good example of this is customers, customer data on Shopify. So we do have a customer API. When I first started at Shopify a number of years ago, there was no ability to delete customers in Shopify. Don’t worry. You can definitely delete customers in Shopify now, especially any Europeans in the room, you can totally delete all your data from Shopify anytime you want. So there was no ability to delete customer data. So what we had was, we had some apps that would redact customer data by overwriting it with other things.
So when we released the ability to delete customers from the Shopify admin, we also released the ability to do that through the API. What this meant is that people apps that were already built in the ecosystem in order to fulfill the merchant need of getting rid of customer information, could then build on top of this new API and actually improve the quality of their own product. It also allowed them to move upstream into increasingly diverse and niche merchant needs, which is better for everyone. I mean, we already have these developers in the community that are building solves for use cases that people have. It’s not ever gonna make sense. I mean not ‘not ever’. But usually, it’s not gonna make sense for us to steamroll those features without giving them the ability to provide more value to more people by offering APIs on top of those products.
So the next piece is involving go-to-market teams during tech designs and integrating deeply with R&D. For a lot of people in the room, that go-to-market piece is just gonna be developer advocacy. This image used to be way better, but the PR team made me block out all the open issues on an architect designs repos, so it’s not as good as it was before. But we have a specific repo that’s used by R&D to show all the new tech designs for new features that are gonna be launched on Shopify. So what that means is that right from the very beginning of product development, at the very start when it’s just being thought of, we can engage with these teams to make sure that developers are being thought of. And the R&D teams like it too because it means that they’re not about to ship a feature and then platform comes in and it’s like, “Okay. Well, there’s a bunch of developers that are gonna be in serious trouble if you launch this, like you have to delay it or whatever.” So it ends up being a really good way to relay feedback.
So the next thing that I wanna talk about is a situation that happened this year at Unite. So we announced the ability to be able to edit orders on Shopify. Before this, we considered orders to be like in an immutable like source of truth, but we do realize that merchants at times will need to edit an order for exchanges and things like that. So we announced at Unite the ability for merchants to be able to edit orders. Now, this was slightly troubling for some of the apps in the ecosystem. We can see that this app is called Edit Order by Cleverific. This app all it does is allows you to edit orders. So we could see how us moving into their space could potentially be very worrying for them considering they’ve built their entire business on Shopify. Cleverific only builds apps for Shopify, and this is their biggest app. So this could be a scary thing for them, which moves me into my next point.
Developer betas are a powerful tool for building trust and getting great feedback. So when we decide that we’re gonna launch this new feature, there’s nobody that knows more about a merchant’s need to edit orders than the Edit Order app. Bringing them into the development process of this new endpoint super early is an amazing way for us to get feedback about what people are already doing and what they already need to use and a great way to build trust with these developers because we’re giving them access to these new APIs. So they’re gonna be able to improve their product to stay ahead of the tidal wave of features that are always coming from the platform.
So after bringing Cleverific, which is the app company, they’re based in California, into this beta, they were actually really excited for all the new things that we’re adding to the ability to the order’s API because it means they’re gonna be able to add a whole swath of new features to their app and serve a lot more merchants. So I can’t exactly spoil what they’re gonna do, but they’re gonna be able to do a lot more than just exchange products on orders in the future.
So, last thing I’m gonna talk about is how to monitor if an API is even good once you launch it. So this is what a lot of people are using right now, so dashboards. This is a Splunk dashboard again with a bunch of blurred out information, unfortunately. What we’re measuring here is the amount of errors compared to okay responses that we’re getting from web hooks or from API calls. So, this is a good way to see if people are successfully hitting the endpoint. But the biggest problem with this is that if people aren’t successfully hitting the endpoint. It’s difficult to know exactly why they’re having that problem.
So enter the next thing that we monitor. Again, there’s so much blurred out but was a slide of all of our tags that we have for incoming issues. So anytime that we talk to a developer about anything, we’re tracking every single thing that we talk about and we’re tagging everything with the endpoint it is that they’re having trouble with. And over time, we can either present this to the product team that manages that endpoint or we can improve the documentation in order to prevent the incoming flow of support requests. So this has allowed us to…right now our support team who handles all incoming APIs support requests is four people. And we have doubled the amount of developers on the platform in the last year but actually have the number of API support requests because we’ve always been fixing forward when we have problems based on the data that we gather.
My favourite of all of the ways to gather feedback is actually this one here. So this is a little screen. How helpful was this page? It’s at the bottom of our API reference page. So often, one of the first interactions that people will have with an API when they’re looking to build a feature is by looking at the API reference for the specific endpoints that they might need to use. Similarly, when they have a problem with an endpoint, the first place that you’re usually gonna check is the API reference. So if people are having a bad time, they can fill in this little box and they can submit their feedback based on how helpful the page was. And we’ve got a gigantic amount of helpful feedback from these pages in order to improve our API products or to improve the documentation itself.
So here’s some examples of some feedback that we’ve got. So this one a developer working with customer addresses said, “I smile so I won’t cry.” We ended up working with this customer in order to solve the problem. This problem was with documentation rather than with the API. But it’s always good feedback when people are having a bad time. This one just wants simple information, “A video would have been great. Sorry, not shouting.” So it looks like maybe they didn’t have a backspace key. I don’t know. That’s their real problem. Yeah, so this was a developer who is working with product variants, and we ended up releasing a video series that handle…that ended up dealing with product variance. And it’s now one of our most successful. This isn’t the only comment where people were asking for videos, but it’s now one of our most successful kind of areas that we’re sharing information about our API.
This one is actually not useful at all. I just think that it’s really funny that someone just wanted to write “no”. You can just vote and say that it’s bad without typing anything. I don’t know why they would have done this. And then the last one…So this is, all of our documentation, our API reference is actually created from tests that we write in core. So when someone comes up with something like this saying that parameters and examples are having different parameters there, actually that’s because problems in the tests and often problems in the core platform. So using this feedback, we were able to go back and actually fix the problems with the API itself.
So, as a quick recap, whenever possible trying to augment new features with API is going to, number one, build strength in the trust that your developer ecosystem has with you and also prevent pushing people off your platform when you’re releasing new features. Developers who are already engaged in the space you’re moving into are ideal beta testers for new APIs. Again, all platforms are built on trust, and this is a great way to build trust. But it’s also an amazing way to create super tight feedback loops for new features or APIs that are gonna be released. Again like in the last talk, open API spec is a really good way to show people what an API is probably gonna look like and have something they can hit right away in order to kind of spec it out.
And last, integrating deeply with your R&D teams all the way from tech designs into feature tweaks, so through the entire life cycle of the product. The way that we started looking at this ourselves because often you might get some pushback from product teams especially when you may have differing opinions on what’s important when there’s a product being released. We started seeing ourselves almost like an internal agency trying to bring value to these product teams when it was that they had…when they were gonna release something new and making sure that their products are gonna be really successful for our developer community. And now we’ve almost become a victim of our own success and they’re knocking down our door in order to figure out what the effect on platform is gonna be.
So thanks very much. You can follow me on Twitter. I have 27 followers right now. I think I actually have 28 because a very nice man in the airport followed me. So you can follow me on Twitter. And thank you so much, everybody. I’ve absolutely loved being at this conference
Mary Thengvall tells you how to prove your team’s business value and show a direct impact on every department across a company.
What if marketing metrics don’t tell the full story of your dev rel programme? What else should you be measuring?