Using the classic customer development process we worked with educators to identify the problem and define a minimum viable product that could solve it. We came up with some initial hypotheses about what the problem might be and tested these quickly and iteratively with teachers. All of them were wrong. Lean is all about proving how wrong you are at speed so that you can get on with finding out what's right.
As it turned out, the single biggest problem teachers were facing was finding other teachers and classes to Skype with. Last December, we released a product in beta: an online directory for teachers who use Skype to find each other and share useful resources. Teachers could filter the directory by location and interest to find like-minded people to partner with. And that's where the lean learning adventure really began.
Working fast to get a product out in the wild and in the hands of real users is incredibly exciting but conversely once you've done that the first thing you have to do is wait. How will people actually use the service? Will your hypotheses be proved correct? What surprises might lie in store? We deliberately kept instruction minimal on Skype in the classroom as we wanted to see how users would hack it for their own means.
First port of call for any lean operation is to check the metrics. Our metrics told us that members were using the site to connect with each other: there was a ratio of 1:2 Skype contact requests to members. But they weren't telling us whether these connections were successful and resulted in Skype projects being carried out. At the same time, by watching how people were using the site we began to pick up on an interesting set of use cases.
Members were using their profile biography and the resources section of the site to post up what kind of Skype partner they were looking for:
Users wanted to use the site to connect with people around specific projects but there was no way for other people to surface these requests. So successful connections around a particular project were down to chance. Our hypothesis was that the core of the service was simply about connecting people. But now we could see that perhaps it was about something else: connecting people around projects.
The natural next step was to speak to members. We'd been running various channels of communication via feedback forms, email, Twitter and Facebook but we needed a more targeted approach. The black hole of a lengthy online survey beckoned, and then we came across Kiss Insights.
A service produced by another group of lean afficiandos, Kiss Insights is designed to help product managers carry out fast, effective customer research. It lets you set up super light-weight mini surveys which pop up in the bottom right hand corner of your site and ask your users one or two questions. And it forces you to focus on only asking the questions which will have the greatest impact on what you do next. Within 2 days of asking our users whether they wanted help connecting around specific projects, we had 500 responses, 1/4 of our beta community. We knew it mattered.
We had to make our idea around projects real so that we could test it with users. A favourite Made by Many approach is to sketch up use cases we can use as conversation starters in interviews with users. Keeping sketches basic is key to success. Even though sketches are by implication unfinished work, a splash of colour or lick of polish can be very distracting.
People are reluctant to criticise work which they think you have worked hard to create and their eye is easily drawn to areas of colour on a sketch. In the worst case scenario you'll end up building features that people responded well to because they liked the look of them not because they fulfill a real need. "I like this" is not a helpful insight. The aim is to use the visual depiction of an idea to get people talking so that you can get to the subtext of what matters to them. Having these conversations helped us figure out what the key needs were for a teacher trying to find a partner for a specific project.
Ok time for a small confession, this post is a bit of a cheat. I've presented the story of how Skype in the classroom has evolved in a nice, neat linear fashion. It wasn't like that at all. While tracking metrics and watching how people were using the site we were already sketching up new ideas and use cases. The nugget of projects was nested in some of these early ideas but it seemed like some other features were more important at that stage. It was only after we started another development cycle that we began to notice a pattern of project-like use cases appearing on the site.
And so, half way through the first iteration of development we set up the Kiss Insights survey on projects. The phenomenal response to that made us re-prioritise the next iteration so that we could build a projects section. In just a few weeks we went through several rapid, messy cycles of make, test, learn on our strategy for how the service should evolve. If you want to play buzzword bingo you could say we executed a classic pivot.
For me it's something more fundamental. At its heart lean is all about being comfortable with uncertainty and change. Seriously if it doesn't hurt then you're doing it wrong. If you're ok with that process, then you've got a chance of making a service that people want.