It took us almost 10 months to put our product in the hands of real users, then we discovered that we were building the wrong product and decided to make a dramatic shift.
When the first stable version of the product was ready I invited 40 subscribers on our waiting list to try it out. The users I invited were highly engaged, I had email correspondence with every one of them, they were very interested in trying the product and the perfect examples of our target audience.
Although I handpicked the best candidates for beta testing, only three (yes 3!) users actually completed the integration process and started sending data to Kilometer.io and using it on a daily basis, less than 10%.
That was a big warning sign, those three users who completed the integration were very active and used the product regularly, they clearly got a significant value out of it, but the real problem was that I didn’t know what happened to the other 37, and how to fix it.
I assumed there were two possible problems:
- It’s too hard to complete the integration.
In order to integrate Kilometer.io a developer needs to embed our event code snippets.
This often requires from our users to create a developmental task, prioritize it, implement it and test it. Which means lots of work in order to integrate a new tool that doesn’t yet have any reputation.
- The motivation to integrate Kilometer.io isn’t high enough.
Since our potential users never saw the tool, they had a hard time understanding the exact value the tool will provide, and to see the value from it lots of work was required.
Here is an typical reply I got when asking one of our subscribers why he didn’t complete the integration.
I think startups should make big dramatic decisions only if they have a visible impact, rather than doing small optimizations (but that’s probably for another post).
Following this principle, I decided that starting immediately no one could sign Up to Kilometer.io unless they first had a product demo with me on Skype.
My goal was to show the value of the tool to potential users in order to increase their motivation to integrate the tool.
I’m still continuing with the product demos and am currently sending 100 emails to beta waiting list subscribers every day inviting them for a demo, I use YouCanBook.me for demo booking which is great for corrdinating between diffrent time zones.
I had about five demos per day and after a week I already found out what was the problem.
Product demos are extremely helpful, you can learn X1000 more from a user you talk to compared to a user who just signed up without any direct communication. Starting with the demos was one of my best decisions I took in Kilometer.io.
During the demos I demonstrated the product and explained what it takes to integrate with it.
The ability to see face expressions, voice fluctuations and even breaths taken, is extremely valuable and really helped me come to a conclusion that our plug & play concept might be very difficult to achieve.
Our initial goal was to focus on SaaS companies, instruct them how to transmit 5 basic events through the Kilometer.io API (website visit, user signup, app visit, user cancel and user billed) so that we can automatically calculate 25 charts and metrics we believe every SaaS company should have.
During the product demos I found out that different companies have different use cases. Some have a cancellation option, while others don’t. Some have only a trial and paid subscription plans, others have a freemium model and so on.
What we were trying to do is fit our users into some kind of template of metrics and events, and fitting them into this template required plenty of work including embedding 5 events, importing historic data, adjustments, testing etc.
It just takes too much time until users start getting value from our product.
Also, I found out that the most companies don’t even want our template, they want to visualize data and events they already have and most of the time they already have all their events set up via Segment.com. Segment is a data hub that allows with a single integration to send them data, from there on you can tell Segment to which tools the data should be sent on fourth.
This means that companies want to create their own dashboards, visualize data that they’re already sending elsewhere to different tools. The problem is that they just don’t have an easy way to create “big-picture” dashboards based on this data.
I decided that we will give our users what they want, instead of forcing them to use something that we think they want. We are moving away from the plug & play concept into the do it yourself concept.
This will both simplify our integration and dramatically increase our potential market size, even coffee machines can now send events to Kilometer.io and create their own dashboards!
We actually just started sending events every time we drink coffee or tea and created a simple dashboard for tracking our team’s drinking habits. A post about the fun process of setting this up is yet to come:
Fortunately, at the time I made the decision to focus our product on event based dashboards, our dev team just finished working on a new feature called custom widgets.
This feature originally was about letting our users add their own metrics on top of the 25 predefined metrics we already provide. But following our new concept we are no longer providing any pre-defined widgets and the custom widgets function will be used to fill the dashboards with any chart or metric a user decides on.
During the last weekend I thought about changing the focus of the product.
On Sunday we discussed with the development team the changes we needed to make to make this happen. During the week I had several product demos introducing the new concept, and on Thursday before the week was over I already saw our users working with custom widgets, some of them spending hours analysing their business with Kilometer.io (!!).
Here is a little sneak peak of my weekly recap of these changes, part of a series of videos I’ve been doing to keep our company’s journey documented:
“Moving fast and breaking things” is exactly the culture I want to embrace in Kilometer.io and optimise for.
The biggest advantage of the event-based dashboards approach is that whoever is already using Segment.com (which is thousands of companies) can just plug into Kilometer.io and start building their dashboards in seconds (Yes, that’s correct!).
What we could have done better?
As I mentioned at the beginning of the post, it’s been almost 10 months from when we started working on the product, until we gave it to someone to use. 10 months is a really long time in startup terms, and I am constantly thinking how we could have done this quicker.
Could we have found out that the plug & play concept is not going to work, before we developed 25 pre-defined charts and metrics?
Could we have gotten to the same conclusion just with two dashboards instead of three?
Could we have reached the same conclusion by spending less time on the design?
Sadly the answer to all of the above questions is yes.
We could have reached this turning point earlier.
Fortunately, the changes we decided on don’t require a complete re-design of our product, just some adjustments. Still I estimate that we did lose about 2 months.
I’m sharing this article in a hopes that it will inspire other entrepreneurs to think about the product they’re building and how they can put it in the hands of real users earlier.
That was our story of how we built the wrong product. If you have your own it’ll be great if you could share it here so we can all learn from each other.