Before you get the impression that I’m against custom visuals, let me say this: I love custom visuals! I myself have used many custom visuals in the past and have been very quick to look for a custom visual when I couldn’t get something to display or work the way I needed it to in Power BI.
Custom visuals fill an important gap where the base product is not yet where it needs to be, and what better way for Microsoft to see what people need and where they need to invest more time from a visualization standpoint? It’s an awesome concept and I like it.
Unfortunately there are a few BUT’s to follow, but let me first tell you my story…
My story
It’s Monday morning and a customer sends a note that some of their reports are looking odd. Some of the labels on the Power BI report pages are missing.
At face value it doesn’t seem like the end of the world, but they are in the beginning stages of the self-service analytics adoption process. Power BI reports are being evangelized across the organization with frequent demos to executives in an attempt to change the corporate culture regarding information delivery. It’s a highly visible and sensitive juncture.
Another customer sends a similar email…some customer-facing reports they use in the field on a daily basis are missing some elements. <insert expletive here/>
I take a closer look and see that a (free) custom visual which we use to show static text in combination with dynamic labels (for both customers), is no longer working and the cause of my Monday morning headache. At the same time, I log an issue in the vendor’s GitHub repo with an explanation of what’s going on and in the hope that it is something that can be fixed easily. At this point I had already decided to move away from the custom visual because I know it is free and without any guarantees, and the risk on my project(s) was simply too high to wait it out. It takes around 20 hours to find and implement a suitable replacement on about 15 Power BI report pages in total.
A few days later and after not getting a response from the vendor, I start looking at their GitHub repos in a bit more detail and don’t like what I see…many open issues and some have been there for many months without resolution or explanation regarding the way forward. Very few releases for the free custom visual we were using, generally pointing in the direction of a long wait before the visual will be fixed.
Moral of the story
There’s a few lessons to be learnt from my experience, and the first (and most important) one is that this was my fault. I should have done a bit more research and considered the potential risks before jumping in an using it. I think many of us have become somewhat complacent because of the success of the custom visual ecosystem…this is personally the first issue I’ve had with custom visuals since the inception of the program.
Should the vendor be blamed here? Not necessarily. Vendors use the free visuals as a feeder program to their paid ones, and using free software has its implied risks.
Should we use the free custom visuals for Power BI at all? I don’t know…it’s a tricky situation with more questions than answers. On the one hand it is great to have a program where vendors and community members can contribute to the functionality of the product. On the other hand it can only be as good as the program’s ability to respond to these kinds of issues, or prevent them all together. There has been some debate over the certification process of Power BI custom visuals, and as you can see from this blog post and Marco Russo’s comments below, there are some flaws with the current system and process.
Where to next?
To be completely honest, I don’t know. This situation prompted me to think about how we can have an ecosystem where custom visuals (even the free ones) can be trusted and used without fear of it breaking, especially when you consider the frequency of updates in the Power BI service which I am sure requires a lot of time end effort from the creators of these custom visuals.
I don’t have any answers at this point, but some transparency about the process would go a long way for me. How long in advance do the creators have access to upcoming service releases in order to test their custom visuals? How long does it take to review and release a new version of a custom visual in AppSource? Is there an agreement on the turnaround time to fix issues? What recourse do we have if bugs are not fixed, apart from giving a one-star rating? I simply don’t know, and that makes me a little nervous.
Don’t know about you, but I am going to have to think twice (unfortunately) before using any custom visual in the future.
I think some kind of rating system and tangible metric to gain said rating for the custom visual should help to give a level of confidence. Does custom visual author actively fix bugs quickly, how many bugs are there, etc? Otherwise, as you suggest, what is the point of adopting something that is going to break. I’ve personally run into a similar issue with custom components in Azure DevOps so its a much wider issue for MS to address imho.
Yeah, agreed. A more comprehensive rating system will definitely help, but only if Microsoft themselves can speed up the certification process on their end.
I finally got a response from the vendor after 3 weeks, but only to tell me that we either have to wait for the fix to be certified (i.e. tested) by Microsoft, or they could give me access to a private build. Not worth waiting 3 weeks for a response, or an additional few months for the certification process to complete.