New Experience for Sending Feedback

00:10 - Find out more joining us with the ever so [inaudible] , Mads Kristensen on this episode of Visual Studio Toolbox.

00:18 - [MUSIC] Hey, everyone. Welcome to another episode of Visual Studio Toolbox.

00:27 - I’m your host, Leslie Richardson. Today, I’m joined by Senior Program Manager who also happens to be on the customer outreach or customer feedback team and is also an extensions wizard.

00:39 - This is Mads Kristensen. Welcome, Mads. >> Hey, Leslie. Thanks for having me on.

00:43 - >> Thank you for coming. I love the pipe, by the way.

00:47 - Very Sherlock Holmes. >> Thank you. That was exactly the logo I was going for.

00:53 - >> Awesome. Channeling it. Good job. Customer outreach and just customer feedback, if you’ve been a dev in Visual Studio for a while, chances are you have used VS and I really wish VS had x feature or why doesn’t this feature do this instead of what it currently does? I really wish I could tell the team about it and that they would get to work, fix it immediately.

01:22 - That’s what we have the dev community for. Can you just talk about that? >> Absolutely. That’s exactly right.

01:29 - As you were saying, it would be cool if Visual Studio had this feature or that feature or things were a little bit different, more optimized for my personal workflow, well, then we would love a feature request from you.

01:43 - The other side of that is, hey, something doesn’t work as I thought it was going to work or that you told me that it was going to work.

01:49 - We also would like you to send us a bug report.

01:52 - Those are the two types of feedback that we are very closely monitoring coming in from Visual Studio, and here shortly, I’ll show you how you can do that.

02:06 - But Leslie, I don’t know if you know this, but every update to Visual Studio that we do, we actually have a bunch of fixes and feature requests that are implemented in those updates.

02:20 - I think the next update that’s coming out is 16. 9.

02:25 - Visual Studio 2019, Update 9. It’s currently in preview.

02:32 - Right now, I’m looking at the number here. Right now, it has 511 pieces of feedback, bug reports or feature requests that have been implemented.

02:45 - Five hundred and eleven and it’s still in preview, it’s not done yet.

02:49 - We average over 600 per update. It’s quite a lot of stuff actually that makes it in.

03:01 - Please feel free to send as many feature requests and bug reports as you feel that you need to.

03:08 - >> Cool. But it wasn’t mine though, was it? >> No. I don’t know. I use it too.

03:16 - If I find a bug or something like that, I open a feedback ticket.

03:19 - >> Sweet. Where are these tickets going exactly? Are they just going down a black hole and then a robot looks at them and just decides to judge before ultimately throwing it away in the end or where is it going? >> Right. I’m sure some people think that’s what’s happening.

03:36 - No. What happens is that when a feature request or a bug report is being reported, we have this website called developer community, so everything is captured there.

03:51 - When a new ticket is opened, it’s automatically being funneled to the Visual Studio Teams, Azure DevOps bulk database.

04:04 - We have a team that sits there and monitors the incoming ones and makes sure that it goes to the right team within Visual Studio.

04:11 - If someone had an issue with typing, let’s say in the editor, well, that’s going to be routed directly to the editor team in Visual Studio.

04:21 - If it’s something about C Sharp, or C plus plus, or whatever it goes to those teams respectively.

04:28 - That happens every single ticket that is opened, it gets straight into the team that is responsible for fixing it or implementing it.

04:41 - Every week, they do a triage,. When we triage, that means we look at the incoming bugs and requests and figure out what to do with them.

04:49 - Let’s take them one at a time, like feature requests.

04:53 - Let’s say a feature request comes in and we figure out it’s the editor in Visual Studio.

05:00 - The editor team then looks at it and says, “Well, that’s a good idea. ” We’re going to put it what’s called under review.

05:10 - Under review means that it’s under review both from the community to vote for it, do other people agree that this is a great feature? Add their comments maybe.

05:20 - Sometimes, comments can actually help clarify exactly what it is that the feature request should do.

05:28 - Sometimes there’s some iteration that happens by the community in those comments to narrow in and make it an even better feature request.

05:39 - It matures a little bit out there on developer community.

05:44 - At some point, it might fit with sprint planning or it might fit with other work that we’re doing in those areas.

05:54 - Then the editor team will then take it and say, “Let’s plan for this particular one to include that in the next update. ” That is pretty much how it works for any feature request that comes in that’s actionable and valid.

06:11 - Now, there are also cases where we can’t really act on them and there’s a lot of confusion, I think about it, about this, which is if we get something in that we don’t fully understand, it might not be clear or it could be ambivalent between various different things.

06:31 - We will ask for clarification. We send it back.

06:36 - We sent the feature requests back because we need more information.

06:41 - Now, the original user that posted that feature request has to reply back and I think they’ve got basically three weeks.

06:49 - If we don’t hear back after three weeks, we close it because it just becomes noise and orphaned a little bit in the system if we can’t really do anything about it.

06:59 - There’s a couple of things like that. Another one would be, we might close something that we find is out of scope.

07:05 - It could be that a feature request is like, “Hey, can we have Microsoft Word inside Visual Studio?” Because if in my solution, I have a Word document, I want to edit that directly inside of Visual Studio.

07:19 - We may want to say, “Hey, there’s actually nothing wrong with this suggestion, but we feel like that’s out of scope of what Visual Studio is supposed to do. ” Then we’ll close it down.

07:29 - That’s how it works for feature requests and bug reports is a very similar thing.

07:37 - It’s a little bit different because we look at the severity of the bug, like, “Is this something that could impact a lot of people? Is it urgent? Do we really need to do something now? Are we jeopardizing some very important scenarios that a lot of people have?” Then we prioritize them very highly or maybe they get a lower priority.

07:55 - The ones that get a lower priority or normal priority, they follow that pattern that I just described.

08:01 - They gather feedback, they get under review, see if other community members are going to vote for them, and comment on them, and provide more information.

08:10 - The cool thing about bug reports is that actually allows you to upload trace files.

08:16 - With Visual Studio that happens automatically for you.

08:20 - You can simply just say yes, collect trace files and attach to this bug report.

08:25 - Then that happens in the background. That’s really cool.

08:28 - Also, if other people wants to comment on an existing bug report, they too can say, “Hey, I also want to share my trace logs from Visual Studio that might help the team figure out exactly what’s going on. ” Instead of just having one dataset, you can have multiple and that up the chances.

08:47 - It’s very rich system for collecting feedback, organizing it, and then acting on it.

08:55 - It’s getting better and better and we are rolling out the new one as we speak.

08:59 - The title of the show is like we have a new system now.

09:03 - >> What is that new system, speaking of which? >> Right. Let’s just take a look at my screen here.

09:11 - First of all, it turns out that not everybody that uses Visual Studio knows how to request a feature or fire up a bug report.

09:21 - Let me just show that. See if I can hit the right button.

09:26 - Let me Zoom in. That doesn’t work. I’m just going to go up in the help menu in Visual Studio.

09:33 - There’s something called Send Feedback. Right here, you can say report a problem or suggest a feature.

09:38 - >> See, I always forget that exists. I usually just go directly to the DevCom website.

09:43 - > Yeah, you can do that too. This opens just your default browser, whatever that might be.

09:51 - But what’s really interesting here is that it actually connects you back to Visual Studio.

09:56 - I didn’t have to sign in because I was signed in Visual Studio and so there’s actually a connection between the two.

10:04 - >> Keep it convenient. >> Keeping it convenient. Right here, you can come in and submit your bug report.

10:13 - Let me just go “Home,” to the homepage here.

10:16 - This is, again, a new homepage. We can see exactly what was fixed in what version.

10:25 - We can see all the feedback that I have opened myself or that I’ve interacted with, or voted for, or commented on, that I’m following as we call it.

10:36 - You can see all that. You can see I reasonably had some things that I was following.

10:41 - Three fixes here that went in, so that’s nice.

10:45 - I work at the feedback team. I got probably 1,000 of these.

10:52 - The average user doesn’t have 1,000. >> [inaudible] on average, 1,000 people have tickets.

10:59 - >> Actually, the average user has zero point something, right? But we really want you to come in here and interact more with the system.

11:11 - Because the more people that come in and look at what’s there and vote and comment, the better the feature request become, the Visual Studio team, the more signal we get to understand what is a higher priority.

11:24 - That’s actually really, really important. One of the goals with the new website here is to drive what we call engagement.

11:33 - If we scroll down at the bottom here, we can see I’m signed in, so I get to see all my stuff here.

11:39 - But if I wasn’t signed in, I would just see this.

11:42 - There’s a couple of things that we would like you to be able to find a solution to a problem you might have, because someone else might have actually reported it, right? >> Right.

11:51 - >> We also really want to hear from you, like what new features would you like in Visual Studio? Then at the end here, explore existing feature requests.

12:00 - If we go up here, let’s start with that one because that’s one I think is really interesting.

12:06 - In the menu, I can go up here. It’s actually easy.

12:10 - There’s just three things that we invite you to do that’s the most important, right? Explore feedback, or report a problem, or request a feature.

12:19 - If I go to Explore Feedback and I click Visual Studio, notice here there’s other things that we collect, not just from Visual Studio.

12:28 - For the Visual Studio toolbox, let’s concentrate on the Visual Studio aspect of it.

12:33 - In here, I can search, I can filter, I can find.

12:38 - You got quite a bit of feedback, yeah? A hundred and three thousand.

12:43 - >> Yeah. >> But if I just look at Feature requests here, I got almost 11,000.

12:48 - I can search through it, I can filter, I can sort it, and so on.

12:52 - >> Like a tag. If I wanted to find all issues pertaining to the editor, can I just search for those? >> That’s coming. We don’t have it yet.

13:04 - The idea is that you would be able to come in here at some point and you can say, “Only show me things that has to do with web development. ” Or maybe even more granular thing, just JavaScript or just C-sharp, or just subset of Visual Studio because you don’t necessarily care about everything, right? If you’re a. NET developer, you might not care at all about any problems in the C++ area of Visual Studio because you never go there.

13:31 - You never installed those components. It’s super important that you’re able to be able to filter to your interests, and so that’s coming.

13:38 - We don’t have it yet. >> Got you. >> Here’s one. Just fold them out and you can see, okay, it’s green.

13:48 - It was fixed in version 16. 4. How many votes it’s got.

13:54 - >> So I can search by which ones have been completed and which ones haven’t? >> Yeah.

13:59 - >> You can do that? Cool. >> Yeah. You can see which ones are active, which ones were closed. Yeah, absolutely.

14:05 - This was a big ask from before where people had a hard time browsing and exploring and figure out.

14:10 - So now we make that much easier, although people couldn’t figure out, well, we then fix something.

14:17 - What version of Visual Studio was it fixed in.

14:20 - This is all very basic stuff when you think about it, right? But it’s something like that, having that very visible up top, it makes total sense.

14:28 - >> The little things. >> The reason we’re doing a new site, some people say, “Well, why don’t you just use GitHub issue tracker for that, like create a new GitHub repo and just point people there for instance?” >> Or bring back user voice.

14:40 - >> Yes, all sorts of things, right? Those are the reason we did a new version of the site is the old developer community site that we had was basically something that, it was a third-party product that we just bought and themed to us.

14:56 - We’ve had some issues. It wasn’t as flexible as we wanted to.

15:02 - We had some issues getting them to agree about what we thought was a higher priority feature request from our point of view or bug report.

15:11 - They didn’t fix it or they didn’t agree that they should do that.

15:15 - So there were just features we could never really do.

15:18 - What we’re doing now is that we’re replacing the entire front end, but keeping the same back end as before.

15:24 - We do that while we’re working on the back end.

15:28 - We just launched the new front end. We’re working on the back end and at some point, hopefully in a few months, we’ll have a brand new back end that will support all these things like we talked about.

15:38 - Like being able to filter by the area. The only shown things that are relevant to my interests.

15:45 - >> That’s great. >> A bunch of other things, right? That’s something we really wanted to do.

15:54 - Hopefully, just by having this “Explore feedback” button out here, which we didn’t have before, we hopefully attract some of you guys viewing this right now, to hopefully come, check out what was there.

16:07 - See the feature requests that exist, vote on some of them that you like, and give your comments.

16:12 - It’s actually super important unless that you know this, you’re program manager on one of the Visual Studio teams, right? >> Yeah.

16:19 - >> Unless we have clear amount of votes and comments, it’s hard for us to then know what priority to give between, let’s say, you have 10 different feature requests.

16:30 - >> Exactly. >> Well, which one is the most important, right? >> Yeah. I’ll go through 20 and they all have one vote.

16:36 - I don’t know which one to prioritize, which brings me to addressing the elephant in the room.

16:44 - I’m sure there’s tons of people out there who are very familiar with the deaf community and have posted countless times, numerous feedback tickets or suggestion tickets, that sort of thing.

16:55 - They never, ever get completed or they just get closed and you leave a bunch of people unhappy when you’re on the PM side and you’re having to close it even though it’s a good idea.

17:05 - Why are there so many tickets that don’t get fixed or added? >> That’s a really good question.

17:14 - There’s a couple of reasons. Let’s take bug reports first because that’s the one thing where it’s like okay, feature request is one thing, but if the product doesn’t work as advertised, then that’s problematic, hence the name.

17:30 - We get a lot of bug reports. We get hundreds a week.

17:37 - >> Yeah, 500 got completed in 16. 8. How many didn’t get completed.

17:44 - >> Yeah. Right. When we get as many as that, we have over a 100,000 in our database.

17:54 - Then we have to prioritize, there’s no way around it.

17:58 - You know this from your own team, your own company, if you’re an engineer out there, you can’t fix everything.

18:05 - If you are fixing everything, that means you cannot do anything else.

18:10 - You cannot build new features, you can’t stabilize other important things or whatever.

18:15 - You can only fix bugs and that’s the situation we’re probably in.

18:19 - So we prioritize. There’s a couple ways that we prioritize.

18:25 - We look at the impact that we can conclude the bug report that it has.

18:35 - Like how many users are affected by this? How broadly is this bug? How many people are affected? Then also is this something that’s blocking? Is there no workaround? Of course we take that into account when we prioritize.

18:56 - Then we also look at something like votes and comments.

18:58 - Like if we have a lot of people saying, “No, I have this problem,” then that is another signal for us to figure out, “Well, but maybe it has more impact than we thought. ” We see these as signals.

19:12 - There is not one of these things that determines it.

19:16 - It’s a combination of probably a handful or two, like a dozen different things and the same is true for feature requests, though they are little bit different in nature.

19:31 - Then there’s the ones that they don’t go through the eye of the needle.

19:38 - Let’s say they have no votes, no comments, really.

19:44 - We understand them to have a like a low impact.

19:47 - It might be that they have a high impact for an individual and we very much empathize with that.

19:52 - But we have to be a little bit hard when we prioritize because if we get hundreds, we can’t do them all as I mentioned.

19:59 - >> Right. >> Those are left in what’s called under consideration.

20:05 - That’s where we let them mature on the developer community website.

20:13 - Leslie, we just came from a meeting about extensibility.

20:17 - >> [inaudible] >> It’s a similar concept.

20:23 - But in this case, we look and the mature on the developer community website for votes, comments, and those sort of things.

20:29 - They can stay there for quite awhile. It feels like, “Well, they’re never fixing my problems and that’s really hurtful to me as a customer. ” I’m a paying customer and it’s really problematic.

20:49 - There’s a few things we can do to make it more likely to have the team prioritize your your bug report.

20:59 - One is that you have very clear repro steps.

21:03 - Can we easily reproduce the issue? That’s actually something that takes a long time.

21:10 - That’s one of the things that takes the longest.

21:12 - Because sometimes we don’t really know how to reproduce the step and so we investigate and that can take an hour for just a single bug report.

21:23 - A super specified here: step one, I open new project, step two, I open that C-Sharp file, step three, I execute a bill and so on and so forth.

21:37 - The clear you can make that, the easier it is and the faster we can get to it.

21:43 - If you need to attach like a demo project, then that is also super-helpful.

21:48 - It’s not because we don’t want to do it, it’s just simply we have to be a little bit hard in our prioritization.

21:53 - I hope it’s understandable. This was a Studio shoot, it has millions and millions of users.

22:02 - So that’s how that works. >> Yeah, I get excited that said, whenever I do see a duplicate ticket, whenever I’m triaging personally it’s just like, “Okay, great.

22:13 - Other people think that this is an issue too, so maybe it makes it easier to prioritize. “ I find it that way.

22:19 - >> All right. >> Anytime anyone includes pictures too, I get excited because pictures [inaudible] about more.

22:26 - >> Exactly as much information you can provide that makes it easy to either understand or reproduce, that’s going to up the chances of it going somewhere significantly.

22:37 - Again, if it’s not just you that has the issue, but maybe all your colleagues on your team, get them to vote on it too.

22:47 - Maybe comment, help you clarify some of the stuff like just in the comments on the ticket.

22:53 - That should be very helpful as well. >> Awesome. Wrapping things up, what’s next for the deaf community? I mean, you already talked about the extensive filtering capabilities that are coming up in the back end.

23:09 - Is there anything else that you want to add? >> Yeah, we’re going to look at a deeper integration into Visual Studio.

23:14 - That’s going to be one. We don’t have it yet like all thought up, but the idea is instead of just having like a links from Visual Studio to the browser that lets you create tickets, we’ll have more information.

23:29 - Once something is fixed, maybe there’s a notification, or if we request more information from you like, “Hey, can you clarify this?” there’s actually a notification inside Visual Studio and you’ll be able to track along a more integrated fashion.

23:44 - Then also a way for you to more easily discover new suggestions and feature requests out there to help give us that signal that at the end of the day, benefits everyone.

23:55 - Sometimes when I talk about this, it sounds like it’s all about us, we want you to do something, we want good repro steps, we want good bug reports, we want this and that.

24:08 - That’s true, but we wanted not just for us, we want to solve as many of these tickets as we possibly can.

24:19 - If we can do something that can up the chances of us fixing your bug report, we would love to do it.

24:29 - That’s why that sounds like that sometimes, but we really go in for the win-win, absolutely.

24:35 - >> Sweet. It doesn’t give me joy personally to close good ideas.

24:40 - I get excited when we do get to put that under a roadmap on the team personally.

24:46 - We’re not trying to be heartless. >> It’s the best feeling. It’s the best feeling when you can put something on the roadmap and everyone can see it and then a week later, you can mark it as, “Hey, it’s completed. ” >> Yeah. Exactly. People can just try out this new version of DevCom right now, right? >> Yeah. If you go into Visual Studio, hop in the help, top menu, and then sent feedback, you’re going to end up on the new version of Developer Community.

25:16 - The old one still there. We’re reiterating everything at the moment. It’s that new.

25:22 - You can still find the old one, I’m not going to tell you how, but you could create and if you might be able to.

25:28 - But not for long, and then it’ll be gone. >> I think some people are going to take that as a challenge.

25:35 - Cool. Well, thank you so much for coming on and sharing why triaging and linking those tickets and feedback, suggestions are so important Mads.

25:45 - >> Hey, my pleasure. >> Sweet. Until next time, happy coding and happy ticketing.

25:51 - [MUSIC].