USB Device Not Recognized in USB 3.0

Unless you’re having this problem:

For USB 3.0 ports on an Arduino-like stack you may safely stop reading. Nothing to see here just trying to fill out the Internet.

The Problem

This problem occurred for me on an ATMEL ATMEGA32U2 chip with the LUFA USB framework for AVRs and the HID USB driver. Everything worked fine for USB 2.0, but on modern PC's it would pop up "USB device not recognized".

In device manager the device showed up as Unknown device. Device status would look like this:

Windows has stopped this device because it has reported problems. (Code 43)

And in the event viewer it showed something like this:

Device USB\VID_0000&PID_0002\5&ea89d57&0&2 had a problem starting.

Driver Name: null
Class GUID: {00000000-0000-0000-0000-000000000000}
Service: 
Lower Filters: 
Upper Filters: 
Problem: 0x2B
Status: 0x0

The Solution

Despite a lot of fruitless packet monitoring and upgrading various packages and help from master-hacker Joe Ferner, solving this ultimately boiled down to compiling the GenericHid sample project from LUFA (which did work in USB 3.0) and then commenting out more and more of my project until it worked and then incrementally adding everything back via binary search.

The problem ended up being the amount of time that passed between the call to LUFA's

USB_Init();

And the USB calls in the main program loop. So essentially I was doing something like this:

int main(void);

  MCUSR &= ~(1 << WDRF);

  wdt_disable();

  clock_prescale_set(clock_div_1);

  USB_Init();

  _delay_ms(500); // <---- DON'T DO THIS

  for (;;)

  {

    HID_Device_USBTask(&Generic_HID_Interface);

    USB_USBTask();

  }

}

Take that delay out and everything works.

Summary

The takeaway: move all of your initialization logic - anything that might take any time at all - prior to the USB_Init call. It might just save you a lot of pain down the road. That's it. Hope this helps someone, somewhere.

Google Play vs Windows Store: A Stark Contrast

It’s a familiar story: anything-goes-wild-west vs bureaucracy-ridden-walled garden. But you probably already knew that. What surprised me is how incredibly stark the contrast is. Getting an app in Android and on Windows 8 couldn’t be more different: from fees, to technical support, to the app-submission process.

Setup for the Letdown

You’d think an app store coming from behind might try to get their app count up by accepting new submissions first and asking questions later. Couldn’t be further from the truth.

My first Windows Store app took a total of 33 days from the first time I tried to submit through when it was finally accepted. My Google Play app? 4 days.

Let’s be honest, 33 days is completely unacceptable. Why so long? Partly it’s because they have standards. No really. Believe it or not they have live human beings manually test your app and provide a write-up of why it wasn’t good enough for their marketplace. It’s sort of like getting a free tester.

This approach actually makes sense for them. Classic Windows is similar to what Android is today. You can pretty much install anything from any random website. Google just centralizes their chaos. The result for classic Windows? Malware, Trojan viruses, countless unwanted browser plug-ins, and a slowly degrading computer experience over time as more crap gets installed. In short: a lack of trust in apps. If Microsoft allowed everything in they would be right back to the problem they had in the first place.

But 33 days? Really!?

In my case my first rejection actually came in just 3 days. It was a couple of minor issues. The app needed a privacy policy and apparently also a link to create a new account rather than just a log in screen (even though you could do some stuff without logging in). That second item was an excellent point I’d never even thought of. I’d incorrectly assumed most of my app users would be existing website users. There was a brief mention of what to do, but no link or in-app account creation.

My previously submitted Google Play app had these problems too, but they certainly didn’t have human beings considering how users will interact with my app. Go manual testing! Go Microsoft!

But my initial excitement was just a set up for a big letdown. I resubmitted within a day and then waited.

And waited.

And waited.

Customer Support: Microsoft++

How a company responds to problems can be more insightful than the fact that they have them. After all everyone screws up at some point, right?

The following was one of several interactions I had with Microsoft support.

Dear Andy,

It has now been 22 days since I resubmitted my app. It has been almost a month since my initial submission. It has been three days since you estimated it would be a day or two.

Is this going anywhere? Am I being quietly ignored for some reason? If there are reasons my app is being ignored could you please list them so that I could resolve the issues? Or if Microsoft simply does not want me developing apps for Windows 8 is there a facility for me to get my money back?

I am extremely frustrated by this process. If I were ever to get my app in the store and needed to get an update out quickly, might it take as long as the initial submission has been? Because a month turn-around to fix bugs is completely unacceptable: not just to me, but to every single user of the Windows Store.

I genuinely want Microsoft to succeed and will do what I can to help, but if this is how it’s treating its developers, customers, and supporters then I’m sorry to say my expectations are low.

Sincerely,

- Lee

A bit too strong? I was getting angry. To date I had been at least impressed with their professional and quick responses.

However I was floored by their response to my angry e-mail.

Hi Lee,

Thank you for using the Microsoft Windows Store.

Thank you for getting back to me. I have been checking your account on a daily basis to see if your app has been passed. I can tell you I am looking into this issue for you. I haven't ignored you at all. On the contrary I am eager to help you get this issue resolved.

I understand that it is really frustrating for you and I am doing everything I can to have this issue resolved for you. I hope to have an answer for you soon. Again I am sorry that it is taking so long to get through the process.

If you have any more questions please reply and I will be happy to help you.

Thank you
Andy
Your Partner at Microsoft
Use what you know. Do what you've always imagine

I don’t know about you, but I think sincere compassion is something I’m secretly hoping for in every customer service interaction. If I could just get a quick resolution to the problem I’d still give Microsoft a big win in my book. Sadly that wasn’t to be.

By way of comparison Google had issues with my initial Google Play submission too. It gave me a nice opportunity to compare how the companies responded to adversity.

When Google accepted my money but wouldn’t let me submit my app I got a response like this:

Hi Lee,

Thank you for your note. We apologize for the delay in your account registration. We have activated your account and you may begin uploading applications. Thank you for your interest in Google Play.

If we can assist you further, please let us know.

Regards,
The Google Play Team

Not bad. But not compassionate. No name of a person who wrote the note, no “it must be frustrating”.

But whatever. It was a relatively quick resolution from Google. Two days to respond to me for a total of 4 days from time of initial submission until final app acceptance. I’ll take an emotionless quick resolution over a drawn-out unresolved empathetic one any day.

Fees: Google FTW

As a quick aside do you know how much it costs to get an app on Android? It’s $25 … once and only once. Just enough to keep the riffraff out. And Windows 8? $50 … every .. single .. year.

Why should I have to pay another $50 fee next year just for the privilege of making a quick update to my free, no-advertisements, non-revenue generating app?

The chances are good I won’t be renewing that fee. The chances are good a lot of developers won’t. And that will lead to stale apps and a bad customer experience. Poorly done Microsoft. #win8 #fail.

Fail, Fail, and More Fail

So how does the Windows Store story end? 25 days from my initial submission I got this response:

Notes from Testers:

.D:: This app appears to be incomplete in its functionality to our reviewers. The app appears to have dependencies on non-Windows Store style APIs. In the app description as well as in the app, the users are asked to download additional software to be functional.

Rejected. Again. This time for a different reason. In their defense I should have read their submission guidelines. But couldn’t they have told me this during the first rejection? Did it really require 25 days for this response?

I was crushed. They were asking for a massive rearchitecting that didn’t make sense and maybe wasn’t even possible. Google Play was beginning to look golden.

The problem was in how my app works. It shows the realtime status of a continuous integration server, but requires a desktop app to collect the data and upload it to “the cloud”. The mobile app just gives a view into the cloud data and receives push notifications and live tile updates from a server. Seems reasonable right? Apparently the entire architecture violates Microsoft’s submission guidelines.

My plan involved added a single CI server that didn’t require a desktop app to sneak by their policies while leaving the existing process all other ci servers. Travis, since it’s guaranteed to never be behind a firewall. I had a low confidence the plan would work.

After a week of evenings followed by an all-nighter I resubmitted at 5 am EST (I’m a sick puppy, I know) and crossed my fingers that it wouldn’t be another 22 days.

3.5 hours later, at 8:30 am, I got an e-mail from Microsoft. With that kind of turn-around it couldn’t be good news.

It was. It passed. I was shocked.

They couldn’t have run the automated tests in that amount of time let alone manually tested it. And what humans do business at like 7 am EST? Did they just take pity on me? Did Andy, my compassionate customer support representative come through for me? I still don’t know.

Updates

One final comparison. Updates on the Windows Store seem to take around 8 hours. On Google Play they’re seemingly instantaneous. In both stores customers have an update to install by the next day. No complaints there.

Summary

So who has the better process? Google. Hands down. They allow for a lean, minimum viable product style of development. Publish early and often, get feedback quickly, push the features your customers really want.

But I like elements of the Windows Store process. Friendly customer support people rock. Manual testing made my app better. But were their “improvements” the kinds of things real customers would prioritize highly? Maybe, maybe not.

I guess my ideal app store would cost $25 once; incorporate a manual tester that provides helpful suggestions not requirements; reject apps rarely if ever; give empathetic responses and quick resolutions when problems do happen; oh and turn submissions around in 3.5 hours or less every time.

That’s not too much to ask for, is it?

It’s a familiar story: anything-goes-wild-west vs bureaucracy-ridden-walled garden. But you probably already knew that. What surprised me is how incredibly stark the contrast is. Getting an app in Android and on Windows 8 couldn’t be more different: from fees, to technical support, to the app-submission process.

Setup for the Letdown

You’d think an app store coming from behind might try to get their app count up by accepting new submissions first and asking questions later. Couldn’t be further from the truth.
My first Windows Store app took a total of 33 days from the first time I tried to submit through when it was finally accepted. My Google Play app? 4 days.
Let’s be honest, 33 days is completely unacceptable. Why so long? Partly it’s because they have standards. No really. Believe it or not they have live human beings manually test your app and provide a write-up of why it wasn’t good enough for their marketplace. It’s sort of like getting a free tester.
This approach actually makes sense for them. Classic Windows is similar to what Android is today. You can pretty much install anything from any random website. Google just centralizes their chaos. The result for classic Windows? Malware, Trojan viruses, countless unwanted browser plug-ins, and a slowly degrading computer experience over time as more crap gets installed. In short: a lack of trust in apps. If Microsoft allowed everything in they would be right back to the problem they had in the first place.
But 33 days? Really!?
In my case my first rejection actually came in just 3 days. It was a couple of minor issues. The app needed a privacy policy and apparently also a link to create a new account rather than just a log in screen (even though you could do some stuff without logging in). That second item was an excellent point I’d never even thought of. I’d incorrectly assumed most of my app users would be existing website users. There was a brief mention of what to do, but no link or in-app account creation.
My previously submitted Google Play app had these problems too, but they certainly didn’t have human beings considering how users will interact with my app. Go manual testing! Go Microsoft!
But my initial excitement was just a set up for a big letdown. I resubmitted within a day and then waited.
And waited.
And waited.

Customer Support: Microsoft++

How a company responds to problems can be more insightful than the fact that they have them. After all everyone screws up at some point, right?
The following was one of several interactions I had with Microsoft support.

Dear Andy,



It has now been 22 days since I resubmitted my app. It has been almost a month since my initial submission. It has been three days since you estimated it would be a day or two.




Is this going anywhere? Am I being quietly ignored for some reason? If there are reasons my app is being ignored could you please list them so that I could resolve the issues? Or if Microsoft simply does not want me developing apps for Windows 8 is there a facility for me to get my money back?




I am extremely frustrated by this process. If I were ever to get my app in the store and needed to get an update out quickly, might it take as long as the initial submission has been? Because a month turn-around to fix bugs is completely unacceptable: not just to me, but to every single user of the Windows Store.




I genuinely want Microsoft to succeed and will do what I can to help, but if this is how it’s treating its developers, customers, and supporters then I’m sorry to say my expectations are low.




Sincerely,


- Lee

A bit too strong? I was getting angry. To date I had been at least impressed with their professional and quick responses.
However I was floored by their response to my angry e-mail.

Hi Lee,




Thank you for using the Microsoft Windows Store.




Thank you for getting back to me. I have been checking your account on a daily basis to see if your app has been passed. I can tell you I am looking into this issue for you. I haven't ignored you at all. On the contrary I am eager to help you get this issue resolved.




I understand that it is really frustrating for you and I am doing everything I can to have this issue resolved for you. I hope to have an answer for you soon. Again I am sorry that it is taking so long to get through the process.




If you have any more questions please reply and I will be happy to help you.




Thank you


Andy


Your Partner at Microsoft


Use what you know. Do what you've always imagine

I don’t know about you, but I think sincere compassion is something I’m secretly hoping for in every customer service interaction. If I could just get a quick resolution to the problem I’d still give Microsoft a big win in my book. Sadly that wasn’t to be.
By way of comparison Google had issues with my initial Google Play submission too. It gave me a nice opportunity to compare how the companies responded to adversity.
When Google accepted my money but wouldn’t let me submit my app I got a response like this:

Hi Lee,


Thank you for your note. We apologize for the delay in your account registration. We have activated your account and you may begin uploading applications. Thank you for your interest in Google Play.


If we can assist you further, please let us know.


Regards,

The Google Play Team

Not bad. But not compassionate. No name of a person who wrote the note, no “it must be frustrating”.
But whatever. It was a relatively quick resolution from Google. Two days to respond to me for a total of 4 days from time of initial submission until final app acceptance. I’ll take an emotionless quick resolution over a drawn-out unresolved empathetic one any day.

Fees: Google FTW

As a quick aside do you know how much it costs to get an app on Android? It’s $25 … once and only once. Just enough to keep the riffraff out. And Windows 8? $50 … every .. single .. year.
Why should I have to pay another $50 fee next year just for the privilege of making a quick update to my free, no-advertisements, non-revenue generating app?
The chances are good I won’t be renewing that fee. The chances are good a lot of developers won’t. And that will lead to stale apps and a bad customer experience. Poorly done Microsoft. #win8 #fail.

Fail, Fail, and More Fail

So how does the Windows Store story end? 25 days from my initial submission I got this response:

Notes from Testers:


.D:: This app appears to be incomplete in its functionality to our reviewers. The app appears to have dependencies on non-Windows Store style APIs. In the app description as well as in the app, the users are asked to download additional software to be functional.

Rejected. Again. This time for a different reason. In their defense I should have read their submission guidelines. But couldn’t they have told me this during the first rejection? Did it really require 25 days for this response?
I was crushed. They were asking for a massive rearchitecting that didn’t make sense and maybe wasn’t even possible. Google Play was beginning to look golden.
The problem was in how my app works. It shows the realtime status of a continuous integration server, but requires a desktop app to collect the data and upload it to “the cloud”. The mobile app just gives a view into the cloud data and receives push notifications and live tile updates from a server. Seems reasonable right? Apparently the entire architecture violates Microsoft’s submission guidelines.
My plan involved added a single CI server that didn’t require a desktop app to sneak by their policies while leaving the existing process all other ci servers. Travis, since it’s guaranteed to never be behind a firewall. I had a low confidence the plan would work.
After a week of evenings followed by an all-nighter I resubmitted at 5 am EST (I’m a sick puppy, I know) and crossed my fingers that it wouldn’t be another 22 days.
3.5 hours later, at 8:30 am, I got an e-mail from Microsoft. With that kind of turn-around it couldn’t be good news.
It was. It passed. I was shocked.
They couldn’t have run the automated tests in that amount of time let alone manually tested it. And what humans do business at like 7 am EST? Did they just take pity on me? Did Andy, my compassionate customer support representative come through for me? I still don’t know.

Updates

One final comparison. Updates on the Windows Store seem to take around 8 hours. On Google Play they’re seemingly instantaneous. In both stores customers have an update to install by the next day. No complaints there.

Summary

So who has the better process? Google. Hands down. They allow for a lean, minimum viable product style of development. Publish early and often, get feedback quickly, push the features your customers really want.
But I like elements of the Windows Store process. Friendly customer support people rock. Manual testing made my app better. But were their “improvements” the kinds of things real customers would prioritize highly? Maybe, maybe not.
I guess my ideal app store would cost $25 once; incorporate a manual tester that provides helpful suggestions not requirements; reject apps rarely if ever; give empathetic responses and quick resolutions when problems do happen; oh and turn submissions around in 3.5 hours or less every time.
That’s not too much to ask for, is it?

How To Get The Most From Twitter: Scoping Rules

Ever noticed a tweet disappear? As in someone else can't see something you posted? There's a common mistake even twitter pros make that causes this to occur.

Are you sure you're seeing all of the replies to your tweets -- even from people you don't follow? Do you ever find yourself missing important posts -- like those from your real life friends? Do you know which will reach a wider audience: new style retweets vs old style retweets?

This article will answer these and other twitter related questions. It will help beginner to intermediate twitter users get the most out of the service.

#1 Standard #Tweets

These appear simple. These show up in the timeline of your followers (@admirer and @vet in the example above). And they don't show up in the timeline of anyone who doesn't follow you (@effusive). If this was all there was to twitter it would be a complete waste of time.

The subtlety with this type of tweet is that your content may show up to someone that doesn't follow you but that is following a hashtag or a topic you mention. In the example above @hiro thows a hashtag onto the word aardvarks to indicate that it's special somehow. His tweet is then picked up by @spca who runs a constant search in a twitter client for the word "#aardvark".

The first time I realized how powerful this could be I had was at a conference. I was extremely bored by a terrible keynote. With nothing else to do I discovered the conference hashtag on twitter and started watching it. Suddenly I realized I wasn't alone. I was in a huge room full of people making fun of the presenter! It was like telepathy, or group consciousness or something. I've never felt alone or bored at a conference since.

Interesting statistic: As of June 2010, about 65 million tweets were posted each day, equaling about 750 tweets sent each second. (reference)

#2 Replies

Replies are more complicated. In the example above the guy on the left, @vet, replies to @hiro by prefixing his tweet with "@hiro". However, this probably doesn't work like you think. The tweet obviously shows up in @hiro's timeline, but it also shows up in @admirer's timeline because she follows both people mentioned in the tweet (@hiro and @vet). But the tweet does not show up to @effusive because, while he follows @vet, the author of the tweet, he doesn't follow @hiro, the person being mentioned.

Getting this scoping rule wrong is way too easy. For instance you can't just announce that someone did something great. If @hiro started a tweet with "@vet gave a great anti-aardvark presentation" then most of his followers (e.g. @effusive) wouldn't actually see the tweet! I've seen seasoned twitter users make this mistake.

As an interesting aside twitter can help track conversations. If you click a reply button in a twitter client or on the website twitter will keep track of which tweet you replied to and then help piece a conversation together. But if you simply start a new tweet with "@somone" without clicking reply, the above scoping rules still apply, but twitter won't help anyone reconstruct the conversation.

Finally a word on notifications: If someone mentions you (via a reply or anywhere in their tweet) and you follow them then you will get an e-mail and possibly an SMS notification. If someone that you do not follow replies to you, you will not get an email. For this reason you must either frequently check the "mentions and more" section of the website or use a column based twitter client, discussed in the final section on lists.

Interesting statistic: On average only 23% of tweets get a reply. (source)

Another one: 38% of tweets are conversational. In other words 38% of tweets start with an "@". If your percentage of replies is lower you probably aren't using twitter to its fullest. (source)

#3 Reply-All's

So what happens if you want to reply to someone, but still have it show up in the timelines of all of your followers? As long as your tweet doesn't start with an "@" symbol then you're fine. So the convention that has grown up is to prefix your tweet with a "." and then the "@someone". In the example above @effusive now sees the tweet where he didn't in a standard reply.

Interesting statistic: The highest usage rate of twitter to date occurred during the 2011 FIFA Women's World Cup Final between Japan and the United States when 7,196 tweets were published per second! (source)

#4 Old Style Retweet or "Retweet with Comment"

Prior to late 2009 if you saw a tweet you wanted to share with your followers you would retweet (RT) it by copy and pasting their tweet and prefixing it with "RT @author". The problem from a scoping perspective was that people would get duplicates. In the example above @admirer sees a duplicate from @vet when he retweets @hiro's tweet.

There are other problems with old style retweets unrelated to scoping. One problem is that you get long chains of "RT @person1 RT @person2 …". Another problem is it was hard to follow a person but not all their retweets. The biggest problem is old style retweets take valuable characters away from your available 140, and sometimes require mangling the original.

Old style retweeting is still in use today primarily from people wish to retweet but add a comment, but also because it allows people to reach a wider audience (since new style retweets won't show up in searches or lists, more on this later).

Interesting statistic: Only 6% of tweets get retweeted (source)

#5 New Style Retweet

In November of 2009 twitter rolled out a new technique for retweeting aimed at solving the problems mentioned above. It caused a lot of confusion, but did solve the problems.

The end result of new style retweets from a scoping perspective is that sometimes people see tweets from people they don't follow. In the example above @effusive sees @hiro's tweet in his timeline even though he doesn't follow @hiro, because @vet performed a new-style retweet.

If you click the retweet button on the twitter website today it will perform a new style retweet. Some twitter clients like TweetDeck give you the option. The only way to perform an old style retweet or retweet with comment via the website is to copy and paste the message.

Interesting statistic: Less than half of tweets are posted using the web user interface. By far the most common twitter client is Tweetdeck with 8.48% of the market. (source)

#6 Direct Messages

Direct messages are simple. Prefix your tweet with "D somone" (no at symbol before the name) and they will be the only person that will see it. Just like e-mail except the recipient will get an e-mail and a text message if they've set that up on their phone. Note that neither people watching an included hashtag (e.g. @spca) nor people mentioned will see direct messages unless they are the recipient.

Interesting statistic: Six percent of all tweets are sent via SMS. (source)

#7 Lists

The last scoping topic worthy of mention is lists. This fantastic feature was added by twitter in late 2009. It allows you to organize the people you follow. For instance I have a private "Infrequent Posters" list that I try very hard to never miss a tweet from. Meanwhile I follow enough people that my main list is more like a stream that I dip into but don't get stressed out if I miss some of.

Lists work best when used in conjunction with a column based twitter clients (e.g. TweetDeck). These types of twitter clients are the only way to go if you want to become a twitter pro. If you've gotten to the point where you start missing content from important people (e.g. your real life friends), or you're missing responses to your tweets, or mentions from people you don't follow: then you probably need lists, but you absolutely need something like TweetDeck.

In regards to scoping, lists contain one subtlety that you need to be aware of. They do not support new style retweets. In the example above @vet posts a normal tweet. He then new-style retweets one of @hiro's tweets. @effusive follows @vet and has also placed him into a list. While the list helps @effusive not miss tweets from @vet, it does not end up displaying @hiro's tweet.

Note that this rule applies both to people who maintain their own lists as well as to people who follow lists maintained by other people (e.g. your company may maintain a public list of its twitter enabled employees).

Interesting Statistic: Pointless babble is the most common type of content on twitter. It represents 40.55% of all traffic. (source)

Conclusion

Twitter may appear to be fairly simple at first glance. In reality it has grown over the years to become very sophisticated. Sophisticated and incredibly powerful. But to fully tap that power you must understand the scoping rules and subtleties surrounding things like replies, retweets, and lists. Hopefully you're closer now.

CodeProject

Why Do "Not Windows Store"?

On September 16, 2011, three days after Microsoft announced a new tablet-focused operating system code named, Windows 8, Joe Ferner and I released an app store for the new pre-beta operating system called the Not Windows Store. This has caused some controversy on twitter. So why do it?

Windows Store Doesn't Exist

First and foremost, we released this service so that developers would have a place to share apps today. This is because as of right now, Microsoft has neither:

• Released the Windows Store
• Announced a date for it, nor
• Given any indiciation it will exist in the foreseeable future

This is not an appropriately great start to the fledgling operating system. Without an app store now, Windows 8 apps will exist as nothing but a loose affiliation of codeplex projects, blogs, and websites: in other words, what Windows is today. The Windows of tomorrow can and should do better than the Windows of today.

And when it's released the real Windows Store will do better. The process and the infrastructure look significantly better than any other app store in existence. In particular, it looks better than both Apple's and Android's app stores. But ultimately it's just vaporware at this point. We can do better than that.

The Problem with Windows Store Vaporware

Windows 8, both the tablet and the image, is distributed with a number of apps. These apps on the whole are fine for a proof of concept. Some of them are actually quite good (weather is amazing). But some of them, the important ones for long term use like the RSS reader, the twitter and facebook apps, and some of the games are downright mediocre. And there are huge functionality gaps like mail, contacts, calendar, tasks, let alone little things like timers and calculators.

This means Windows 8 won't demo nearly as well as it could when we show these things off to co-workers, friends, and acquaintances. And perhaps more importantly they won't keep us using the tablet or image after the novelty wears off. If these apps are all we have for the foreseeable future, that will be a detriment to Microsoft's new effort, and that would be a shame.

I for one want Windows 8 to win! I want it to win big by starting with its best foot forward; by beginning to win the hearts and minds of people making up their minds right now with more good apps; and by keeping the lucky 4,000 of us with tablets engaged and showing off our new hardware for more than the next couple of weeks.

That's why we need a place to share apps, to get feedback on apps, to improve our apps, and to show off how cool Windows 8 Metro is capable of being. However, we can't do that without something like the 'Not Windows Store'. The entire Windows community, the early adopters and developers in particular, deserve better than starting out on the wrong app-foot.

There have been rumblings on twitter that this is a bad idea. Here are some of the argument's and why I disagree:

Fallacy #1: Delaying Windows Store Increases Quality

One thought I've heard is that Microsoft is intentionally delaying the Windows Store launch so that developers will have more time to focus on quality and features instead of rushing apps in order to be first to market. That's ridiculous for a couple of reasons. First, because the decision to release a lower quality app with fewer features today in exchange for the potential loss of some customers is a business decision that we should have the option to make.

That, for example, is what the 'Not Windows Store' did. We decided to release quickly to capitalize on the excitement surrounding Windows 8 right now. We felt that was more important than delivering later with more features. If there is enough interest, we will continue to release more functionality and increase quality. If Microsoft truly delayed the Windows store intentionally (and I really don't think they did, since they could have done a limited release) that was a poor decision because it denies people the opportunity to ride the initial wave of Windows 8 excitement.

The other reason we need an app store today is that real feedback from real customers is essential for flushing out bugs and increasing quality. In the real world, settings may conflict with other apps, security permissions may change, unexpected hardware conflicts occur, internet connections fail, or functional or non-functional requirements may change. Lab testing only gets you so far. Having feedback sooner from real users in real environments will improve the quality of apps once Windows 8 goes live and once they make it into the real Windows Store.

Fallacy #2: No One Wants Un-verified Apps

The Not Windows Store does not verify apps before publishing them. There are no virus scans, no quality checks, and no disqualification criteria. But according to some on twitter people don't want a black-market-esque unmanaged place for trading apps. People may publish malware.

There are several problems with this argument. First: this is the same as Windows today. How often do people download CodePlex projects without reading a line of the source code? How often do they download applications from sketchy websites based solely on the recommendation of a friend? It happens all the time! All an app store does is centralize and simplify the process of finding, downloading, and installing apps. The real windows store will improve the process with scanning and approval. The Not Windows Store is simply a middle-ground between what we have today and what Windows users should expect in about a year.

The second reason app-verification is not essential is that Microsoft made it extremely easy to reset your machine to its factory defaults in Windows 8. This is an awesome new feature and probably will be worth doing from time to time, even if you aren't worried about malware. More importantly, out of the box Windows 8 includes a virus scanner. Don't like it? Then download something better.

The third reason app-verification is not all that important at the moment is that it is too early for there to be bad apps. A malicious user just couldn’t affect enough users today to make porting or writing a virus to a Metro app worthwhile.

The fourth and most important reason app verification is not that important is that a good app store has meta-data to increase user's confidence in an app. I'll wager this is far more important to most people than whatever process Microsoft puts apps through. We don't do much more than list author names today, but if this project takes off we will integrate ratings, comments, and social media, and that will lend more confidence than any checks Microsoft might do.

To put it another way, which would you rather have: an unverified app rated 5/5 by 200 users, published by someone you follow on twitter, with glowing comments, some of whom are posted by your friends on Facebook? Or one that has no ratings or comments but that has been through the Microsoft store process? Mmm, yea, I thought so.

Fallacy #3: One App Store to Rule Them All

Some readers might think that having multiple app stores on Windows is bad: after all, what a hassle to go to multiple places for all your apps. It might end up being as bad as the decentralization of Windows apps today, right?

I don't believe so. First of all there won't be many app stores. It would be a waste of time to compete with Microsoft for long on this front. A Microsoft sponsored store that's integrated into the OS will always be the first place people go. Anyone wanting to make money with a high quality app will need to publish there first.

I strongly believe that competition is good for an ecosystem. A great app store might completely integrate with all my social media. It might recommend apps based on what my friends like. It might suggest stuff based on my interests and hobbies as discovered by related services like Facebook. It might not charge developer's any money to publish (as is the case with the Not Windows Store of course). Or it might get the hell out of my way and guarantee complete anonymity. I can't even begin to imagine what app stores will look like in the future, but I do know that the ecosystem will be worse off if there is one, and only one, app store for Windows.

Summary

We released the Not Windows Store because we love what we see in Windows 8, because we want Microsoft to succeed, and because we want a strong third competitor to Android and Apple. If you do too then please support us either by giving the Not Windows Store a try, by uploading an app, by contributing a patch to our source code, or by following us on twitter. Let’s work together to make Windows 8 great!

Integrating JavaScript and C# with Script#

Have you ever had an enum in your server code that you wanted to access in client side code? Or wanted the safety of compile time errors when there are discrepancies between your server-side and client-side code? Or had a C# Data Transfer Object (DTO) that you wanted to enable Intellisense for in JavaScript? I hadn't found a good solution either ... until now.
Compiling C# to What!?
Compiling C# code into JavaScript may seem foreign, but Script# is a mature technology that is absolutely worth a look. Our team has been using it very happily for about three months. We've found a number of benefits including:

• Consistently working Intellisense
• Better encapsulation
• Simpler Object Orientation
• Easier deployment (Script# compiles multiple files to a single, optionally minified file)
• Safer refactoring
• Simpler unit testing
• Type safety; and
• Design time syntax checking (no more mistyping a variable and accidentally declaring it to the global scope)

Scott Hanselman covered most of these topics in last month's Hanselminutes episode: Script# Compiles to JavaScript. If you have a spare car ride I definitely recommend the listen. But what wasn't covered were some additional benefits provided by an off the beaten path approach to this great technology. The main benefit is tighter server side C# to client side JavaScript integration. A secondary benefit is less of a dependency on Script#.
To the Command Line
Typically to get going with Script# you:

1. Install the Script# Visual Studio plug-in
2. File -> New project
3. Select "Script Library" and
4. Compile to generate JavaScript

Nice and easy. The down side to this approach is you can't easily include files outside of your project. Specifically, you can't include data transfer objects or enum's from your server side code. Furthermore, all of your team members must install Script#. If instead you compile with Script#'s "ssc.exe" command you obtain more control and get less dependency.
How-To
The how-to looks something like this:

1. Download and install Script#
2. Add a Class Library (not a Script# project)
3. Project properties -> Build -> Advanced -> "Do not reference mscorlib"
   
4. Ideally move the Script# files (C:\Program Files (x86)\ScriptSharp) locally to the solution and check them into source control to a Libs or something similar
5. Remove all references, but add: ScriptSharp.dll, ScriptSharp.Web.dll, Script.Jquery
6. Edit your .csproj to manually reference Script#'s mscorlib (right click, Unload project, Edit MyProject.csproj)
   
   <Reference Include="mscorlib, Version=0.7.0.0, Culture=neutral, PublicKeyToken=8fc0e3af5abcb6c4, processorArchitecture=MSIL">
     <SpecificVersion>True</SpecificVersion>
     <HintPath>..\Libs\ScriptSharp\v1.0\mscorlib.dll</HintPath>
   </Reference>
   
7. Modify AssemblyInfo.cs and remove the following lines:
   
   [assembly: ComVisible(false)]
   [assembly: Guid("b5e2449f-193c-46d1-9023-9143618d8491")]
   
   
8. Modify AssemblyInfo.cs and add the following:
   
   [assembly: ScriptAssembly("ScriptSharpDemoAssembly")]
   
   
9. Ensure it compiles in Visual Studio
10. Create a batch script or PowerShell script that compiles using ssc.exe like this:
    
    ..\Libs\ScriptSharp\v1.0\ssc.exe ^
        /debug ^
        /out:MyScriptSharp.js ^
        /ref:"..\Libs\ScriptSharp\v1.0\Framework\mscorlib.dll" ^
        .\Properties\AssemblyInfo.cs ^
        .\Class1.cs
    
    

Notice the last two lines in the script are a list of the files that you want to include. The point of this exercise is that you can now include files outside of your main Script# project in that list.
Now for completeness if you put a simple static method in Class1.cs, something like this:
namespace MyScriptSharp
{
    public class Class1
    {
        public static string HelloWorld()
        {
            return "Hello World!";
        }
    }
}
Then run the batch file you should get something like this:
Type.registerNamespace('MyScriptSharp');

////////////////////////////////////////////////////////////////////////////////
// MyScriptSharp.Class1

MyScriptSharp.Class1 = function MyScriptSharp_Class1() {
}
MyScriptSharp.Class1.helloWorld = function MyScriptSharp_Class1$helloWorld() {
    return 'Hello World!';
}

MyScriptSharp.Class1.registerClass('MyScriptSharp.Class1');
Obviously you could get these results faster with approach #1. But now you have a lot more control.
Summary
The main downside to this approach is maintaining the batch file is a bit of a hassle. But the upsides are that you can include any file from your server-side C# code. And any changes in that server-side code are automatically reflected in your JavaScript. And any breaking changes in your server-side code generate compile time errors in your client side code. And furthermore none of your team members need to install Script#. For our team it's an easy tradeoff. What about for yours? Please share your thoughts in the comments or on twitter.

Advanced Burn Up Charts

If you’ve read my previous posts on the subject you know that I love agile burn up charts for managing SCRUM style projects, particularly compared to burn down charts.

The problem is that burndown charts lack two essential pieces of information. First, how much work was actually accomplished during a given iteration (as opposed to how much work remains to be completed) and second how much total work the project contains (or if you prefer how much scope has increased each iteration).

What you might not know is how flexible they can be. In particular the example I gave last time has a problem. Can you spot it?

The problem is that the scope increase of 20 points in iteration six might very well have had zero impact on the actual deployment date. Those points might have all been low priority tasks that the team will work on after they deploy the initial 100 points. Wouldn’t it be nice if the burn up chart could convey priority information as well?

So that’s exactly what I did on my current project last week, and it appeared to be a big hit. And with the addition of some Excel trend lines the chart was able to convey a lot of insight into the project:

Obviously all that data is made up, but the chart still tells an interesting story:

• The customer hasn't been adding high priority tasks, perhaps because there is a hard deadline approaching
• Because of the customer’s restraint in adding high priority tasks the team can be fairly confident that they will finish all the high priority tasks by Iteration 3
• While the customer hasn’t added high priority tasks they have still been able to be agile by adding normal and low priority tasks

What’s really wonderful about this way of reporting is that it gives the customer the flexibility to add scope to future iterations while maintaining near term deadlines. In short it supports what agile is supposed to be about.

But why stop there? Depending on the story you need to tell or the scenario you need to evaluate perhaps you could incorporate team member contributions. For example:

In this example we can try to show what might happen if we remove a particular team member or if we’d never had a particular team member. What’s unique about this way or reporting is that it’s incorporating the backlog’s increasing trend into account. While it’s possible that the backlog may increase or decrease at a different rate with a different team makeup, it’s a better scenario than ignoring the rate of increase of the backlog all together.

Perhaps we could incorporate other information. Maybe bug vs. feature information. Or planned vs. actual (e.g. task slippage) information. It seems like there is a lot of potential. Any other ideas out there? Feel free to post in the comments or via twitter. If there’s enough interest I’d be happy to post another screencast similar to the last one I did on how to produce burn up and burn down with SharePoint and Excel except this time with trends and priorities.

So overall while these charts may be a little more complicated than a traditional burn down chart they would make an excellent talking point during a PowerPoint presentation, or as part of your iteration close out. And with a little training I bet just about any customer would learn to love the insight they provide.

Death to the DAO and How to Test LINQ

Occasionally I hear complaints that LINQ is hard to unit test. These complaints aren’t about LINQ to objects, mind you, they’re specific to the complexities of the flavors of LINQ that turn C# code into something else like SQL or CAML using expression trees. The most common technologies are LINQ to SQL, the Entity Framework, or in my case at the moment LINQ to SharePoint. In this post I’m going to propose a technique that makes testing LINQ not just easy, but downright elegant – assuming you’re ok with extension methods – lots of extension methods. And assuming you’re ready to kill your Data Access Objects (DAO) tier.

The Unit Testing Problem

Any architecture needs a place to put code that finds entities. For instance FindBySocialSecurityNumber(). In a traditional architecture we might put a method like this is in a DAO layer. If so our method will look something like this:

public class EmployeesDao {
    public Employee FindBySSN(Context ctx, string ssn) {
        return ctx.Employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

So how would we go about unit testing this?

One fairly typical solution would be to use an in-memory database. That approach works if our data store is a database, but it certainly doesn’t work if the data store is something less traditional like SharePoint. But even if our store is a database, we’ll still have the hassle of setting up the in-memory database.

Another solution might be to use a mock Context that returns an IQueryable. But wouldn’t it be wonderful if we could avoid mocking all together?

Killing the DAO

The first question is why we even have a DAO tier to begin with. The original idea was that we wanted a place to put code specific to a particular data store. In other words we wanted to isolate the code that will need to be changed should the data store switch from SQL Server to Oracle. But isn’t that exactly what LINQ does? I’d be pretty surprised if there wasn’t a decent LINQ provider for just about any data store at this point that required more than minimal code changes. So why not embrace LINQ and reconsider alternatives to a DAO tier?

One alternative that I’ve been using for over a month now is to switch to extension methods. To give credit where it’s due the idea originated with a conversation with fellow Near Infinity employee Joe Ferner. And I'm sure the idea isn't particularly original (please post in the comments if you know others that use this approach).

Using this technique our code changes from something like this:

var employeeDao = new EmployeesDao(); // or use IOC of course
employeeDao.FindBySSN(ctx, "111-11-1111");

To something like this:

ctx.Employees.FindBySSN("111-11-1111");

Among other things I find this far more aesthetically pleasing because each of the three elements to the statement represent a subsequent filtering of data. It's a more functional way of looking at things.

We could implement this off of the Employees property of the context if we have control over that (which I don't with spmetal). But if we implement this as an extension method like this:

public static class EmployeeExtensions {
    public static Employee FindBySSN(this IQueryable<Employee> employees, string ssn) {
        return employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

We now have something that’s considerable easier to unit test.

Testing It

Once we’ve refactored our function as an extension method that filters down the corpus of entities, we can test the code using in-memory objects with a call to .AsQueryable(). For instance:

public void FindBySSN_OneSsnExists_EmployeeReturned() {
       var employees = new [] { new Employee { Ssn = "111-11-1111" } };
       var actual = employees.AsQueryable().FindBySSN("111-11-1111");
       Assert.IsNotNull(actual);
}

Notice we didn’t have to mock anything.

Testability, but at What Cost?

This technique works great for the example above, but how does it scale to harder problems and what other downsides are there?

As far as scalability I’ve found this technique works great for every scenario I’ve run across in the month I’ve been doing it. It works for joins, aggregations, and even for inserts, update, and deletes.

As far as downsides the astute reader may be wondering about mockability. For instance what if we want to mock the call to FindBySSN and give it the exact Employee that will be returned. This scenario is admittedly harder. But what I've found is that far more often than not I don’t really need to mock the types of things that used to live in the DAO tier. Instead I just mock the Employee object off of context to return in-memory objects and make my tests slightly larger in scope. Most of the time I find the larger scope increases the usefulness of the test. In the occasional case where I do really want to mock the "DAO" tier I use a technique described in either this post or this post by Daniel Cazzulino.

Conclusion

Obviously there is more to this architecture, for instance how do you handle insert and update operations? The short answer is it’s easy, but I’ll save that topic for a future post. For now why not give this approach a try? You weren’t really happy with that useless old DAO tier anyway, were you? I say we eradicate it and never look back.