Monday, November 28, 2011

How To Get The Most From Twitter: Scoping Rules

Ever noticed a tweet disappear? As in someone else can't see something you posted? There's a common mistake even twitter pros make that causes this to occur.

Are you sure you're seeing all of the replies to your tweets -- even from people you don't follow? Do you ever find yourself missing important posts -- like those from your real life friends? Do you know which will reach a wider audience: new style retweets vs old style retweets?

This article will answer these and other twitter related questions. It will help beginner to intermediate twitter users get the most out of the service.

#1 Standard #Tweets

These appear simple. These show up in the timeline of your followers (@admirer and @vet in the example above). And they don't show up in the timeline of anyone who doesn't follow you (@effusive). If this was all there was to twitter it would be a complete waste of time.

The subtlety with this type of tweet is that your content may show up to someone that doesn't follow you but that is following a hashtag or a topic you mention. In the example above @hiro thows a hashtag onto the word aardvarks to indicate that it's special somehow. His tweet is then picked up by @spca who runs a constant search in a twitter client for the word "#aardvark".

The first time I realized how powerful this could be I had was at a conference. I was extremely bored by a terrible keynote. With nothing else to do I discovered the conference hashtag on twitter and started watching it. Suddenly I realized I wasn't alone. I was in a huge room full of people making fun of the presenter! It was like telepathy, or group consciousness or something. I've never felt alone or bored at a conference since.

Interesting statistic: As of June 2010, about 65 million tweets were posted each day, equaling about 750 tweets sent each second. (reference)

#2 Replies

Replies are more complicated. In the example above the guy on the left, @vet, replies to @hiro by prefixing his tweet with "@hiro". However, this probably doesn't work like you think. The tweet obviously shows up in @hiro's timeline, but it also shows up in @admirer's timeline because she follows both people mentioned in the tweet (@hiro and @vet). But the tweet does not show up to @effusive because, while he follows @vet, the author of the tweet, he doesn't follow @hiro, the person being mentioned.

Getting this scoping rule wrong is way too easy. For instance you can't just announce that someone did something great. If @hiro started a tweet with "@vet gave a great anti-aardvark presentation" then most of his followers (e.g. @effusive) wouldn't actually see the tweet! I've seen seasoned twitter users make this mistake.

As an interesting aside twitter can help track conversations. If you click a reply button in a twitter client or on the website twitter will keep track of which tweet you replied to and then help piece a conversation together. But if you simply start a new tweet with "@somone" without clicking reply, the above scoping rules still apply, but twitter won't help anyone reconstruct the conversation.

Finally a word on notifications: If someone mentions you (via a reply or anywhere in their tweet) and you follow them then you will get an e-mail and possibly an SMS notification. If someone that you do not follow replies to you, you will not get an email. For this reason you must either frequently check the "mentions and more" section of the website or use a column based twitter client, discussed in the final section on lists.

Interesting statistic: On average only 23% of tweets get a reply. (source)

Another one: 38% of tweets are conversational. In other words 38% of tweets start with an "@". If your percentage of replies is lower you probably aren't using twitter to its fullest. (source)

#3 Reply-All's

So what happens if you want to reply to someone, but still have it show up in the timelines of all of your followers? As long as your tweet doesn't start with an "@" symbol then you're fine. So the convention that has grown up is to prefix your tweet with a "." and then the "@someone". In the example above @effusive now sees the tweet where he didn't in a standard reply.

Interesting statistic: The highest usage rate of twitter to date occurred during the 2011 FIFA Women's World Cup Final between Japan and the United States when 7,196 tweets were published per second! (source)

#4 Old Style Retweet or "Retweet with Comment"

Prior to late 2009 if you saw a tweet you wanted to share with your followers you would retweet (RT) it by copy and pasting their tweet and prefixing it with "RT @author". The problem from a scoping perspective was that people would get duplicates. In the example above @admirer sees a duplicate from @vet when he retweets @hiro's tweet.

There are other problems with old style retweets unrelated to scoping. One problem is that you get long chains of "RT @person1 RT @person2 …". Another problem is it was hard to follow a person but not all their retweets. The biggest problem is old style retweets take valuable characters away from your available 140, and sometimes require mangling the original.

Old style retweeting is still in use today primarily from people wish to retweet but add a comment, but also because it allows people to reach a wider audience (since new style retweets won't show up in searches or lists, more on this later).

Interesting statistic: Only 6% of tweets get retweeted (source)

#5 New Style Retweet

In November of 2009 twitter rolled out a new technique for retweeting aimed at solving the problems mentioned above. It caused a lot of confusion, but did solve the problems.

The end result of new style retweets from a scoping perspective is that sometimes people see tweets from people they don't follow. In the example above @effusive sees @hiro's tweet in his timeline even though he doesn't follow @hiro, because @vet performed a new-style retweet.

If you click the retweet button on the twitter website today it will perform a new style retweet. Some twitter clients like TweetDeck give you the option. The only way to perform an old style retweet or retweet with comment via the website is to copy and paste the message.

Interesting statistic: Less than half of tweets are posted using the web user interface. By far the most common twitter client is Tweetdeck with 8.48% of the market. (source)

#6 Direct Messages

Direct messages are simple. Prefix your tweet with "D somone" (no at symbol before the name) and they will be the only person that will see it. Just like e-mail except the recipient will get an e-mail and a text message if they've set that up on their phone. Note that neither people watching an included hashtag (e.g. @spca) nor people mentioned will see direct messages unless they are the recipient.

Interesting statistic: Six percent of all tweets are sent via SMS. (source)

#7 Lists

The last scoping topic worthy of mention is lists. This fantastic feature was added by twitter in late 2009. It allows you to organize the people you follow. For instance I have a private "Infrequent Posters" list that I try very hard to never miss a tweet from. Meanwhile I follow enough people that my main list is more like a stream that I dip into but don't get stressed out if I miss some of.

Lists work best when used in conjunction with a column based twitter clients (e.g. TweetDeck). These types of twitter clients are the only way to go if you want to become a twitter pro. If you've gotten to the point where you start missing content from important people (e.g. your real life friends), or you're missing responses to your tweets, or mentions from people you don't follow: then you probably need lists, but you absolutely need something like TweetDeck.

In regards to scoping, lists contain one subtlety that you need to be aware of. They do not support new style retweets. In the example above @vet posts a normal tweet. He then new-style retweets one of @hiro's tweets. @effusive follows @vet and has also placed him into a list. While the list helps @effusive not miss tweets from @vet, it does not end up displaying @hiro's tweet.

Note that this rule applies both to people who maintain their own lists as well as to people who follow lists maintained by other people (e.g. your company may maintain a public list of its twitter enabled employees).

Interesting Statistic: Pointless babble is the most common type of content on twitter. It represents 40.55% of all traffic. (source)

Conclusion

Twitter may appear to be fairly simple at first glance. In reality it has grown over the years to become very sophisticated. Sophisticated and incredibly powerful. But to fully tap that power you must understand the scoping rules and subtleties surrounding things like replies, retweets, and lists. Hopefully you're closer now.

Sunday, September 18, 2011

Why Do "Not Windows Store"?

On September 16, 2011, three days after Microsoft announced a new tablet-focused operating system code named, Windows 8, Joe Ferner and I released an app store for the new pre-beta operating system called the Not Windows Store. This has caused some controversy on twitter. So why do it?

Windows Store Doesn't Exist

First and foremost, we released this service so that developers would have a place to share apps today. This is because as of right now, Microsoft has neither:

  • Released the Windows Store
  • Announced a date for it, nor
  • Given any indiciation it will exist in the foreseeable future

This is not an appropriately great start to the fledgling operating system. Without an app store now, Windows 8 apps will exist as nothing but a loose affiliation of codeplex projects, blogs, and websites: in other words, what Windows is today. The Windows of tomorrow can and should do better than the Windows of today.

And when it's released the real Windows Store will do better. The process and the infrastructure look significantly better than any other app store in existence. In particular, it looks better than both Apple's and Android's app stores. But ultimately it's just vaporware at this point. We can do better than that.

The Problem with Windows Store Vaporware

Windows 8, both the tablet and the image, is distributed with a number of apps. These apps on the whole are fine for a proof of concept. Some of them are actually quite good (weather is amazing). But some of them, the important ones for long term use like the RSS reader, the twitter and facebook apps, and some of the games are downright mediocre. And there are huge functionality gaps like mail, contacts, calendar, tasks, let alone little things like timers and calculators.

This means Windows 8 won't demo nearly as well as it could when we show these things off to co-workers, friends, and acquaintances. And perhaps more importantly they won't keep us using the tablet or image after the novelty wears off. If these apps are all we have for the foreseeable future, that will be a detriment to Microsoft's new effort, and that would be a shame.

I for one want Windows 8 to win! I want it to win big by starting with its best foot forward; by beginning to win the hearts and minds of people making up their minds right now with more good apps; and by keeping the lucky 4,000 of us with tablets engaged and showing off our new hardware for more than the next couple of weeks.

That's why we need a place to share apps, to get feedback on apps, to improve our apps, and to show off how cool Windows 8 Metro is capable of being. However, we can't do that without something like the 'Not Windows Store'. The entire Windows community, the early adopters and developers in particular, deserve better than starting out on the wrong app-foot.

There have been rumblings on twitter that this is a bad idea. Here are some of the argument's and why I disagree:

Fallacy #1: Delaying Windows Store Increases Quality

One thought I've heard is that Microsoft is intentionally delaying the Windows Store launch so that developers will have more time to focus on quality and features instead of rushing apps in order to be first to market. That's ridiculous for a couple of reasons. First, because the decision to release a lower quality app with fewer features today in exchange for the potential loss of some customers is a business decision that we should have the option to make.

That, for example, is what the 'Not Windows Store' did. We decided to release quickly to capitalize on the excitement surrounding Windows 8 right now. We felt that was more important than delivering later with more features. If there is enough interest, we will continue to release more functionality and increase quality. If Microsoft truly delayed the Windows store intentionally (and I really don't think they did, since they could have done a limited release) that was a poor decision because it denies people the opportunity to ride the initial wave of Windows 8 excitement.

The other reason we need an app store today is that real feedback from real customers is essential for flushing out bugs and increasing quality. In the real world, settings may conflict with other apps, security permissions may change, unexpected hardware conflicts occur, internet connections fail, or functional or non-functional requirements may change. Lab testing only gets you so far. Having feedback sooner from real users in real environments will improve the quality of apps once Windows 8 goes live and once they make it into the real Windows Store.

Fallacy #2: No One Wants Un-verified Apps

The Not Windows Store does not verify apps before publishing them. There are no virus scans, no quality checks, and no disqualification criteria. But according to some on twitter people don't want a black-market-esque unmanaged place for trading apps. People may publish malware.

There are several problems with this argument. First: this is the same as Windows today. How often do people download CodePlex projects without reading a line of the source code? How often do they download applications from sketchy websites based solely on the recommendation of a friend? It happens all the time! All an app store does is centralize and simplify the process of finding, downloading, and installing apps. The real windows store will improve the process with scanning and approval. The Not Windows Store is simply a middle-ground between what we have today and what Windows users should expect in about a year.

The second reason app-verification is not essential is that Microsoft made it extremely easy to reset your machine to its factory defaults in Windows 8. This is an awesome new feature and probably will be worth doing from time to time, even if you aren't worried about malware. More importantly, out of the box Windows 8 includes a virus scanner. Don't like it? Then download something better.

The third reason app-verification is not all that important at the moment is that it is too early for there to be bad apps. A malicious user just couldn’t affect enough users today to make porting or writing a virus to a Metro app worthwhile.

The fourth and most important reason app verification is not that important is that a good app store has meta-data to increase user's confidence in an app. I'll wager this is far more important to most people than whatever process Microsoft puts apps through. We don't do much more than list author names today, but if this project takes off we will integrate ratings, comments, and social media, and that will lend more confidence than any checks Microsoft might do.

To put it another way, which would you rather have: an unverified app rated 5/5 by 200 users, published by someone you follow on twitter, with glowing comments, some of whom are posted by your friends on Facebook? Or one that has no ratings or comments but that has been through the Microsoft store process? Mmm, yea, I thought so.

Fallacy #3: One App Store to Rule Them All

Some readers might think that having multiple app stores on Windows is bad: after all, what a hassle to go to multiple places for all your apps. It might end up being as bad as the decentralization of Windows apps today, right?

I don't believe so. First of all there won't be many app stores. It would be a waste of time to compete with Microsoft for long on this front. A Microsoft sponsored store that's integrated into the OS will always be the first place people go. Anyone wanting to make money with a high quality app will need to publish there first.

I strongly believe that competition is good for an ecosystem. A great app store might completely integrate with all my social media. It might recommend apps based on what my friends like. It might suggest stuff based on my interests and hobbies as discovered by related services like Facebook. It might not charge developer's any money to publish (as is the case with the Not Windows Store of course). Or it might get the hell out of my way and guarantee complete anonymity. I can't even begin to imagine what app stores will look like in the future, but I do know that the ecosystem will be worse off if there is one, and only one, app store for Windows.

Summary

We released the Not Windows Store because we love what we see in Windows 8, because we want Microsoft to succeed, and because we want a strong third competitor to Android and Apple. If you do too then please support us either by giving the Not Windows Store a try, by uploading an app, by contributing a patch to our source code, or by following us on twitter. Let’s work together to make Windows 8 great!

Tuesday, August 23, 2011

Integrating JavaScript and C# with Script#

Have you ever had an enum in your server code that you wanted to access in client side code? Or wanted the safety of compile time errors when there are discrepancies between your server-side and client-side code? Or had a C# Data Transfer Object (DTO) that you wanted to enable Intellisense for in JavaScript? I hadn't found a good solution either ... until now.

Compiling C# to What!?

Compiling C# code into JavaScript may seem foreign, but Script# is a mature technology that is absolutely worth a look. Our team has been using it very happily for about three months. We've found a number of benefits including:

  • Consistently working Intellisense
  • Better encapsulation
  • Simpler Object Orientation
  • Easier deployment (Script# compiles multiple files to a single, optionally minified file)
  • Safer refactoring
  • Simpler unit testing
  • Type safety; and
  • Design time syntax checking (no more mistyping a variable and accidentally declaring it to the global scope)

Scott Hanselman covered most of these topics in last month's Hanselminutes episode: Script# Compiles to JavaScript. If you have a spare car ride I definitely recommend the listen. But what wasn't covered were some additional benefits provided by an off the beaten path approach to this great technology. The main benefit is tighter server side C# to client side JavaScript integration. A secondary benefit is less of a dependency on Script#.

To the Command Line

Typically to get going with Script# you:

  1. Install the Script# Visual Studio plug-in
  2. File -> New project
  3. Select "Script Library" and
  4. Compile to generate JavaScript

Nice and easy. The down side to this approach is you can't easily include files outside of your project. Specifically, you can't include data transfer objects or enum's from your server side code. Furthermore, all of your team members must install Script#. If instead you compile with Script#'s "ssc.exe" command you obtain more control and get less dependency.

How-To

The how-to looks something like this:

  1. Download and install Script#
  2. Add a Class Library (not a Script# project)
  3. Project properties -> Build -> Advanced -> "Do not reference mscorlib"
  4. Ideally move the Script# files (C:\Program Files (x86)\ScriptSharp) locally to the solution and check them into source control to a Libs or something similar
  5. Remove all references, but add: ScriptSharp.dll, ScriptSharp.Web.dll, Script.Jquery
  6. Edit your .csproj to manually reference Script#'s mscorlib (right click, Unload project, Edit MyProject.csproj)

    <Reference Include="mscorlib, Version=0.7.0.0, Culture=neutral, PublicKeyToken=8fc0e3af5abcb6c4, processorArchitecture=MSIL">
      <
    SpecificVersion>True</SpecificVersion>
      <
    HintPath>..\Libs\ScriptSharp\v1.0\mscorlib.dll</HintPath>
    </
    Reference>


  7. Modify AssemblyInfo.cs and remove the following lines:

    [assembly: ComVisible(false)]
    [assembly: Guid("b5e2449f-193c-46d1-9023-9143618d8491")]

  8. Modify AssemblyInfo.cs and add the following:

    [assembly: ScriptAssembly("ScriptSharpDemoAssembly")

  9. Ensure it compiles in Visual Studio
  10. Create a batch script or PowerShell script that compiles using ssc.exe like this:

    ..\Libs\ScriptSharp\v1.0\ssc.exe ^
        /debug ^
        /out:MyScriptSharp.js ^
        /ref:"..\Libs\ScriptSharp\v1.0\Framework\mscorlib.dll" ^
        .\Properties\AssemblyInfo.cs ^
        .\Class1.cs

Notice the last two lines in the script are a list of the files that you want to include. The point of this exercise is that you can now include files outside of your main Script# project in that list.

Now for completeness if you put a simple static method in Class1.cs, something like this:

namespace MyScriptSharp
{
    public class Class1
    {
        public static string HelloWorld()
        {
            return "Hello World!";
        }
    }
}

Then run the batch file you should get something like this:

Type.registerNamespace('MyScriptSharp');

////////////////////////////////////////////////////////////////////////////////
// MyScriptSharp.Class1

MyScriptSharp.Class1 = function MyScriptSharp_Class1() {
}
MyScriptSharp.Class1.helloWorld = function MyScriptSharp_Class1$helloWorld() {
    return 'Hello World!';
}

MyScriptSharp.Class1.registerClass('MyScriptSharp.Class1');

Obviously you could get these results faster with approach #1. But now you have a lot more control.

Summary

The main downside to this approach is maintaining the batch file is a bit of a hassle. But the upsides are that you can include any file from your server-side C# code. And any changes in that server-side code are automatically reflected in your JavaScript. And any breaking changes in your server-side code generate compile time errors in your client side code. And furthermore none of your team members need to install Script#. For our team it's an easy tradeoff. What about for yours? Please share your thoughts in the comments or on twitter.

Monday, April 11, 2011

Advanced Burn Up Charts

If you’ve read my previous posts on the subject you know that I love agile burn up charts for managing SCRUM style projects, particularly compared to burn down charts.

The problem is that burndown charts lack two essential pieces of information. First, how much work was actually accomplished during a given iteration (as opposed to how much work remains to be completed) and second how much total work the project contains (or if you prefer how much scope has increased each iteration).

What you might not know is how flexible they can be. In particular the example I gave last time has a problem. Can you spot it?

The problem is that the scope increase of 20 points in iteration six might very well have had zero impact on the actual deployment date. Those points might have all been low priority tasks that the team will work on after they deploy the initial 100 points. Wouldn’t it be nice if the burn up chart could convey priority information as well?

So that’s exactly what I did on my current project last week, and it appeared to be a big hit. And with the addition of some Excel trend lines the chart was able to convey a lot of insight into the project:

Obviously all that data is made up, but the chart still tells an interesting story:

  • The customer hasn't been adding high priority tasks, perhaps because there is a hard deadline approaching
  • Because of the customer’s restraint in adding high priority tasks the team can be fairly confident that they will finish all the high priority tasks by Iteration 3
  • While the customer hasn’t added high priority tasks they have still been able to be agile by adding normal and low priority tasks

What’s really wonderful about this way of reporting is that it gives the customer the flexibility to add scope to future iterations while maintaining near term deadlines. In short it supports what agile is supposed to be about.

But why stop there? Depending on the story you need to tell or the scenario you need to evaluate perhaps you could incorporate team member contributions. For example:

In this example we can try to show what might happen if we remove a particular team member or if we’d never had a particular team member. What’s unique about this way or reporting is that it’s incorporating the backlog’s increasing trend into account. While it’s possible that the backlog may increase or decrease at a different rate with a different team makeup, it’s a better scenario than ignoring the rate of increase of the backlog all together.

Perhaps we could incorporate other information. Maybe bug vs. feature information. Or planned vs. actual (e.g. task slippage) information. It seems like there is a lot of potential. Any other ideas out there? Feel free to post in the comments or via twitter. If there’s enough interest I’d be happy to post another screencast similar to the last one I did on how to produce burn up and burn down with SharePoint and Excel except this time with trends and priorities.

So overall while these charts may be a little more complicated than a traditional burn down chart they would make an excellent talking point during a PowerPoint presentation, or as part of your iteration close out. And with a little training I bet just about any customer would learn to love the insight they provide.

Thursday, July 22, 2010

Death to the DAO and How to Test LINQ

Occasionally I hear complaints that LINQ is hard to unit test. These complaints aren’t about LINQ to objects, mind you, they’re specific to the complexities of the flavors of LINQ that turn C# code into something else like SQL or CAML using expression trees. The most common technologies are LINQ to SQL, the Entity Framework, or in my case at the moment LINQ to SharePoint. In this post I’m going to propose a technique that makes testing LINQ not just easy, but downright elegant – assuming you’re ok with extension methods – lots of extension methods. And assuming you’re ready to kill your Data Access Objects (DAO) tier.

The Unit Testing Problem

Any architecture needs a place to put code that finds entities. For instance FindBySocialSecurityNumber(). In a traditional architecture we might put a method like this is in a DAO layer. If so our method will look something like this:

public class EmployeesDao {
    public Employee FindBySSN(Context ctx, string ssn) {
        return ctx.Employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

So how would we go about unit testing this?

One fairly typical solution would be to use an in-memory database. That approach works if our data store is a database, but it certainly doesn’t work if the data store is something less traditional like SharePoint. But even if our store is a database, we’ll still have the hassle of setting up the in-memory database.

Another solution might be to use a mock Context that returns an IQueryable. But wouldn’t it be wonderful if we could avoid mocking all together?

Killing the DAO

The first question is why we even have a DAO tier to begin with. The original idea was that we wanted a place to put code specific to a particular data store. In other words we wanted to isolate the code that will need to be changed should the data store switch from SQL Server to Oracle. But isn’t that exactly what LINQ does? I’d be pretty surprised if there wasn’t a decent LINQ provider for just about any data store at this point that required more than minimal code changes. So why not embrace LINQ and reconsider alternatives to a DAO tier?

One alternative that I’ve been using for over a month now is to switch to extension methods. To give credit where it’s due the idea originated with a conversation with fellow Near Infinity employee Joe Ferner. And I'm sure the idea isn't particularly original (please post in the comments if you know others that use this approach).

Using this technique our code changes from something like this:

var employeeDao = new EmployeesDao(); // or use IOC of course
employeeDao.FindBySSN(ctx, "111-11-1111");

To something like this:

ctx.Employees.FindBySSN("111-11-1111");

Among other things I find this far more aesthetically pleasing because each of the three elements to the statement represent a subsequent filtering of data. It's a more functional way of looking at things.

We could implement this off of the Employees property of the context if we have control over that (which I don't with spmetal). But if we implement this as an extension method like this:

public static class EmployeeExtensions {
    public static Employee FindBySSN(this IQueryable<Employee> employees, string ssn) {
        return employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

We now have something that’s considerable easier to unit test.

Testing It

Once we’ve refactored our function as an extension method that filters down the corpus of entities, we can test the code using in-memory objects with a call to .AsQueryable(). For instance:

public void FindBySSN_OneSsnExists_EmployeeReturned() {
       var employees = new [] { new Employee { Ssn = "111-11-1111" } };
       var actual = employees.AsQueryable().FindBySSN("111-11-1111");
       Assert.IsNotNull(actual);
}

Notice we didn’t have to mock anything.

Testability, but at What Cost?

This technique works great for the example above, but how does it scale to harder problems and what other downsides are there?

As far as scalability I’ve found this technique works great for every scenario I’ve run across in the month I’ve been doing it. It works for joins, aggregations, and even for inserts, update, and deletes.

As far as downsides the astute reader may be wondering about mockability. For instance what if we want to mock the call to FindBySSN and give it the exact Employee that will be returned. This scenario is admittedly harder. But what I've found is that far more often than not I don’t really need to mock the types of things that used to live in the DAO tier. Instead I just mock the Employee object off of context to return in-memory objects and make my tests slightly larger in scope. Most of the time I find the larger scope increases the usefulness of the test. In the occasional case where I do really want to mock the "DAO" tier I use a technique described in either this post or this post by Daniel Cazzulino.

Conclusion

Obviously there is more to this architecture, for instance how do you handle insert and update operations? The short answer is it’s easy, but I’ll save that topic for a future post. For now why not give this approach a try? You weren’t really happy with that useless old DAO tier anyway, were you? I say we eradicate it and never look back.

Saturday, April 10, 2010

ASP.AJAX 4 Templates Part 4 - jQuery, and Manual JSON Object Manipulation

Note: This is part 4 of a multi-part series exploring building client side AJAX applications with ASP.Net AJAX 4 Templating and the WCF Data Services (aka ADO.Net Data Services, aka oData, aka Astoria) in SharePoint 2010.

Part 1 – Exploring WCF Data Services in SharePoint 2010
Part 2 – Creating a read-only page with ASP.Net AJAX 4.0 Templates
Part 3 – Writing data back to SharePoint with ASP.Net AJAX 4.0 Templates
Part 4 - jQuery and Manual JSON Object Manipulation

In the previous posts in this series I’ve shown how you can access SharePoint data using WCF Data Services, how you can display that data using ASP.Net AJAX 4.0 Templates, and how you can write data back to SharePoint.

In this post I’ll take writing data back to SharePoint a step further by showing how you can modify non-visible fields by modifying JSON objects directly. More specifically I’ll enable dragging user story index cards on a virtual bulletin board using a jQuery plugin, save the X and Y coordinates back to JSON objects, and then save the updated records back to SharePoint.

Cards on a Corkboard

The first thing I’ll do is style the cards and display them with absolute position. While I’m at it I’ll also reference the draggable jQuery UI plugin. If you’re following along at home just built a custom .js download from the jQuery UI site, drop it in your layouts directory and add a reference. So my PageHead content control now looks like this:

<asp:content
    id="PageHead"
    contentplaceholderid="PlaceHolderAdditionalPageHead"
    runat="server">

    <script type="text/javascript"
        src="../ajax/MicrosoftAjax.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxDataContext.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxTemplates.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxAdoNet.js"></script>
    <script type="text/javascript"
        src="../jquery-ui-1.8.custom/js/jquery-1.4.2.min.js"></script>
    <script type="text/javascript"
        src="../jquery-ui-1.8.custom/js/jquery-ui-1.8.custom.min.js">
    </script>
    <style type="text/css">
        .sys-template
        {
            display: none;
        }
        .userstorycard
        {
            border: 1px solid #777777;
            width: 200px;
            position: absolute;
            cursor: move;
        }
        .carddescription
        {
            font-size: 13px;
            background-image: url('card_bg.jpg');
            padding: 0px 5px 5px 5px;
        }
        .cardtitle
        {
            font-size: 15px;
            font-weight: bold;
            padding: 0px 0px 0px 0px;
            border-bottom: 1px solid red;
            background-color: White;
            padding: 0px 5px 0px 5px;
        }

        ...

The next thing to do is add X and Y fields to the user story list in SharePoint. And finally display the cards with absolute positioning:

<div xmlns:sys="javascript:Sys" class="background">
  <div id="userStories" class="sys-template">
    <div class="userStoryCard"
      sys:style="{{ 'left: ' + X + 'px; top: ' + Y + 'px;'}}">

      <div class="userStoryTitle">{{ Title }}</div>
      <div class="userStoryBody"><div>{{ Description }}</div>
    </div>
  </div>
</
div>

Notice the sys:style attribute. The sys: part is the namespace you need to add for attributes when using templating that I mentioned in part three. Also notice the JavaScript string concatenation inside the binding. You can put in any JavaScript in there that you like. So now if you manually set X and Y values in SharePoint for a card the card shows in the right place. But we’re still missing the ability to drag cards.

Making Cards Draggable with jQuery

In order to enable dragging support you’re supposed to call the draggable() function on the DOM elements you want to make draggable. Simple right? You just call $(".userstorycard").draggable() which should find every element with a class of userstorycard and call draggable() on it.

But if you do this onLoad() then ASP.Net AJAX Templating engine hasn’t had a chance to render the new DOM elements yet. Fortunately the DataView has the JavaScript equivalent of an OnRendered event that you can tie into:

function onLoad() {
    dataContext = $create(
        Sys.Data.AdoNetDataContext,
        { serviceUri: "/demoprep/_vti_bin/ListData.svc" }
        );

    dataView = $create(
        Sys.UI.DataView,
        {
            autoFetch: true,
            dataProvider: dataContext,
            fetchOperation: "UserStories",
            fetchParameters: { $top: 20 }
        },
        {
            rendered: onRendered
        },
        null,
        $get('userStories')
    );
}

function onRendered() {
    // from http://jqueryui.com/demos/draggable/
    $(".userstorycard").draggable();
}

And that’s it for enabling dragging. Pretty simple. The result looks like this:

Ok, well you can’t see the dragging support, but it’s there, trust me. And it’s pretty cool except for one thing.

Saving Non-Visible Properties

The problem is that every time you refresh the page the nice layout you’ve done disappears and everything goes back to being stacked one on top of the other in the upper left. My first thought was to create input type=hidden form elements on the page. This approach causes a lot more plumbing code than is necessary. The better way is that you can access the underlying in-memory JSON objects. Step one is to register an “event” for when the user stops dragging a card.

function onRendered() {
    // from http://jqueryui.com/demos/draggable/
    $(".userstorycard").draggable({
        stop: onDragStop
    });
}

Once you wire that up the onDragStop() function is where all the interesting stuff happens:

function onDragStop(event, ui) {
    var userStoryCard = ui.helper[0];
    var selectedUserStoryJsonObject =
        dataView.findContext(userStoryCard).dataItem;

    var newX = ui.position.left;
    var newY = ui.position.top;

    Sys.Observer.setValue(selectedUserStoryJsonObject, "X", newX);
    Sys.Observer.setValue(selectedUserStoryJsonObject, "Y", newY);
    dataContext.saveChanges();
}

The first line gets the DOM element that just the user just completed dragging. The second line retrieves the in memory JSON object represented by that DOM element. The next couple of lines get the X and Y that the user story card was dragged to relative to their parent DOM element. And then something odd happens.

Sys.Observer.setValue looks so complicated. Why couldn’t we just call selectedUserStoryJsonObject.X = newX?

Well, if we’d done that the DataView wouldn’t know about the change we just made. If we then tried to call dataContext.saveChanges() nothing would get sent back to the server. Sys.Observer.setValue notifies any interested parties, in this case the DataView, that a value has been changed. And this enables the functionality we saw in part two where the DataView only sends the relevant records back to SharePoint instead of the entire set of JSON objects.

And so, with this code in place, we can now move user story cards around on the page and when we refresh the page or come back days later our layout has persisted back into SharePoint.

Conclusion

In this post you’ve seen how to directly modify the in-memory JSON objects and send them back to SharePoint. Our application is finally getting interesting and I find it quite remarkable how little code it’s taken. In the next post I’ll show how to do master-detail scenario’s for editing user story cards.

Sunday, March 21, 2010

Client Side AJAX Applications in SharePoint 2010 - Part 3

Note: This is one of a multi-part series exploring building client side AJAX applications with ASP.Net AJAX 4 Templating and the WCF Data Services (aka ADO.Net Data Services, aka oData, aka Astoria) in SharePoint 2010.

Part 1 – Exploring WCF Data Services in SharePoint 2010
Part 2 – Creating a read-only page with ASP.Net AJAX 4.0 Templates
Part 3 – Writing data back to SharePoint with ASP.Net AJAX 4.0 Templates
Part 4 - jQuery and Manual JSON Object Manipulation

In the previous two posts in this series I’ve shown how you can access SharePoint data using WCF Data Services and consume that data using ASP.Net AJAX 4 templates. In this post I’ll update the previous example in order to show how to write data back to SharePoint.

Templates and Linking

Before we go into writing data back let’s modify our example to link each item back to the SharePoint DispForm.aspx page. A naïve approach would look like this:

<ul id="userStoriesList" class="sys-template">
       <li><a href="../../Lists/User%20Stories/DispForm.aspx?ID={{ID}}">{{ Title }}</a></li>
</
ul>

It turns out this approach doesn’t work because the DataView won’t look for the data binding syntax inside of attributes. The next approach that might make sense would now be to set href to a data binding syntax and inside there do some string concatenation like so:

<li><a href="{{ '../../Lists/User%20Stories/DispForm.aspx?ID=' + ID }}">{{ Title }}</a></li>

It turns out this doesn’t work either, but for a different reason. Apparently the DataView can’t natively convert some HTML attributes like href and src for reasons Bertrand La Roy describes. The solution is to prefix the attribute with the ASP.Net AJAX system namespace. Typically you would define the system namespace on the HTML body element. But with SharePoint you don’t have access to this element. Fortunately you can declare this namespace on any element so the following code works:

<div xmlns:sys="javascript:Sys">
    <
ul id="userStoriesList" class="sys-template">
        <li><a sys:href="{{ '../../Lists/User%20Stories/DispForm.aspx?ID=' + ID }}">{{ Title }}</a></li>
    </
ul>
</
div>

Great. Let’s move on to updating data.

Live Binding Syntax

The {{ }} template syntax we’ve seen so far is what’s known as one time binding. When you use this syntax the expression is evaluated only once when the template is rendered. What we want is one way live binding and/or two way live binding.

In order to understand the live bindings you need to understand that behind the scenes the DataView keeps an in memory copy of all of the JSON objects it downloads. It's possible for the data in these objects to change. And that's where one way live bindings come in. When you use the syntax { binding [FieldName] }, the DataView will automatically update the binding when the underlying JSON object changes.

Two way live binding happens automatically when you use the one way live binding syntax on HTML INPUT elements. For these scenarios the DataView automatically updates the in memory JSON objects. Then the DataView updates any one way live binding's.

So let’s throw two DataViews onto the page, one with one way live bound hyperlinks, one with two way bound HTML INPUT textbox elements and see what happens:

<script type="text/javascript">
    function pageInit() {
        var dataContext = $create(Sys.Data.AdoNetDataContext, { serviceUri: "/demoprep/_vti_bin/ListData.svc" });

        $create(Sys.UI.DataView,
                {
                    autoFetch: true,
                    dataProvider: dataContext,
                    fetchOperation: "UserStories",
                    fetchParameters: { $top: 20 }
                },
                null,
                null,
                $get("userStoriesList")
            );

        $create(Sys.UI.DataView,
                {
                    autoFetch: true,
                    dataProvider: dataContext,
                    fetchOperation: "UserStories",
                    fetchParameters: { $top: 20 }
                },
                null,
                null,
                $get("userStoriesTable")
            );
    }
    Sys.Application.add_init(pageInit);
</script>

<div xmlns:sys="javascript:Sys">
    <ul id="userStoriesList" class="sys-template">
        <li><a sys:href="{{ '../../Lists/User%20Stories/DispForm.aspx?ID=' + ID }}">{ binding Title }</a></li>
    </ul>

    <table>
        <thead>
            <tr>
                <td>Title</td>
            </tr>
        </thead>
        <tbody id="userStoriesTable" class="sys-template">
            <tr>
                <td><input type="text" sys:value="{binding Title}" /></td>
            </tr>
        </tbody>
    </table>
</div>

Notice the input element with the sys:value bound to title and the new binding syntax in the text of the a href. The result looks like this:

It may not be all that pretty yet, but if you change the text in any of the text boxes the result is immediately updated in the hyperlinks on mouse out. But if you hop back to SharePoint or refresh the page you’ll see that our data hasn’t been updated.

Updating SharePoint, For Real This Time

Just because the DataView updated the in memory JSON objects it’s bound to doesn’t mean it sent a POST request back to ListData.svc. To accomplish that we need to call dataContext.saveChanges(). If you toss in a button below the table like this:

<button onclick="dataContext.saveChanges()">Save Changes</button>

And move the dataContext variable to a higher scope like this:

var dataContext;

function onLoad() {
       dataContext = $create(Sys.Data.AdoNetDataContext, { serviceUri: "/demoprep/_vti_bin/ListData.svc" });

Then when you click the button the AdoNetDataContext figures out which JSON objects changed and sends just those objects back to ListData.svc. How do you know it only sends the rows that changed? If you look at Fiddler there will be a POST to ListData.svc/$batch with something like the following:

--batch_d425-6668-6b05
Content-Type: multipart/mixed;boundary=changeset_8a05-bdcd-0b3e
--changeset_8a05-bdcd-0b3e
Content-Type: application/http
Content-Transfer-Encoding: binary
MERGE http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3) HTTP/1.1
If-Match: W/"3"
Host: localhost
Accept: application/json
Accept-Charset: utf-8
Content-Type: application/json;charset=utf-8

{
"__metadata":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)",
"etag":"W/\"3\"",
"type":"Microsoft.SharePoint.DataService.UserStoriesItem"},
"ContentTypeID":"0x01080070E2807D369BD94FBD6C057D3110E6D3",
"Title":"Renamed Task",
"Predecessors":{"__deferred":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)/Predecessors"}},
"Priority":{"__deferred":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)/Priority"}},
"PriorityValue":"(2) Normal",
"Status":{"__deferred":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)/Status"}},
"StatusValue":"Not Started",
"Complete":null,
"AssignedToID":null,
"TaskGroupID":null,
"Description":"<div></div>",
"StartDate":"\/Date(1266624000000)\/",
"DueDate":null,
"Points":8,
"Iteration":{"__deferred":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)/Iteration"}},
"IterationID":1,
"ID":3,
"ContentType":"Task",
"Modified":"\/Date(1269128451000)\/",
"Created":"\/Date(1266679964000)\/",
"CreatedByID":1,
"ModifiedByID":1,
"Owshiddenversion":3,
"Version":"3.0",
"Attachments":{"__deferred":{"uri":"http://localhost/demoprep/_vti_bin/ListData.svc/UserStories(3)/Attachments"}},
"Path":"/demoprep/Lists/User Stories"
}
--changeset_8a05-bdcd-0b3e--
--batch_d425-6668-6b05—

And that, as you may have noticed, is the JSON for just a single of the rows that we were displaying.

But how did the AdoNetDataContext know which objects were changed? Jonathan Carter explains this better than I could but basically when the data context downloads JSON objects it injects them with the ability to notify itself when they change. As Jonathan points out it’s a neat trick that would not be possible in a statically typed language.

Summary

We’ve finally gotten to some of the core benefits of using ASP.Net AJAX 4 Templating. With very little JavaScript we were able to cleanly separate our UI from our data access code and write data back to the server. And the amount of HTTP traffic and server operations was kept to a bare minimum, keeping things as fast and responsive as possible. So all we’re missing at this point is how to use it to make something real out of these technologies. And that will be the topic for the next post.